An economics, investment, trading and policy blog with a focus on Modern Monetary Theory (MMT). We seek the truth, avoid the mainstream and are virulently anti-neoliberalism.
Pages
▼
Pages
▼
Friday, November 2, 2012
Justin Fox — What Does Nate Silver's '80.9% Chance of Winning' Mean, Anyway?
Justin Fox explains how the probability scoring works. It's not what many think.
Thanks. That is what I thought Silver's prediction meant. Although it was a long way to explain it.
Instead it could have just stated: It model of various variables that is run through a Monte Carlo simulation (with link to what that is.) Then ended with: basically, Silver's updated model is run over and over and over and X% of the time candidate A wins in those simulations. That is what X represents.
What I am wondering is what Nate Silver's model is testing. I assume polling error because that is the probability variable not the polls' results, right?
What I mean, if most polls have Ohio at +2% Obama. Say 51% Obama and 49%Romney, that does not mean Obama has a 51% chance of winning - he has a 100% chance of winning of the poll is right in its average. So the chance of him winning or losing based on a 51% to 49% poll is strictly based on the poll's error in his favor being more than 1% from the actual vote.
So Nate Silver's mode, at its heart, is really predicting that error?
Adam2, I do not think many people can understand it (perhaps the majority of people cannot.)
From casual discussion of probabilities (usual centered around weather forecasts), I get the impression that in a situation of 2 outcomes, many people can only visualize 3 states. (1) certainty that an event will occur - it will rain. (2) certainty that an event will not occur - it will not rain. (3) fifty-fifty an event either will or will not occur - it might or might not rain. Anything else, such as a forecast of 30% rain, makes no sense to them.
I suspect many people are that way for two reasons: 1) most people are not comfortable and/or good with math, especially fractions and 2) many people do not think abstractly, so a hypothetical situation of a series of events where X happens 800 out of 1000 times, makes no sense to them in generalizing what a 80% probability for a real event that will only happen once ("800 out of 1000 times, what do you mean dude, the election happens only one time" is likely what the little voice inside their heads is saying, when they hear that explanation.)
Tom Hickey, you are an educator, what is your take on people understanding probabilities?
To find out how even people with an education, that should lend itself to understanding statistics, are generally weaker, than you would think, in applying probabilities to real life events, get into the following discussion with an accountant, computer programmer, engineer, etc.
Ask if 1,2,3,4,5,6 in a lottery with 6 numbers has as good a chance of winning as other 6 number combinations.
Tom Hickey, you are an educator, what is your take on people understanding probabilities?
I would say that most people who have not studied probability would react intuitively, and so would a good number that have studied probability. Even a quite a few that are well-versed in probability would also be affected in an emotionally charged situation like an election
There is a lot of ambiguity involved in intuition, which is why we say, "Do the math." Intuition is highly susceptible to cognitive bias and emotional coloring.
Math can assist us in identifying and dealing with that, but it can't make it go away. In critical thinking we try to alert students to the traps they are likely to fall into and also the games of self-sabotage they play due to their biases and over-reliance on intuition. Usually we rely on intuition since it is not efficient to use rigorous thinking on most decisions.
The trick is knowing when to use intuition and when rigorous thinking is appropriate. For most people, figuring elections is not that crucial to them, but bettors in the prediction markets use rigorous thinking since they have money on the line. So relatively small sample in a prediction market can turn out to be pretty accurate.
What this means practically speaking is that in the case of the election, the present 70-30 odds in the prediction market would be interpreted differently by Obama and Romney supports that are not using rigorous thinking, which is most of them.
Obama supports would tend to be over-confident of Obama's winning, translating it into a sure bet, whereas Romney supporters might see it as still a good chance of their candidate pulling an upset, which would make that a wise bet indeed at those odds.
What many people don't get is that the important thing to be looking at is the direction and momentum of change, just as they don't get that it's not raw poll numbers that are meaningful, but rather numbers plus intensity. In talking to friends, many just don't know what to be looking for. So they get sidetracked by insignificance and are confused by their own biases.
However, campaign strategists understand all this very well and use it to target resources right down to the final hours. But they use their own pollsters and tightly target voters down to counties and voting districts still in play.
With the prediction markets breaking for Obama today, I suspect that the game is over for Romney. The evidence is mounting that he just couldn't get the momentum going that he needed.
But, as they say, it ain't over until the fat lady sings. And she's warming up right now.
Considering that Romney's chance is hinged on systematic errors in most polls, I am wondering how the make-up of polls is formed. I spent the morning reading some conservative pundits calling the election for Romney. Pretty much all are based on polls being wrong around the sampling of Republicans and Democrats in the polls. For that to be right two things must occur: 1) the make-up of Republicans and Democrats and the polls must be a premise set by the pollsters and 2) the pollsters must be wrong, in the direction of Obama bias, in that premise.
Here is what I can't find out: Is the first one correct - do pollsters set up the polling sample to get a fixed percentage of Republicans and Democrats (i.e. if they get more Republicans than their premise, they throw out the extra samples)? Or do they do random samples, and whatever percentage of the sample indentifies Republican and Democrats is what it is (no adjustments.) I have looked at some polls' data and cannot find the answer. But I think I recall that most are simply what the sample is (no adjustments) or the adjustments are based on other polls specifically looking at what percentage are what (therefore not just an assumption on the part of the pollster but one based on what polls are showing now.) In both cases, the conservative claim is hollow but again I can not confirm which it is (adjusted on pure historical assumption or is what the sample is or what polls are showing now.)
Thanks I read it but still no answer to the question about how the make-up is sampled (just how the sample is or if unrealistic assumptions are made to alter the # of Republicans and Democrats in each sample), which conservatives claim is the systemic error in question.
I think the samples either are random samples or are focused based on how polls show the electorate likely is but I cannot find anyplace to confirm that. (In other words, are the conservatives right or just blowing smoke?)
I believe that it is based on previous trends with most weight on the past election. So the conservative claim is that Obama's chances are too heavily wighted by amount of support he received last time, which has diminished this time around. Unless this is affecting all the polling, it's wishful thinking.
More significantly, the prediction markets have broken in favor of Obama and the prediction markets get more accurate closer to the election.
So while it's not over until it's over, the weight of the polling and prediction results favor Obama over Romney and pretty strongly.
It looks like a big stretch to say that the election is a toss-up at this point, which is what a lot of the rightward pundits claim if they don't try to spin a Romney win based on invisibles.
Obama supports would tend to be over-confident of Obama's winning, translating it into a sure bet, whereas Romney supporters might see it as still a good chance of their candidate pulling an upset, which would make that a wise bet indeed at those odds.
Both are the type of person I'd love to invite over for poker.
Someone just replied, on another board, that race and other demographics are adjusted in polls but party affiliation tends to not be.
That makes sense since minorities make up smaller parts of the whole, a random sample might miss them by 50% or more (for example, only be 5% of sample when reality is more that the electorate will be 8% for that race) but with party affiliation makes up a much larger part of the electorate (30-35%) a large sample should collect it more appropriately.
That's what the prediction markets are for, Matt. One weighs the evidence as best one can and puts one's money down. The prediction markets are breaking for Obama now, so it's a great opportunity for people that are pretty sure Romney will win to make some easy money.
Intrade isn't as US friendly as it was last time I bet on Bush. Tried several weeks ago and they ask for account info etc rather than just a credit card.pita. any critiques of this article being obtuse....doubt it. There are plenty on the right who are logical as well. I'm working off a 7in tablet on vacation so I can't argue more with you guys! Too slow.
any critiques of this article being obtuse....doubt it.
The reasoning seems OK to me. But I would not be betting on it given the direction of the prediction markets. They tend to be pretty accurate at this stage of the game.
Doesn't mean that Mitt's not going to win, of course. Just means that a bet on Obama is expensive and a bet on Mitt is risky right now.
I am generally a fan of intrade but it really seems to be a reflection of the news. didn't it spike in favor of kerry on actual election afternoon to over 85%? Doesn't get any later or more wrong than that Tom. Based solely on bad exit polls I might add. Same story here imho.
Thanks. That is what I thought Silver's prediction meant. Although it was a long way to explain it.
ReplyDeleteInstead it could have just stated: It model of various variables that is run through a Monte Carlo simulation (with link to what that is.) Then ended with: basically, Silver's updated model is run over and over and over and X% of the time candidate A wins in those simulations. That is what X represents.
What I am wondering is what Nate Silver's model is testing. I assume polling error because that is the probability variable not the polls' results, right?
ReplyDeleteWhat I mean, if most polls have Ohio at +2% Obama. Say 51% Obama and 49%Romney, that does not mean Obama has a 51% chance of winning - he has a 100% chance of winning of the poll is right in its average. So the chance of him winning or losing based on a 51% to 49% poll is strictly based on the poll's error in his favor being more than 1% from the actual vote.
So Nate Silver's mode, at its heart, is really predicting that error?
Really? we have to explain this?
ReplyDeleteIt means 8 out of 10 times we see polls like this Obama would be the winner.
Adam2, I do not think many people can understand it (perhaps the majority of people cannot.)
ReplyDeleteFrom casual discussion of probabilities (usual centered around weather forecasts), I get the impression that in a situation of 2 outcomes, many people can only visualize 3 states. (1) certainty that an event will occur - it will rain. (2) certainty that an event will not occur - it will not rain. (3) fifty-fifty an event either will or will not occur - it might or might not rain. Anything else, such as a forecast of 30% rain, makes no sense to them.
I suspect many people are that way for two reasons: 1) most people are not comfortable and/or good with math, especially fractions and 2) many people do not think abstractly, so a hypothetical situation of a series of events where X happens 800 out of 1000 times, makes no sense to them in generalizing what a 80% probability for a real event that will only happen once ("800 out of 1000 times, what do you mean dude, the election happens only one time" is likely what the little voice inside their heads is saying, when they hear that explanation.)
Tom Hickey, you are an educator, what is your take on people understanding probabilities?
To find out how even people with an education, that should lend itself to understanding statistics, are generally weaker, than you would think, in applying probabilities to real life events, get into the following discussion with an accountant, computer programmer, engineer, etc.
ReplyDeleteAsk if 1,2,3,4,5,6 in a lottery with 6 numbers has as good a chance of winning as other 6 number combinations.
Tom Hickey, you are an educator, what is your take on people understanding probabilities?
ReplyDeleteI would say that most people who have not studied probability would react intuitively, and so would a good number that have studied probability. Even a quite a few that are well-versed in probability would also be affected in an emotionally charged situation like an election
There is a lot of ambiguity involved in intuition, which is why we say, "Do the math." Intuition is highly susceptible to cognitive bias and emotional coloring.
Math can assist us in identifying and dealing with that, but it can't make it go away. In critical thinking we try to alert students to the traps they are likely to fall into and also the games of self-sabotage they play due to their biases and over-reliance on intuition. Usually we rely on intuition since it is not efficient to use rigorous thinking on most decisions.
The trick is knowing when to use intuition and when rigorous thinking is appropriate. For most people, figuring elections is not that crucial to them, but bettors in the prediction markets use rigorous thinking since they have money on the line. So relatively small sample in a prediction market can turn out to be pretty accurate.
What this means practically speaking is that in the case of the election, the present 70-30 odds in the prediction market would be interpreted differently by Obama and Romney supports that are not using rigorous thinking, which is most of them.
Obama supports would tend to be over-confident of Obama's winning, translating it into a sure bet, whereas Romney supporters might see it as still a good chance of their candidate pulling an upset, which would make that a wise bet indeed at those odds.
What many people don't get is that the important thing to be looking at is the direction and momentum of change, just as they don't get that it's not raw poll numbers that are meaningful, but rather numbers plus intensity. In talking to friends, many just don't know what to be looking for. So they get sidetracked by insignificance and are confused by their own biases.
However, campaign strategists understand all this very well and use it to target resources right down to the final hours. But they use their own pollsters and tightly target voters down to counties and voting districts still in play.
With the prediction markets breaking for Obama today, I suspect that the game is over for Romney. The evidence is mounting that he just couldn't get the momentum going that he needed.
But, as they say, it ain't over until the fat lady sings. And she's warming up right now.
Thanks. I guess I naturally see the logic, and have trouble seeing the "emotion" which causes social misunderstanding a lot of times.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteOnline sites are hitting at about 70%. But it seems the pay-out, at casinos, is implying even greater odds:
ReplyDeletehttp://www.oddschecker.com/specials/politics-and-election/us-presidential-election/winner
President Obama bets are flirting with 1 to 5+ payout. Is that implying a 83.3%+ odd?
Considering that Romney's chance is hinged on systematic errors in most polls, I am wondering how the make-up of polls is formed. I spent the morning reading some conservative pundits calling the election for Romney. Pretty much all are based on polls being wrong around the sampling of Republicans and Democrats in the polls. For that to be right two things must occur: 1) the make-up of Republicans and Democrats and the polls must be a premise set by the pollsters and 2) the pollsters must be wrong, in the direction of Obama bias, in that premise.
ReplyDeleteHere is what I can't find out: Is the first one correct - do pollsters set up the polling sample to get a fixed percentage of Republicans and Democrats (i.e. if they get more Republicans than their premise, they throw out the extra samples)? Or do they do random samples, and whatever percentage of the sample indentifies Republican and Democrats is what it is (no adjustments.) I have looked at some polls' data and cannot find the answer. But I think I recall that most are simply what the sample is (no adjustments) or the adjustments are based on other polls specifically looking at what percentage are what (therefore not just an assumption on the part of the pollster but one based on what polls are showing now.) In both cases, the conservative claim is hollow but again I can not confirm which it is (adjusted on pure historical assumption or is what the sample is or what polls are showing now.)
Crake, take a look at Nate Silver's post today
ReplyDeleteFor Romney to Win, State Polls Must Be Statistically Biased
Tom Hickey,
ReplyDeleteThanks I read it but still no answer to the question about how the make-up is sampled (just how the sample is or if unrealistic assumptions are made to alter the # of Republicans and Democrats in each sample), which conservatives claim is the systemic error in question.
I think the samples either are random samples or are focused based on how polls show the electorate likely is but I cannot find anyplace to confirm that. (In other words, are the conservatives right or just blowing smoke?)
Do you know?
I believe that it is based on previous trends with most weight on the past election. So the conservative claim is that Obama's chances are too heavily wighted by amount of support he received last time, which has diminished this time around. Unless this is affecting all the polling, it's wishful thinking.
ReplyDeleteMore significantly, the prediction markets have broken in favor of Obama and the prediction markets get more accurate closer to the election.
So while it's not over until it's over, the weight of the polling and prediction results favor Obama over Romney and pretty strongly.
It looks like a big stretch to say that the election is a toss-up at this point, which is what a lot of the rightward pundits claim if they don't try to spin a Romney win based on invisibles.
Obama supports would tend to be over-confident of Obama's winning, translating it into a sure bet, whereas Romney supporters might see it as still a good chance of their candidate pulling an upset, which would make that a wise bet indeed at those odds.
ReplyDeleteBoth are the type of person I'd love to invite over for poker.
Someone just replied, on another board, that race and other demographics are adjusted in polls but party affiliation tends to not be.
ReplyDeleteThat makes sense since minorities make up smaller parts of the whole, a random sample might miss them by 50% or more (for example, only be 5% of sample when reality is more that the electorate will be 8% for that race) but with party affiliation makes up a much larger part of the electorate (30-35%) a large sample should collect it more appropriately.
Anyone know if that is true?
Could Gallup be predicting a blow-out but not the way you think it would:
ReplyDeletehttp://www.wbur.org/npr/164242319/for-whom-will-you-vote-may-be-wrong-question
Crake, see also David Rothschild, YouGov/Xbox Poll Respondents Expect Obama Victory in Ohio (and the election)
ReplyDeleteThis is the best summary of GOP position and one I agree with.
ReplyDeletehttp://www.redstate.com/2012/10/31/on-polling-models-skewed-unskewed/
That's what the prediction markets are for, Matt. One weighs the evidence as best one can and puts one's money down. The prediction markets are breaking for Obama now, so it's a great opportunity for people that are pretty sure Romney will win to make some easy money.
ReplyDeleteIntrade isn't as US friendly as it was last time I bet on Bush. Tried several weeks ago and they ask for account info etc rather than just a credit card.pita. any critiques of this article being obtuse....doubt it. There are plenty on the right who are logical as well. I'm working off a 7in tablet on vacation so I can't argue more with you guys! Too slow.
ReplyDeleteany critiques of this article being obtuse....doubt it.
ReplyDeleteThe reasoning seems OK to me. But I would not be betting on it given the direction of the prediction markets. They tend to be pretty accurate at this stage of the game.
Doesn't mean that Mitt's not going to win, of course. Just means that a bet on Obama is expensive and a bet on Mitt is risky right now.
I am generally a fan of intrade but it really seems to be a reflection of the news. didn't it spike in favor of kerry on actual election afternoon to over 85%? Doesn't get any later or more wrong than that Tom. Based solely on bad exit polls I might add. Same story here imho.
ReplyDeleteState polls and intrade right. Matt wrong. Hate it when that happens. Amazing dem turnout.
ReplyDelete