October 08, 2003

Markets as Information Sources

Hal Varian writes:

The Iowa Electronic Markets won again with an extremely accurate prediction 5 days before the election:

Actual Iowa Electronic Markets
for recall 54.3%53.5%
against recall45.7%44.5%
Schwarzenegger46.8%47.0%
Bustamante32.4%30.1%

Source:

http://apnews.myway.com//article/20031008/D7U20PN80.html
http://128.255.244.60/quotes/69.html
http://www.biz.uiowa.edu/iem/markets/Recall.html

Posted by DeLong at October 8, 2003 08:32 PM | TrackBack

Comments

Was it accurate because it was a market, or was it accurate because it was a poll?

Did it predict Krugman's big day today as accurately as last night's rumor?

How did it do on the past ten recessions?

Posted by: jerry on October 9, 2003 12:23 AM

Congratulations!!!..I suppose...;))

Posted by: Lucy on October 9, 2003 03:47 AM

And do you know that Arnie has won in California?:) Khe-he? that's cool

Posted by: Michael on October 9, 2003 03:55 AM

Finding an accurate predictor after the fact is much easier than finding an accurate predictor before the fact.

Posted by: richard on October 9, 2003 04:01 AM

I guess John Poindexter was right and Hilary was wrong.

Posted by: Keith on October 9, 2003 04:42 AM

If "after the fact" means that we could only in retrospect have known the IEM would be accurate, that's just wrong. The market has offered up better-than-poll predictions for fifteen years now. Of course, that still doesn't mean I'm used to it. I still find it eerie that this small, relatively random group of traders should be able to forecast so well.

Posted by: James Surowiecki on October 9, 2003 05:21 AM

> I still find it eerie that this small, relatively random
> group of traders should be able to forecast so well.

Note, however, that your comparison between markets and polls isn't quite fair. In determining prices, the speculators on the political futures market made use of all available poll data, but the polls had no way of using any of the market data. So it would be surprising if the market's prediction _wasn't_ better than the polls'. The futures market by itself would probably do a lot worse than polling by itself.

Posted by: Thomas Blankenhorn on October 9, 2003 05:32 AM

Nobody tell Gregg Easterbrook! Such gambling is wrong! (Men over 55 leering at 20 year olds is A-Ok!)

About 3/4ths of the way down the page headline about DARPA:

http://msn.espn.go.com/page2/s/tmq/031007.html

Posted by: Rob on October 9, 2003 05:41 AM

Thomas --

It's true that the IEM gets to use the poll data, and that that's undoubtedly essential to its success. But I'm not sure that it follows that it's predictable that the market would do better. After all, a poll in theory is asking the very people who are going to vote how they are going to vote. The IEM reflects a small crowd's interpretation of how those people are going to vote. To me, it seems strange that the second should be more accurate -- that is, provide a better sense of what people are going to do than the people themselves can -- than the first.

Posted by: James Surowiecki on October 9, 2003 06:03 AM

> But I'm not sure that it follows that it's
> predictable that the market would do better.

Consider a political futures market consisting of just a single investor and the bank. The investor knows where to get and how to average poll data, but is totally ignorant of politics otherwise. To maximize his profit, he buys from the bank whenever it sells at a lower price than predicted by the forecast, and sells to the bank whenever its price is higher.

My point is that to match the predictive performance of polls, all the political futures market needs is a one-man 'crowd' that slavishly mirrors poll results. To do worse than polls, the market would have to _destroy_ information on net somehow; to beat polls, its speculators only need to know _something_ that the polls don't -- which isn't that hard I think.

If you believe the market creates any information at all, it follows that the market must beat polls in predicting political outcomes.

Posted by: Thomas Blankenhorn on October 9, 2003 07:23 AM

James, there are many predictors out there. Searching through all past predictors and choosing one that has done well is not the best methodology, IMO, for choosing a good predictor going forward. Compare to the statement that for the past 15 years butter production in Bangladesh has been a good predictor of the S&P 500. I realize this is a bit far from a perfect analogy, but illustrates the concern.

If Brad had posted about IEM before the election, I'd feel better about IEM.

Posted by: richard on October 9, 2003 07:25 AM

As far as as the Iowa Market tending to predict better than the polls goes, what I would like to know is whether it predicts the market better than the optimal combination of the poll data. So people have pointed out that given N polls of similar structure asking the same multinomial question, you could do better by just averaging together the N polls. As it turns out, you can do better than that by weighting the result of poll i by something like 1/s[i]^2 and then normalizing the result appropriately. (s[i]^2 is the variance of poll i from the truth, which you could estimate somehow from previous performance or some other devious means.) It might not make much of a difference here, but I don't know since I haven't tried it. But if you wanted to know how much error reduction you were getting from non-polling data, I think you should do this calculation first.

Posted by: Jonathan King on October 9, 2003 07:50 AM

OK, so looking at the raw data in Iowa, I note that there
appear to have been some occasions, particularly in early September, when there were some genuine arbitage possibilities:

http://128.255.244.60/graphs/Recall_2.gif

Note that at some time points, Arnold is projected to receive the largest vote share, but Bustamante is projected to win the election. Obviously, a gross inefficiency. Either this would go away with more market participants, or something screwy is going on here.

Posted by: Jonathan King on October 9, 2003 07:59 AM

I really don't get the Bangladesh butter analogy. Here we have a market whose function is to predict election results, and which has done a good job of predicting election results for fifteen years. Asked to predict the California recall, it again did a very good job. What would make us think that there's some spurious correlation here?

Posted by: James Surowiecki on October 9, 2003 08:39 AM

If what we're trying to establish is that IEM did a good job of predicting, no problem - they seem ot have done a good job here. If we're trying to say they have a good methodology for predicting and should succeed in the future, then mining past data to find a winning methodology is not as good as first forming a hypothesis (market based predictors do a better job than normal polling, for example) and testing it against future data.

Posted by: richard on October 9, 2003 08:51 AM

Skeptics should relax... this is interesting stuff. At the very least, it isn't just ONE prediction at stake here but FOUR somewhat related ones, which puts a relatively higher burden on accusations about selecting methodologies by hindsight.

That being said, pegging the actual results within about 1% of their final value doesn't seem an outsanding accomplishment to me. Perhaps the system is to be preferred given the expense of traditional polling methods.

Posted by: trevelyan on October 9, 2003 11:55 AM

"...not as good as first forming a hypothesis (market based predictors do a better job than normal polling, for example) and testing it against future data."

Um, richard, I think that is what the IEM *IS*. Unlike, say, Tradesports, which is a for-profit business that deals mostly in sports betting, IEM is run by accademics and focused solely on politics. If, it is not intended as a test of the hypothesis "market based predictors do a better job than normal polling", then why are they doing it?

BTW, anyone here know how accurate Tradesports was?

Posted by: Decnavda on October 9, 2003 01:00 PM

IEM v Tradesports: IEM ran both a vote share market and a winner-take-all market. Both IEM and Tradesports forecast a Swartzenegger victory in the winner-take-all market, but that isn't nearly as impressive as getting the vote share nailed.

I agree that one election does not make a winner, but as several people have pointed out, IEM has typically beat the polls for 15 years in a row. However, the reason why is a bit subtle. See
http://www.sims.berkeley.edu/~hal/people/hal/NYTimes/2000-06-29.html
for a 1000 word summary, or the academic papers at the IEM website for more detailed information.

One also has to be careful about polls. In many European countries polls are real polls---they are required to report the actual responses. In the US political polls serve as inputs to a political prediction model, where the responses are reweighted to take account of likelihood of voting, Electoral College issues, and so on. Normally such polls say something about "forecasting the likely vote share/winner" when they issue their results.

Posted by: Hal Varian on October 14, 2003 02:00 PM
Post a comment