### Thursday, September 23, 2004

## It Turns Out This is Not a Job for EconoPundit

There's this paper that's kind of making its way through the blogosphere, which challenges the idea that prediction markets have futures contracts representing subjective probabilities. The paper is by Charles Manski, a very respected econometrician. I started reading his book, Identification Problems in the Social Sciences a month or two ago.

In this context, "identified" means "you can know what it is." Manski's book is all about what you can reasonably know in statistical analysis without any assumptions, and what power you get out of imposing assumptions. For example, if you send out a survey to 100 randomly selected people, and 65 fill it out and give you a response, then the underlying distribution of the entire sample is not identified, since you only have information on the subset of the sample that replied. Manski points out that even in this situation, you can put bounds on the underlying distribution with the censored sample, but that only narrows the range of possible values. If you are willing to impose the assumption that whether someone responds to the survey is independent of what they reply, then the full sample's distribution is identified. That might or might not be a reasonable assumption, though (it probably isn't).

In any case, I was surprised to learn that there's very little research about what the prices of futures contracts actually mean, given how ingrained the conventional wisdom is. Manski has come along and with this paper pointed out that with minimal assumptions, the price of a prediction market contract doesn't actually correspond to the market's mean subjective belief (ie, the average of what everyone in the market believes the probability of the event to be). In fact, he derives the bounds on what the mean subjective belief can be for a given price, and finds that except in certain cases (for example, absolute unanimity), these bounds are wide enough to be practically useless. Furthermore, the true value can be anywhere within these bounds, without the prices giving you any clue about where. This post at Dead Parrots does a great job explaining it in detail without math.

Since I don't think I can improve on the explanation at Dead Parrots, I'm really just making this post to point out that EconoPundit is a fool. Yesterday, he posted this, promising to update with an explanation of the article. I predicted to Claire yesterday that he would in fact not understand the article well enough to explain it, and pull his post, as he has sometimes done in the past. Sure enough, this morning, he has updated with some really, really lame explanation about how some people who read his site every day are friends of Ronald Coase, and he doesn't want to get it wrong. Please.

I've pointed out in the past that EconoPundit doesn't seem to understand simple econometrics very well, and that he likes to massage data to make it show what he wants it to. The paper is not that hard to understand if you've got some experience in statistics, and it should be a piece of cake for someone who works as an economics professor. The way Manski does things, working with minimal assumptions and trying to put bounds on things, is way easier than the statistics typical of econometrics. EconoPundit likes to point to the TradeSports presidential prediction contracts, so we'll see if he stops doing that now.

In this context, "identified" means "you can know what it is." Manski's book is all about what you can reasonably know in statistical analysis without any assumptions, and what power you get out of imposing assumptions. For example, if you send out a survey to 100 randomly selected people, and 65 fill it out and give you a response, then the underlying distribution of the entire sample is not identified, since you only have information on the subset of the sample that replied. Manski points out that even in this situation, you can put bounds on the underlying distribution with the censored sample, but that only narrows the range of possible values. If you are willing to impose the assumption that whether someone responds to the survey is independent of what they reply, then the full sample's distribution is identified. That might or might not be a reasonable assumption, though (it probably isn't).

In any case, I was surprised to learn that there's very little research about what the prices of futures contracts actually mean, given how ingrained the conventional wisdom is. Manski has come along and with this paper pointed out that with minimal assumptions, the price of a prediction market contract doesn't actually correspond to the market's mean subjective belief (ie, the average of what everyone in the market believes the probability of the event to be). In fact, he derives the bounds on what the mean subjective belief can be for a given price, and finds that except in certain cases (for example, absolute unanimity), these bounds are wide enough to be practically useless. Furthermore, the true value can be anywhere within these bounds, without the prices giving you any clue about where. This post at Dead Parrots does a great job explaining it in detail without math.

Since I don't think I can improve on the explanation at Dead Parrots, I'm really just making this post to point out that EconoPundit is a fool. Yesterday, he posted this, promising to update with an explanation of the article. I predicted to Claire yesterday that he would in fact not understand the article well enough to explain it, and pull his post, as he has sometimes done in the past. Sure enough, this morning, he has updated with some really, really lame explanation about how some people who read his site every day are friends of Ronald Coase, and he doesn't want to get it wrong. Please.

I've pointed out in the past that EconoPundit doesn't seem to understand simple econometrics very well, and that he likes to massage data to make it show what he wants it to. The paper is not that hard to understand if you've got some experience in statistics, and it should be a piece of cake for someone who works as an economics professor. The way Manski does things, working with minimal assumptions and trying to put bounds on things, is way easier than the statistics typical of econometrics. EconoPundit likes to point to the TradeSports presidential prediction contracts, so we'll see if he stops doing that now.

Comments:

Very interesting. I was wondering last week if the polls said 70% of the voters would likely vote for Bush if that actually means Bush has a 70% chance of winning.

Now I'm going to have to update my own blog, cause I've mentioned the futures market twice! This paper asside, don't future's markets have a good record of being accurate? What does this paper say about that?

Post a Comment
Now I'm going to have to update my own blog, cause I've mentioned the futures market twice! This paper asside, don't future's markets have a good record of being accurate? What does this paper say about that?