Brad DeLong's Semi-Daily Journal: A Weblog

".. and that the press corps has not called them on this."

I think you meant the stenographer corps...

And, given the brilliant performance by the press collectively over the last 8 years or so, you would be surprised exactly why? I think the surprise would be if they HAD done the work to analyze the problems in the poll modelling and sampling.

".. and that the press corps has not called them on this."

I think you meant the stenographer corps...

I have to wonder if the fix isn't somehow in for Florida this year, given the Republican's use of Gallup's numbers. Man, I hate having these sort of nutbar conspiracy thoughts... :-p

What's the incentive. There is obviously no financial competitive pressure to be the most accurate pollster on the block. If that was the case the Gallup would certainly not be considered a top-tier presidential polling firm based on their results from 2000.

If anything it seems that the wilder the fluctuations you have the more press you get (and therefore more prestige). To put it mildly this is a dreadful incentive.

I recall something about the head of the Gallup poll seeing his job with the polls as something of a "Christian duty," or some similar fundie evangelical cant like that. One might be pardoned for taking this as a confession that scientific objectivity and credibility has been flung out the window for partisan reasons. It's too bad, as Gallup made its mark back in 1936 for its superb poll work.

Fortunately, this is why we have multiple polling services. If one poll is way off, and consistently so, then the proper response is to reject that poll as a valid indicator of reality. But as others have suggested, the steno pool isn't part of the reality-based community.

Ok, this is off topic. It is, nonetheless, of interest.

Has anyone else noticed The Economist has chickened out of a presidential endorsement?

Will they do something for the Oct 31st issue?

I figure they're terrified of losing 1/2 their subscribers. If they do Bush, I plan to mail them the cover along with my cancellation notice.

Gallup is not the only oddball national poll. Here's an estimate of the size and direction of poll bias:
http://anonymous.coward.free.fr/polls/pollbias.html

Did the Economist endorse a candidate four years ago? I can't figure why American newspapers endorse presidential candidates, let alone British weeklies.

Tradesports has Bush set to win Florida -- about 61% chance of him doing so. The market must be figuring on Bush improving in Florida in the next week. Or, the market traders are putting too much weight on the Gallup polls relative to other ones, in which case there's a profit opportunity for those of you who think Kerry will take Florida.

prof, after all you've written about the kind of media that we have, you're flabbergasted by the fetishizing of poll results by the media in lieu of examining them? surely you jest....

Keeping a scatter chart of the Flordia pool results for several months, I find a mild movement towards Bush. The same for Ohio. Bush must win in Ohio and Florida to be elected.

I believe it was the Gallup that showed the President winning the final debate by a margin of eight percentage points. Of course, they also included eight percent more republicans.

At this point, the gallup has been showing too dramatic shifts to be considered trustworthy. I'd simply toss them out to get a more accurate view.

John Faughnan: "I figure they're terrified of losing 1/2 their subscribers. If they do Bush, I plan to mail them the cover along with my cancellation notice."

Of course, they might just chicken out. In which case you should cancel.

Excuse me. You're "flabbergasted" that the press corps hasn't called them on this? How much of the press corps do you think can even parse the phrase, "validation work to identify where their model-based disagreements lie"?

I think Gryn is largely right. First you need to recognize that there are different reasons one would want to hire a pollster, and the pollsters have speciated accordingly. A "new" organization wants a poll that gives them an interesting narrative. Gallup is good at doing this. Accuracy is not a strong concern. Campaign HQs actually want to know what is going on. They hire a different kind of pollster, such as GH&Y.

The copy editors at the Times cannot spell, and you expect that the reporters will be able to mount a critique of the polling organizatons' statistical methods? Oh, I see. It was a rhetorical flourish. Never mind.

Another interesting feature of the latest national Gallup poll: it gives Bush a 5-point lead nationwide among likely voters, but only a 1-point lead among the "battleground" states ( http://www.gallup.com/poll/content/?ci=13792 ). This fits perfectly with the pattern shown by literally all pollsters so far, and it makes it increasingly likely that if the the Electoral College malfunctions again this year Kerry will be the benefactor. In which case we might finally get a bipartisan consensus to get rid of the damn thing.

Trotsky, the Economist did endorse Bush in 2000. In 2001 they had a cover article predicting oil to drop below $10/brl for eternity. Not a great track record.

I'm seeing a lot about how Kerry is doing better in the swing states than nationally.

Are the swing states naturally biased towards Kerry? How did the swing states vote in 2000?

I wouldn't be shocked to learn that they favored Gore by 5 or 6 pts overall.

Anyway, this is the last election where the daily media researches stories. From now on, they will report what a few big organizations research and they will report what the blogs are talking about.

Once the blogosphere finds some statistician who will explain exactly which polls are accurate and why, that will become the conventional wisdom and the press will report it.

What would it cost to get a pre-election poll that statisticians really reasonably expect to be within a few points of the actual vote total?

The press won't know the answer until after we do.

Once the blogosphere finds some statistician who will explain exactly which polls are accurate and why, that will become the conventional wisdom and the press will report it.

That gave me a chuckle.

The problem is, of course, that we have neither the raw data nor any real explanation of the model(s) used by the pollster. Thus, the best anyone can do is semi-educated guessing.

Too bad.

Jeff,
One polling expert is Ruy Teixeira, who has his own polling blog somewhere (called "Talking Donkey" or something like that). He regularly has an article on the Center for American Progress site where he attempts to decipher why the polls wind up the way they do. Good luck!

Flabbergasted? Hmmm, let me think here.

Yep, that's just what we need America. TIME TO HEAD FOR THE GYM. HAHAHA

A dear friend of mine once wrote for the editorial page of a substantial Northern New Jersey daily paper. He's a good writer, and pretty astute, but he is a math-phobe. If anything beyond addition and subtratction is required, he turns it over to someone else. He's not dumb, and he's a fine, moral and ethical person. he's just not numerate.

I wonder how typical he is of the nation's press corps? Might go a long way towards explaining why their collective eyes glaze over when the numbers are rolled out.

I certainly wouldn't rely on him to analyze sampling methods of any polling organization. Yet, I'd trust him with just about any qualitative or ethical issue.

If pollsters had to defend the assumptions built into their models, polling would come to an end. Response rates are
currently 20%, so the whole idea of "random" polling rests
on the spectacular assumption that the missing 80% are
just like the 20% except they didn't happen to pick up the phone.

Needless to say, this assumption has mostly not been proved.
All the different models are ultimately ways of dealing with this problem, but you can't create data where none exists.

According to the Democratic war book the Republicans filched last week, the Kerry campaign is banking on a 400,000 vote majority in Florida. Let's cut that by half and it still puts Gallup pretty deep in the hole. If the Dems have reason to think they can manage that kind of vote, it is going to take a lot of alleged felons to put Mr. B. over the top.

Matt, are you implying that people who pick up the phone because either they still don't have call waiting, or because they have call waiting but are likely to take calls from random people, are more likely to also be the same people that nod their heads in agreement when Bush says we're safer now?

If you don't, I will. The other 80% will speak next Tuesday (I only have a cell phone). :)

I just today looked at voter registration in Florida.
I don't have my notes to hand, but, off the top of my head:
total registration is up ~17%;
in 2000 - when Gore narrowly won - there were roughly 360,000 more Rs than Ds; today, there are roughly 360,000 more Ds than Rs.

To swing statewide registrations so far, the 1.5 million new registrations must be running something like 5:1 Dem:Rep.

So, yeah, even if Bush weren't a dismal failure, just on the demographics I can believe the Kerry campaign's projection that they're going to take Florida by 400,000.

And a 400,000 vote margin is going to be very hard for Jeb to make disappear.

Oh, and my memory of Gallup's polling:

at this point four years ago, Gallup had
the country at something like "Bush 52%, Gore 40%."

Of course, it went Gore 48.5%, Bush 48%.

Four years ago, Gallup was a good 12 points off, in the R direction.

"And a 400,000 vote margin is going to be very hard for Jeb to make disappear."

Not when you have Diebold on your side.

> Matt, are you implying that people who pick up the phone
> because either they still don't have call waiting, or because
> they have call waiting but are likely to take calls from
> random people, are more likely to also be the same people
> that nod their heads in agreement when Bush says we're safer
> now?

Not necessarily, although that's certainly possible.

Here's how I like to think about it. Supposed Gallup conducted
a poll to find out the demographics of the country: how
many men are there with college education who make
over 40K/year, how many married women with only high school
education, etc. If they did this, it would fail miserably: the
non-response rate is so skewed that theire estimates would
be quite bad. Pollsters know this is a big problem, so they
have various models that they use to "fix" their samples
so that the demographics are "correct", which produces a
reweighting of their actual polling results.

These models are *everything*, especially if the samples aren't
truly random. [Obviously, all the demographic subsamples
have large sampling error, which could be random, but
that's pretty much ignored.] They're probably
too simple, since the relationships
between the different demographic categories and
voter preference are complex and not necessarily even linear,
and the data to determine this would be difficult to obtain
(since we're talking about people who aren't responding
in the first place). This goes way beyond cell phones.

It's not that pollsters are dumb, they do try to make these models useful. But the press NEVER reports these models,
they just repeat the same old margin of error (which
itself is reported accurately at only about the 60% confidence interval, usually). As Brad has pointed out in other contexts, it's likely that the reporters don't understand any of this.

A couple of web sites doing a great job with polling and more specifically analyzing the poll data.

http://www.pollkatz.homestead.com/

Doesn't display so hot if you use mozilla, but a great site. Explains a lot of the rudimentary statistics associated with polling. Also if you scroll down you see an example of how he reweights the Gallup poll. I've seen a number of discussions on this and gallup has seemingly just changed the assumed distribution on dem. rep. and ind. ( gallup's 44%R, 31%D, 25%I) vs. (rassmusen 36%R, 39%D, 25%I).

http://www.electoral-vote.com/

First site I check every morning when I wake up. it updates every day between 7-8 et. This guy is doing an absolutly amazing job with the state polls and electoral college. Any time you spend poking around his site is time well spent. He too has a discussion of the gallup re-weighting, though I cant seem to find it now. This was also the first place I heard any discussion of the potential problems of not polling cell phone only people (of which I am one, also young, also a grad student, also supporting Kerry). This site has been attacked, but their are backups at http://www.electoral-vote2.com/, http://www.electoral-vote3.com/.... I think up to five, but atleast three. I cannot recommend this site enough. Also is keeping tab on competative senate races. Can the dems actually win a senate seat in Kentucky, probably, check out the new polling. This sen. bunning guy sounds straight up crazy, which I take back with apologies if turns out he has Alzheimer's.

Did the Economist endorse a candidate four years ago? I can't figure why American newspapers endorse presidential candidates, let alone British weeklies.

I thought everyone knew that despite outward appearances, The Economist is staffed largely by Americans and published largely for an American readership.

At least see prior discussion.

Right-wing republicans have methodically (emphasize "methodically") infiltrated numerous organizations and industries with a view to steering events and opinions their way. Witness the K-Street Project, broadcast and cable television and radio, church organizations.

I don't find it difficult to imagine that republicans have moles in the major polling organizations who will devise and direct subtle and not-so-subtle polling methodologies to produce a desired result. The psychological value of being on the winning end of a poll is enormous and pays off in voter suppression, favorable buzz and the illusion of momentum and invinceability.

It would be nice to extend the investigation of Gallup's methodology to include its management and personnel.

Ok, this is off topic. It is, nonetheless, of interest.

Has anyone else noticed The Economist has chickened out of a presidential endorsement?

Will they do something for the Oct 31st issue?

I figure they're terrified of losing 1/2 their subscribers. If they do Bush, I plan to mail them the cover along with my cancellation notice.

I am still flabbergasted that the pollsters have not done the kind of validation work to identify where there model-based disagreements lie--and that the press corps has not called them on this.

Call me cynical, but I don't think it's strange in the least that the press has not called the pollsters on their wildly skewed results.

Why? Because they're not smart enough to call them on it. They are herd animals.