« Bush's Battleground Blues (Continued) | Main | New Gallup Poll Shows Movement Toward Kerry »

Analysis of Trends in Major Media Polls

By Alan Abramowitz

It isn't just the tracking polls that seem to be fluttering randomly in the wind. An analysis of polls by six major media outlets (Time, Newsweek, CNN/USA Today/Gallup, CBS/New York Times, Pew, and Fox) that released polls during the first half of October and during the second half of October reveals that there is a strong negative correlation (-.84) between the early October results and the late October results.

In other words, the better Bush was doing relative to Kerry in the early October poll, the worse the was doing in the late October poll. Some of this might be explained by the well known phenomenon of regression toward the mean. If a poll's early October sample had either too many Bush supporters or too few Bush supporters just due to chance, it's late October sample should be closer to the true population mean. But these results went well beyond regression toward the mean. Every poll that was above the overall mean in early October was below the overall mean in late October and every poll that was below the overall mean in in early October was above the overall mean in late October. What this bizarre pattern suggests is that the movements of the major media polls in October, like the movements of the tracking polls, reflect sampling error and peculiarties of the polls rather than real change in the underlying preferences of the electorate.

Comments

I hate to say this, but if everyone had a hunch their polls were not making sense then fudging (er... fine tuning) the methods in ways that eventually overcompensates would also create the described effect.

Oct. 25, Rassmussen has Kerry ahead of Bush for the first time since August.

Relax, have faith, we are not alone, ignore all polls if your nerves can't take it and GOTV!!!

I disagree, in part. Some, but not all, of the fluctuations in the tracking polls are simply statistical noise.

Polls that fix the number of Democrats and Republicans in the sample should have less sampling error. Zogby and Rasmussen do so, and they have smaller day-to-day fluctuations. The sampling error MOE with fixed party ID for 3000 respondents (Rasmussen) is around 1.2% and for 1200 respondents (Zogby) it's around 1.9%. The range of fluctuations for both of these polls since a few days after the last debate has been slightly larger than these numbers. (There is random error due to estimation of weighting coefficients, and there were no doubt oscillations in actual voter preference that were too small to pull out of the statistical noise.)

However, Rasmussen's result today of 48.4% for Kerry exceeds the average of the preceding 15 days by 2.3%. This is almost 4 standard deviations and seems to represent real movement.

The statistical significance of Zogby's move in the opposite direction over the last few days is harder to judge, partly because Zogby doesn't give results in tenth of a percent.

Polls that don't fix party ID have bigger sampling error. The Wash. Post's poll oscillations seem to be just slightly larger than the sampling error, although I can't calculate exactly because they partially adjust for party ID and because they sometimes average over 3 days and sometimes over 4. (Their RV MOE would be about 2.1% if there were no party ID weighting at all.)

When polls move together, of course, it has statistical meaning even if the polls taken one-at-a-time could be random movements. If Zogby jumps more Democratic over the next couple of days (as is likely, because two very pro-Bush days will fall out of the sample), and the Post and Rasmussen stay more or less the same, or if Zogby and Rasmussen stay the same and the Post trends more Kerry (which would bring it out of the sampling error range), we will have a clear trend.

Mr. Abramowitz:

I don't think these results necessarily indicate random fluctuations. Maybe the likely voter screens are working as might be expected: screening out the less intense, less informed late-to-tune-in crowd 3 or more weeks out from the election, and then screening them in as the election nears and their attention sharpens.

Today (Oct 25) Prez Track , Rasmussen's
site , shows Kerry ahead for the first time
since late August.

A couple of points on this from Wm. Saletan during the 2000 election:

"On its Web site, Gallup makes clear that its poll seeks to maximize daily change: "Our objective is to pick up movements up and down in reaction to the day-to-day events of the campaign." "

-- also --

"CNN and USA Today are in the news business. They're paying Gallup for new numbers every day. If Gallup's numbers don't change, where's the news? So Gallup has an incentive to keep its filter loose, allowing the winds of shifting partisan intensity to blow its numbers back and forth."