« You Know Michigan's Safe Now! | Main | New Polls Have Kerry Ahead in OH, PA, IA, Lagging in FL »

A Stroll Down Memory Lane

The polls have generally been moving in the right direction lately for John Kerry, both nationally and on the state level, but Democrats are still inclined to be sent into a tizzy by any negative poll result they run across.

They shouldn't. It's time to revisit the thrilling polls of yesteryear to get a sense of just how much the polls in 2000 tended to overestimate Bush's strength and underestimate Gore's. I believe, for reasons I have discussed at length, the polls are likely overestimating Bush's strength this year as well. But this year, Kerry is doing better in the polls than Gore did at the equivalent point in the 2000 race. Therefore, if current polls are overestimating Bush's strength by the same amount as in 2000, Kerry should wind up doing better than Gore on election day--and Gore won the popular vote by half a point. And that's not even factoring in the likelihood that, with Bush as the incumbent, Kerry will receive the bulk of undecided voters' support on election day.

So let's take that stroll down memory lane.

Start with this nugget from Alan Abramowitz:

During the final week of the 2000 campaign, 43 national polls were released, including multiple releases by several polling organizations such as Gallup. George Bush led in 39 polls, Al Gore in 2. Bush's average lead in the polls was 3.6 percent.

Something to keep in mind when people complain that so far (two days) in this final week Kerry has "only" had small leads in the DCorps poll, the Harris Poll and the WP/ABC tracking poll twice (LVs and RVs)!

And here are some readings from specific 2000 polls:

1. The ABC tracking poll averaged a 4 point Bush lead in the last week and its final poll had a 3 point Bush lead.

2. Bloomberg News final poll (October 29) had a 3 point Bush lead.

3. Final Time poll (October 26) had a 6 point Bush lead.

4. Gallup's tracking poll had Bush ahead by an average of 4 points in the final week and by 2 points in its final poll.

5. Marist College's final poll (November 2) gave Bush a 5 point lead.

6. Final NBC/WSJ poll (November 5) had Bush up by 3 and their mid-October poll had him up by 6.

7. Final Newsweek poll (November 2) had Bush up by 2 and their October 27 poll had him up by 8.

8. Final Pew Research poll had Bush up by 2.

9. A November 4 CBS/NYT poll had Bush up by 5 (though the final CBS poll was dead-on, with a 1 point Gore lead).

10. Final ICR poll had Bush up by 2.

11. Voter.com Battleground survey (this year called GWU Battleground) averaged an 8 point Bush lead in the final week and its final poll gave Bush a 5 point lead.

12. TIPP tracking poll gave Bush a average 6 point lead in the final week and a final poll lead of 2 points.

13. Prior to its well-known final reading of a 2 point Gore lead, Zogby's tracking poll gave Bush an average 3 point lead in the final week.

14. Final Hotline poll (November 5) gave Bush a 3 point lead.


It would be interesting if you mentioned the drunk-driving revelation that hit just before the election in 2000. I don't recall exactly when the revelation hit nor how the coverage of it evolved but perhaps that caused remaining undecideds to move toward Gore instead of splitting more evenly. That would be similar to this year's election. However, if the DD issue hit and actually moved decideds away from Bush, perhaps we can't assume the polls truly underestimated Gore's strength.

This is a great reminder of what a good position Kerry is to win this election -- and win it big. Thanks for posting it!

(And this year we have three factors in our favor we didn't have in 2000 -- an incumbent with a horrible record, a record surge in newly registered voters -- which are expected to lean heavily toward Kerry -- and a Democratic/Left that is more united than at any time in modern history).


Something strange is happening in the Rasmussen tracking poll.

I calculated the 95% confidence interval of the day-to-day variation of the Rasmussen poll as 1.03%. (I will be happy to share the details of the calculation with anyone who asks. I made assumptions about Rasmussen's fixed values of party ID and about the current voting distribution of each party's identifiers that will not be exactly what is in Rasmussen's poll, but my assumptions should be good enough that this value is very close.)

The behavior of Rasmussen's poll through October 21 (polling dates) is entirely consistent with this calculation. However, on three of the last five days, at least one candidate in Rasmussen's poll has shown a change of 1.1% or more, and on a fourth day there was a change of 1%. This is an extreme statistical improbability if the day-to-day changes are due only to sampling error. The shift toward Kerry on the 22nd through the 24th seems to reflect something real (but see below about what), since there was a contemporaneous shift in the Post/ABC poll. But it seems to me that the sharp shift toward Bush on the 25th and 26th, mirrored in no other tracking poll, must represent a change of some kind in Rasmussen's methodology.

Since the Post doesn't report tenths, I didn't try to repeat the above analysis for them. But if you compare each of their last 4 days with the separate sample 4 days earlier, the shifts average 3 points -- near the edge of their confidence limit and maybe beyond it. So I'd have to conclude that the shift toward Kerry in the Post poll, looked at by itself, looks like it's probably real, but it isn't strong enough to be able to say for sure.

TIPP has such small sample size that I don't think we can pick anything out of the noise in their results.

Finally, the letdown for Kerry supporters. I have two alternative explanations of the shift toward Kerry that was seen last weekend in both Post/ABC and Rasmussen: 1) a real shift in public opinion, or 2) more Red Sox fans at home in the evening during the World Series to answer the phone. Explanation (2) is favored by the fact that Zogby, who calls in the daytime, did not see this shift.


I've said it here before, but it needs saying again. If I were a nail-biter, I would have nothing below the elbow at this point.

This isn't because I do not believe in the superb analysis and reporting done at this site. The nervousness is due to my equal if not greater belief that the right wing in this country will do anything to win, and their frightening level of influence over the media, plus their ability to stop any investigation by Congress and to get their way in the judicial branch gives them far too much room for manuvering.

So my hope is not that Kerry gets more votes. At this point I am sure he will -- so that's a given. My true hope is that the win has to be big enough that the dirty tricks the other side is prepared to deal out won't be enough to tip the balance. That's what is behind this obsession in building a big "poll lead."

I speak only for myself, but I am sure that many of your readership feels the same way, and that is why we all are jostling your elbow over this.

Thanks for all your hard work.

Happy as I am to see John Kerry's small but growing lead, I must remind everyone that the polls in 2000 moved on the last weekend thanks to the release of an old drunk-driving arrest record from Maine. This time, we've been getting a steady stream of revelations, so it's not quite the same thing.

I won't feel comfortable until Dick Cheney bangs the gavel on the Electoral College.

Perhaps I'm being just the kind of nervous Democrat Ruy is complaining about in this post. But here is my question: haven't the pollsters' likely voter models been updated somewhat to reflect the 2000 results? In other words, wouldn't all these polls have generated different LV results, closer to what happened in 2000, if they'd used today's LV models?

Thanks for going back to 2000. I've been thinking about that election a lot recently and wondering if anything could be/has been learned from that year's polling. As you point out there was no incumbent in that race so I've had my doubts about the validity of comparisons between the two.

I've learned a lot recently, from this site and others, about the incumbent 50% rule (and other polling quirks). But with my limited knowledge, these questions are hard for me to answer:

What explains the Zogby result in 2000? Was the 3 point Gore deficit legitimate or not? Was there a Gore surge in 2000 or was that a myth created by skewed polls before the election?

Did Zogby just get lucky with his final poll?

Lots of people here in the UK are very interested in your election for obvious reasons. The thing which I feel I understand least is the comparison with 2000. It's clear that the polls overstated the likely Bush vote and understated Gore. But is that because Bush is Bush and polls overstated his likely performance? Or because he was the challenger wanting to change the Administration and when it came to it people decided he was too big a risk? If it's the former the current Bush vote is the one which is being overstated; if it's the latter then it's Kerry who will underperform on the day. I find that a lot of people here in England think it's the latter and so tend to downplay Kerry's chances.

What no one has addressed, however, is why the polls favored Bush in 2000. There are at least two possibilities I can think of that have different implications for the current situation. First, Bush might actually have been ahead most of the time; boring but possible. Since polls just before the election continued to show his lead, I think, maybe not correct. Second possibility, there is some way in which polls are systematiclly not reaching Democratic voters that existed in 2000 and may exist now. What is it? Or is there some third or fourth reason.

Are there any arguments one could conceivably make to claim that the polls in this campaign are MORE accurate than those from 2000?

David in NY, you asked about the polls off in 2000, and why they might have missed it.

In my opinion, it tracks the usage of cell phones.

In 2000, cell phones were not the epidemic they are now. They have attained a cultural utility not seen in 2000.

In 2000, the cell phone factor may have accounted for a 2-3 point miss by most pollsters. This time, probably more.

In the post election period, it will be our job to force the pollster review into the national dialogue.

All these surveys are based on "likely voters."

What about unlikely voters.

We know from the lines at polling places around the country there are millions of unlikely voters this time.

We also know, from history, that unlikely voters tend to break one-way. They did in 1994 (for Gingrich), they did in 1992 (for Clinton).

A poll of Registered Voters won't pick this up, because you don't know who among those RVs is an UL, who will actually show up.

Someone, probably after November 3, needs to create a metric that will identify these unlikely voters and calculate their impact.

Otherwise polls are just entertainment.

P.S. -- I think there is an avalanche of unlikely voters coming in, and the great majority are Kerry supporters. I think that's your hunch, too, but like Red Sox fans you don't want to jinx it.

David Blake's question is a good one. Why did the polls consistently overstate Bush's support in 2000?

There are several possible explanations, but I've never seen solid evidence for any.

One simple possibility is that the polls sampling methodology is skewed rightward, and has become more so over time. This theory is very much a product of wishful thinking on my part, but it does offer explanation for what we've seen.

Now, we already know that Democrats behave differently than Republicans in terms of survey responses. That is, democrats are more likely to be non-respondants. This may be partly due to phone screening and opting out (the large majority of people contacted by survey companies do opt out in part because of the time required to do the survey). This may also be partly due to a preference for cell phones.

So, the solution most companies (except Gallup) uses is to weight the response from Democrats higher than Republicans so that it matches the expected Democrat participation in the election. BUT, this assumes that respondant Democrats have similar voting preferences than non-respondant ones. Since Democrats run a spectrum from conservative to very liberal, this may not be the case. The theory here is that the sample of Democrats the pollsters are getting is more conservative than the Democratic population as a whole. There are several points that may suggest this: people who move frequently or are early adopters of technology are more likely to be open-minded a.k.a. liberal, and those are the same people who mare more likely to be non-respondants.

Note, the same may be true of the Independent population, which also varies greatly from arch conservatives to those on the far left. If the sample of Independents is skewed right, then any weighting system based on treating independents as a group will skew right.

Now, if you buy this theory as a possible explanation for the polling errors of 2000, then the error factor for 2004 is likely to be even worse, as the trends regarding non-respondance and cell phone usage have continued.

This theory works as follows: Democrats are less likely to be contacted by sampling methodologies due to a variety of factors, such as a preference for cell phones, a stronger tendency to screen calls, or simply a tendency to opt out of polls more frequently. I've seen one recent study that supports this theory.

Now, the polling firms (except Gallup) attempt to compensate for this by giving increased weight to the self-identified Democrats in the survey so that their proportion of the results matches their expected share of the vote. In other words, the polls inflate the number of respondant Democrats to make up for the absense of the non-respondant Democrats. The problem with this is that it assumes the two will have the same vote pattern.

Again, I don't think you can compare any results with those of 2000. I believe, and I hope I eat my words on this, that those who are undecided will move to Bush in the end, after listening to more and more anecdotal evidence of how people are perceiving the two candidates, and also after looking at how much closer Kerry is in some states Gore won last time. In 2000, Gore was the incumbent VP, and people homed to him.

Also, I was reading that this election is not like any other incumbent referendum in that usually when an incumbent might lose, it is by a large margin. No race has been this close with an sitting president, so to count on the fact that Bush is a hair below 50 in the polls, I think is self-deluding.

Turnout matters; although apparently nationally the repubs have registered as many new voters as the dems, but anger and support fueling actual voting matters.

And of course the battlegrounds matter, and they are horrifically close.

I would like to see a polling model using all of the current information, that takes into account a huge turnout, but the undecideds either not voting, or going largely for Bush, and see what that looks like.

Not to be a Cassandra, but what those polls from 2000 did not reflect was Bush's DUI story that was revealed the Sunday before the election. Since this was a mere 2 days before the election, polls did not have time to reflect the story's impact on the electorate.

This may be the dumbest question yet (if so,please be kind!) but don't we know that Gore won the popular vote nationwide and most likely also won the undercounted popular vote in Florida?

Are we basing the comparison between 2000 and 2004 polls on the false assumption that Bush surely and truly won Florida in 2000? and therefore the election?

Would a look at the polls predicting the 2002 midterms give us more info about the accuracy of predictions by pollsters?

For David in the UK.
You would think from the "cable media" in this country, CNN, MSNBC & of course FOXNews that Bush was a sure thing going by the polls they chose to promote (they "cherry pick" these polls and this has been an issue of consternation from the Kerry supporters for months).
Our cable media has a vested interest in "promoting" Bush, as they want further deregulation of their industry (like they don't monopolize enought media outlets alread?) so they can increase their bottom line for their shareholders. As we saw with Sinclair Broadcasting what can happen when one conglomorate controlls a full 25% of a particular market, the irrisistable urge to use it's (owners)political bias to propogandize.
Not to labor the cable media influence that they have had in this and prior elections (I include their lopsided presentation of Clinton's Impeachment debacle), but I do believe that they have been rumbled! The American people are finally realizing that our cable media that proports to sell news is actually "selling" bias and influence to the higest bidder, i.e. to Bush if he will deregulate their industry further, to the energy companies to losen pollution controlls and ignore rampant price increases, to the Evangelists by dangling the promise (but never delivering) of overturning Roe v.Wade etc etc.
What you see on our TV, especially the cable media is not what is going on in this country! What they are promoting does not reflect reality. It is an image, literally a manufacturedan image, like marketing a product, to try and influence this election and convince rest of the world that America is behind Bush.
I promise you, Kerry will win big next Tuesday...even with rigged voting machines, lost ballots (as of today 60,000 absentee ballots lost in heavily democratic Broward County, Florida), and intimidation in minority areas.

Dana B. wrote:

"I think there is an avalanche of unlikely voters coming in."

Well said.

The UNLIKELY voter has gotten no love this time, but he and she are going to get plenty of attention afterwards.

Who the hell do these pollsters are standing in line for hours to vote BEFORE the election?

Millions of UNLIKELY voters are voting.

And it ain't to say "good job, Dubya!"

It's to say "you're fired!"

"I think there is an avalanche of unlikely voters coming in, and the great majority are Kerry supporters."

I would tend to agree with this, although my opinion is not founded on anything remotely scientific. It's just my gut talking, and my gut is telling me the pundits and pollsters are going to be scratching their heads and pointing their fingers come November 3rd wondering what happened out there.

Well, Mr. Pundit, yank your head out of your computer models and take a look around. Why do you think so many new people are registering to vote? Why are there three-hour lines at the early-voter polling stations? Why are expats up in arms over missing absentee ballots? Do you really think they're standing outside in the sweltering sun to tell the nation how hunky-dory they think things are?

People are pissed off, Mr. Pundit. That's why they're out there. Pissed off people vote in much larger numbers than those who are satisfied with the status quo. And the majority of these pissed off voters aren't pissed off at Kerry.

Bush has hit his ceiling in terms of the number of people who actually approve of his presidency. It ain't going any higher. On the other hand, he's still got five more days to piss off the entire rest of the country. What are the odds he'll manage to do it?

No one has addressed the 2000 situation when Zogby was the only major poll to gather results over the final weekend. This was after the Bush DUI charge came to light. Kerry backers shouldn't count on any such information this time around (if there was any damaging info on Bush, surely it would have been revealed by now!). The undecideds may still break Kerry's way; I wouldn't expect the big swing we saw in 2000, though.

I understand the rationale for not weighting by party ID or party registration: both can always change from year to year.

Why don't polls ask who the person voted for last time and then weight the sample to make it reflect the national popular vote in 2000? If we kept getting surveys where only 45% of people voted for Gore, we could correct for that bias.

To Ben Ross:

A very astute analysis, as always. But I have to question 2 things you posited:
1. Realistically, given the significant minority of people in this country who are Red Sox fans (after all, there are a whole lot of other teams, and lots of people don't even like baseball), is it even feasible that more Sox fans at home could've had THAT much of an influence in the tracking poll numbers?
2. Even if the answer to #1 is "yes," I still strongly doubt that explanation -- if you were a Sox fan watching your team in the World Series, and a pollster called in the middle of the game, would you stay on the phone?!?!

To David Blake from Britain:

If you are a regular reader of this (or other liberal democratic) blogs, you would know that in incumbent presidential elections the VAST majority of undecideds vote with the challenger in the end -- typically 80% OR MORE. Another way to express the same thing is to recognize that incumbent presidents almost always get the % that they get in the final poll, while challengers get 4% more ON AVERAGE (and 2% more at a minimum). For more on this, read Guy Molyneux' article in American Prospect (americanprospect.com, click on "archives", then search by the author's name). The Brits who downplay Kerry's chances are in error; it is clear that Kerry is leading right now. If the election were held tomorrow, Bush would get 48% and Kerry 50%.

Although I want more than anything for Kerry to win, I'm not sure it makes sense to look at the 2000 polls and assume that since they overestimated Bush's support in 2000, they are overestimating it now.

Why? Because I assume that most of the pollsters are intelligent statisticians who probably worked hard to figure out why they were off so much and have, therefore, changed the way they are polling this year to correct the errors they made in 2000.

Suzanne is absolutely right. The media REFUSES to show ANY polls showing Kerry ahead, or even MENTION the fact that the polls that have Bush ahead, still show him completely vulnerable. Everytime Bush gets a 1 percent increase its "BUSH SURGES AHEAD IN POLLS!!!!" When its Kerry, they say "Kerry ties race" NOT ONE STATION has mentioned how much trouble Bush really is in this election. If Kerry wins, it will be FASCINATING to see all these idiotic talking heads and pundits trying to explain HOW and WHY the polls were wrong, and how Kerry won. I would love NOTHING more then to see Sean Hannity and Bill O' Reilly squirm in thier chairs trying to get explanations to it all, and try and find out how thier precious candidate got his hat handed to him.

I wonder whether this year's polls are skewed by the national "Do Not Call" registry. While political campaigns are, of course, free to call anyone, I wonder whether commercial polling operations may be reticent from calling people on the registry. After all, the potential fines are significant.

The net result: are polls measuring that subset of the population that has *not* opted out of telemarketing phone calls?

Psychologically and politically, how might those populations differ from each other? Some theses:

(1) Those on the federal "do not call" registry certainly value their own privacy; those who still accept telemarketing calls may be less concerned about privacy issues.

(2) Those on the registry appreciated an activist federal government that defended their rights against commercial interests; the legislation was opposed by individuals who favored the unfettered freedom of commercial interests to make a profit.

Valuing privacy rights? Promoting consumer interests over business interests? Hmm... Could it be that Kerry supporters are being undercounted by pollsters who are respecting the "do not call" registry?

I don't think there's ever been any evidence, even anecdotal, that George W. Bush's DUI revelation moved many voters. It was ugly, but not dispiriting to many. It was a long time ago. Who really cared? Karl Rove figured it cost Bush millions of votes, but how does he know that? Seems like rationalization.