« More On Those Alleged Security Moms | Main | AP-Ipsos Poll: Kerry Lags by 9 »

Gallup Vs. Democracy Corps

The indefatigable Alan Abramowitz offers the following comparison of the track records of Gallup and Democracy Corps in the last presidential campaign. The results may surprise you and have some important implications for assessing recent poll results.

What should we make of the huge discrepancy in the results of recent national polls? On the one hand, the prestigious Gallup Poll has George Bush leading John Kerry by 13 points among likely voters. On the other hand, a Democracy Corps Poll released just a few days later shows that the race is a dead heat. Who should voters believe? Most people probably assume that the Gallup Poll is a lot more credible than a poll conducted by a partisan polling organization. After all, Democracy Corps is headed by two well-known Democratic political operatives: James Carville and Stan Greenberg. But if that's what you're thinking, an examination of the track record of these two polling organizations during the 2000 presidential campaign might make you change your mind.

On October 25, less than two weeks before the 2000 presidential election, Democracy Corps released a national poll showing Al Gore leading George Bush by 2 points. On the same day, Gallup's national tracking poll showed George Bush leading Al Gore by 7 points. One day later, the Gallup tracking poll had Bush up by an incredible (literally)
13 points. Score one for Democracy Corps.

Okay, you're probably thinking, maybe Gallup just had a bad day (or two or three). But six days later, on October 31, Democracy Corps released it's final pre-election poll showing a tie in the presidential race. Gallup's tracking poll that day still had Bush leading by 5 points.

Make it Democracy Corps 2, Gallup 0.

In its final poll, Gallup did have Bush's lead down to only 2 points--still not as good as Democracy Corps' poll released almost a week earlier.

Now I'm sure that Gallup and its defenders in the polling business would argue that Democracy Corps was just lucky and that the Gallup Poll was actually more accurate all along because it was picking up real shifts in voter sentiment in the final two weeks of the campaign. Maybe Al Gore really did go from a 13 point deficit on October 26 to a small lead on Election Day, maybe there really is a 6 point Republican advantage in party identification among registered voters right now, as Gallup is currently telling us, maybe Iraq will hold free and fair elections next January, and maybe the tooth fairy will pay you a visit tonight, but I wouldn't count on it. If I had to put down a bet right now on which poll will prove to be closer to the actual results of the 2004 presidential election, Gallup or Democracy Corps, I'd put my money on James Carville and Stan Greenberg.

Oh, and if you're looking for a backup poll, I'd recommend Fox News/Opinion Dynamics. Their final pre-election poll in 2000, released five days before the election, also showed a dead heat. Their latest 2004 poll has Bush leading Kerry by 2 points, right in line with the new Democracy Corps poll. And I don't think anyone would accuse Fox News of having a pro-Kerry bias.

Pretty darn interesting. And if reading this has put you in the mood for a little DCorps analysis, you really must check out their latest analysis of "The State of the 2004 Race". Highly worthwhile.

Comments

I average polls. I started doing this just to try and make some sense of the extremely erratic poll results we were seeing last winter in the home stretch of the Democratic primary campaigns. And it worked remarkably well.

There's not much to it, really. If you've got a poll with, say, 500 people, then you know 1% = 5 people. So if candidate X came in at 10% and candidate Y got 20%, you figure that's 50 people for candidate X and 100 for Y. You note the numbers and move on the next poll. When you get to the end, you add up all the total samples and then break out each candidate's people total as a percentage of the aggregate sample.

The one additional thing I do is I down-weight the sample sizes for RV polls to LV levels. I did that because I noticed early in the year that LVs tended to favor Bush, as did 3-way races over 2-way races and rather than agonize about it, I just wanted to know, "OK. Assume the worst is true. What's the worst possible case for Kerry?"

So I started using LV and 3-way numbers over RV and 2-way matches wherever available. But I also wanted the largest possible aggregate sample size, so I wanted to use RV polls that weren't screened for LVs as well as those that were. I just didn't want their larger universes skewing my averages too much. So I multiply the sample sizes for those RV-only polls by 0.85 -- roughly the percentage of RVs who turn out to vote -- before I put them into the average.

The other thing I don't agonize over are differences in methodology, claims of bias, oversampling, whatever. I throw in Zogby, Fox, Gallup, ARG, you name it. If they have any credibility at all, they're in. I figure even if some of the charges are true, most of that stuff should wash out if you put enough of them in the washer together.

What I end up with, over a typical week or two-week period, are an aggregate sample size in the 6000-10,000 range and presumptive MoE in the +/- 1% range. I also end up with an average so stable, you can throw out the biggest outlyer or even the biggest 2 or 3 in a lot of cases without changing the Bush/Kerry point spread by more than a point or so.

I have been doing this since January for the Presidential race and throughout that time, no Democracy Corp poll has been off my average by more than +/- 1%. It's just uncanny. I could have saved myself a lot of trouble by just looking at the Democracy Corp polls, ignoring everyone else, and still have high confidence I was getting the straight poop.

Ruy,

I guess it is no coincidence then, that the first I've ever heard of so-called 'security moms' came from CNN's Political Analyst Bill Schneider yesterday, on the supposed gap closing by Bush among women, based on one singular poll (CNN/USA Today/Gallup)!

Maybe, you can glean an explanation of cable news' newfound obsession with polling? I know it's the reason I'm spending more time here, than thinking up ways to dispose of Wolf Blitzer's corpse.

Good analysis, Ruy. Another interesting internal from the Democracy Corps poll is the fact that respondents went for Bush over Gore 51-43. So Kerry would seem to be ahead of Bush2004 amongst Bush2000 voters. If he can maintain that, hang onto the Gore2000 voters, and we get a good GOTV effort, this race should belong to Kerry. With the week Kerry just got done having, he could be ahead in the next DC poll.

This poll puts Kerry in a good place. Because Nader is not going to get 5 percent. He didn't get more than 2.5 last time, and he's on fewer ballots this time, with no Green party backing, no money, and no celebrities. He will be lucky to get 1 percent. So award most of those votes to Kerry. Bush could lose as many votes to the Libertarian/Constitution parties as Kerry loses to Nader. And the independents will break for the challenger.

Bush peaked after the RNC. The Democratic base and the African-American vote will all come home by Nov. 2. The only way Bush win this time is the way he won the last time. With fraud and dirty tricks.

Dear Ruy:

I certainly haven't been keeping up with the 2004 polls as much as you have, but one thing to keep in mind is that the GOP did narrow the gender gap in the 2002 House races, according to that long lost VNS exit poll data you, me, and about two other guys in the whole country analyzed when it became available a year ago.

Here's an excerpt from my UPI article, including a quote from you:

Analysis: The voting gender gap narrows
By Steve Sailer
UPI National Correspondent
Published 11/20/2003 12:23 PM

LOS ANGELES, Nov. 20 (UPI) -- Data extricated from the collapse of the lone national exit poll in the 2002 congressional elections show that the gap between how men and women vote declined to the narrowest difference since before the 1994 House elections.

A United Press International analysis of the results of election night surveys of 17,872 voters shows that much of the GOP's 5-percentage-point improvement in the House voting last year came from its increased appeal to women.

Republican candidates' share of the male vote grew from 54 percent in 2000 to 55 percent last November. Their fraction of the female vote, however, rose from 45 percent to 50 percent. This was the first time in several decades that at least half of women's votes went to GOP House candidates.

While women were somewhat less hawkish on Iraq and less libertarian about the role of the government at home, the most striking attitude difference was feminine foreboding. Women expressed more worry than men did to Voter News Service pollsters about terrorism, the economy and the stock market.

Gender gaps have tended to be smaller in elections during recessions, such as 1982, 1990 and 1992, and then widen when the economy is strong and worries are fewer.

It's not clear if the gender gap will remain as small as the 5-point margin in 2002. In the California recall, action movie star Arnold Schwarzenegger, a Republican, ran 8 percentage points better among men, and in last week's Mississippi gubernatorial election, the gap was 11 points.

Voting analyst Ruy Teixeira, co-author of "The Emerging Democratic Majority," pointed out to UPI that in midterm elections, the gender gap tends to be smaller than in presidential election years (other than in the 1994 election), when more unmarried women come to the polls.

http://www.upi.com/view.cfm?StoryID=20031113-090231-4928r

My cover story in the October 27th issue of The American Conservative will be a survey of voter demographics.

I am glad that I am not the only one that thinks in such terms of Wolf Blitzer.

I hope I am proven wrong come November 2, but my gut feeling is that unless there are some major developments that help the Democrats between now and then, we are in for another four years of Mr. Bush and his cronies, polls be damned.

I'm still concerned that some of our (Kerry supporters) greater optimism about the polls is based on the presumed "oversampling" of Republicans in many of the polls. This is based on the fact that the partisan breakdown in the samples is way out of synch with any recent or historical polls of party ID. It has been suggested, however, that, when asked their ID in the context of a poll on the Presidential race, respondents tend to make their party ID conform to their candidate choice. Therefore, someone who might be asked their party ID in a vacuum and would say Democrat, instead says Republican, if they are voting for Bush this year. The suggestion was that, if the question is asked at the start of the poll, the respondent is more likely to give an answer reflecting their usual voting pattern, but when asked after stating their candidate preference, a number will conform their party ID to their candidate choice. So, instead of an oversampling, what we're seeing is a certain number of Democrats stating an intent to vote for Bush. It may be that more people usually identifying as Democrats than Republicans will vote this year, but if a bunch of the D's vote for Bush, we still lose. Any thoughts?

I guess the vast right wing conspiracy has won again they managed to get it so no one posts on you anymore. I guess I will say goodby along with everyone else.......

Cal D,

That was a well thought out post regarding how you average polls, and I take note of your suggestion that blending all polls tend to mitigate the bias pollsters and outliers.

But I'm confused about the following: You indicate that Democracy Corps (DC) is a close representation of where the race is today, on the premise that DC represents a consensus average off all polls over, say, a 1 or 2 week period. What confuses me is that ARG, Zogby, Harris, Economist, FOX & Pew all had the race basically tied, close to your DC dead heat - But all other major polls including USA Today/Gallup, CBS, NY Times, AP, Time, Marist and ABC/Wash Post, all had GWB up by margins from 6 to 11 points.

If you mesh all these polls, I get GWB up 4 or 5, not the DC dead heat that you suggest. Nice try though.

Zogby also had the "right" numbers last time around so there is a problem with Gallup.

Everyone should check out this NY Times article on voter registration drives in battleground states. I know the polls are important, but this article points out several things. 1) The polls don't fully take into consideration newly registered voters. 2) The Dems are outpacing the Republicans in voter registration. In some areas its up a HUGE 250 percent compared to the Republicans getting 25 percent increases over 2000. While this is impressive for our side- the next step is getting those people out to vote during the election b/c if they turn out, we win.

Excerpt from NY Times:

In Florida, where The Times was able to analyze data from 60 of the state's 67 counties, new registrations this year also are running far ahead of the 2000 pace, with Republican areas trailing Democratic ones. In the 150 ZIP codes that voted most heavily for Mr. Bush, 96,000 new voters have registered this year, up from 86,000 in 2000, an increase of about 12 percent.

But in the heaviest of Democratic areas, 110 ZIP codes that gave two-thirds or more of their votes to Al Gore, new registrations have increased to 125, 000 from 77,000, a jump of more than 60 percent.

In Duval County, where a confusing ballot design in 2000 helped disqualify thousands of ballots in black precincts, new registrations by black voters are up 150 percent over the pace of 2000.

Check out the article:

http://www.nytimes.com/2004/09/26/politics/campaign/26vote.html

The combination of this plus the polls, Kerry's newly energized campaiqn, the swing of the narrative of the national media (I don't know about you but I noticed a change this week) and what happens at the debate next week could give us the big mo through election day (Barring an Oct surprise regarding Bin Laden that is - which some people are suggesting there is a deal regarding Pakistan with that (ie, we ignore their nuclear proliferation and they hand us Bin Laden (I am hoping this is just internet rumor mongering b/c this would be an incredibly stupid and dangerous thing for us to do to let Pakistan go on the nuclear stuff (I predict that country will be the next hot spot as the extremist there gain more and more power- I am hoping I am wrong though)).

Ruy--

I've seen at least two stories recently talking about how Republicans are more energized about this election. Aside from how conterintuitive this perception seems, how does this mesh with the whole Gallop poll Repub/Dem weighting issue? One of the stories was a CNN story on a CNN/USA Today/Gallup poll in Florida--how does this factor in?

one addendum to the NY Times article. I would add that everyone, everyone, their mama, and any lawyers you know, should be working to prevent any election suppression by Ashcroft, Norquist and others prior to election day b/c by election day its too late to have these systems in play. I am planning to volunteer for this effort. I hope everyone here is also planning to man a phone, act as a aid, etc, during this time period. If the NY Times article bare out, I can see why they are so intent on voter suppression. Finally, let this be your mantra, "Remember 2000."

Great. Now the freepers are pretending to be disenchanted Dems.

They changed their verbiage, but not the message, which is DEMS ARE LOSING.

bull oney

hey, i don't know how on here to make requests- but it would useful for us folks who are just visitors here to have people mention in a subject heading what they are refering to - ie whats a freeper and what is all these other things like bull oney?

bruhrabbit --
the saturday ny times (hardly mere internet rumors) has an op ed piece specifically FOCUSING on the US looking the other way on Pakistani nuclear development, so the idiotic deal has been cut. As I told people in 2000, if I hated America the way some at WBAI.org radio sometimes seemed to (not as popular now to) I would be gung ho for Bush. That is even more true than I had thought. 9-11 was "Christmas for Tories" and we are in a new era of national security centered right wing politics that sacrifices national security on things like Pakistan (and the Iraq war) to pursue their agenda AND STILL get the support of most voters concerned about national security.
The reason for the latter is that "being strong on defense" is more about persona and cachet than it is about what will make the country safer. At any rate, the Sat oped column, which it looks like you probably had read, is based on interviews with Musharraf and other key players and is highly recommended to all.
I still think that they way the press AND the Kerry campaign have never pointed out the fatuousness of the flipflop spin, or rather the flipflop mantra, is an indication of the machine's agenda to insure another four years of Bush, the idea being to "send the world a message" that America stands behind its new "muscular" posture. Hence the sniping at Dean from ostensibly progressive directions, so there wouldn't be the idea that the Democrats were uniting around an unabashed 'antiwar' candidate, followed by the meticulously unchallenged flipflop mantra. The press nailed McCain for criticizing Kerry's language on Iraq when Bush himself had used the same language, but missed Bush's own distorted use of the 'I voted for it before I voted against it' blooper.
The Kerry campaign has failed to confront this issue, what I call "Dukakissing". Merely pointing out Bush flipflops isn't enough; the fatuousness of it as a characterization of Kerry has already solidified, so that challenging it too little too late (as in the fatuous Willlie Horton ads) is insufficient. It requires confronting Bush on this as demagogy at the debates -- which is the sort of thing I would expect Kerry to NOT do. If Kerry loses, his failure to address that will have been decisive, and predictable, as I raised in an email to "Ask George" (Stephanopolous) well before the DNC.
Machine agenda is a better predictor of elections than the polls; the only problem here is why the ruling machine would want to follow a political course so obviously inimical to rationally considered urgent national security needs.
CLOUDY

I always found Ruy's recent "explanations" of poor Kerry polls to be very biased and weak, but not being an expert on polling, I could not tell the validity of his criticism of polls with a larger sample of Republicans than the percentage of Republicans in exit polls in the last election. Then I came across Ruy taking exactly the opposite position back in June when an LA Times poll showed Kerry up 7 points in a poll with 38% Democrats and 25% Republicans in the sample. Enough said -- the lack of intellectual honesty is clear -- so let's take everything Ruy says with a huge grain of salt.

Ruy's earlier statement:

"There are ample grounds for thinking there is, in fact, a surge toward the Democrats and their positions and away from the Republicans and their positions among the broad electorate. A growing Democratic party ID advantage is a logical consequence of that surge, since party ID does not remain stable as political conditions change....Conclusion: there is no good reason to ignore the results of this poll (unless you're Matthew Dowd, of course, who has his own reasons for doing so)."

Smooth:

Sorry, I meant to say the Democracy Corp poll is always within +/- 1% of the average Bush and Kerry numbers, which would obviously be +/- 2% on the spread. It's really remarkable in that no other poll consistantly lands right in the middle of the average like that. And everyone else has an outlyer of at least +/- 5% vs. the average spread to their credit.

Gallup, for example has been off the average spread by as much as 7 points in Kerry's favor and more recently, 8 and 9% in Bush's favor (so much for the gold standard). Same goes for Time. They've been off in Kerry's favor by as much as 7 points an Bush's by 8. ABC/WP, ARG and Newsweek have all been off by 6% and I'm pretty sure everyone *except* Democracy Corps has been off by as much as 5%.

Right now, I've got the race at Bush 48, Kerry 44. That's right about where it's been for about the last two weeks. The 3-way race numbers -- which are the numbers I use when I can get them -- from the two most recent Democracy Corps polls were Bush 47, Kerry 45. That's right down the center of the average and within it's +/- 1% hypothetical MoE.

The one before that came in at Bush 48%, Kerry 45 at a point when the average stood at 49/44. And previous to that, the Democracy Corp polls had an average deviation of zero from the average spreads at the times they were taken.

Interesting KC; I didn't know that. What's good for the goose, to be sure, is certainly good for the gander!

Reminder on Posting Policy

EDM does not host comments that attack other participants as individuals, rather then ideas. This is true even if the personal material represents only part of the content.

Posts that do not appear can be revised to remove such personal material and resubmitted.

This comments section is open to all points of view so long as the commenters discuss issues and ideas and express themselves in civil form.

EDM is currently evaluating other forum systems that allow for the creation of specific communities of interest (e.g. activist democrats). Look for more information after the election.

EDM Staff

RE: The LA Times June poll.

It's funny. The average of all polls taken June 1-15 was Bush 44, Kerry 44. The LA Times poll taken June 5-8 came in with a 6 point lead for Kerry (42/48). A CNN/USA Today/Gallup Poll taken June 3-6 came to exactly the same conclusion (43/49).

A Harris poll taken the following week (6/8-15) also showed Bush ahead by 10 (51/41) among likely voters. But I don't recall anyone really beating up on Harris or Gallup at the time.

Cal D, I think one reason people are beating up on Gallup now is because we know how they conduct their polls now. Even if Republican turnout is higher than in 2000, there is no way it will be 7% more than the Democratic turnout. Therefore, Gallup polls that show Bush running far ahead are viewed as hopelessly flawed, and ones that show a dead heat could mean that, in fact, Kerry is actually ahead.