« Welcome to the New Donkey! | Main | Zogby Poll of Likely Voters Shows Bush 46%, Kerry 44% »

Adventures in Likely Voter Land

It has not escaped my notice that many people are puzzled as to how exactly polls go about determining "likely voters" (LVs). There's a good reason for this: polling firms or sponsors rarely put much effort into explaining, clearly and precisely, the mechanics of how they select these LVs.

So, as a public service, here's how they do it. Let's start with Gallup. According to David Moore of Gallup:

Gallup asks each [RV] respondent seven LV screening questions, and gives each person an LV score of 0 to 7. [Assuming a turnout of 55 percent], the top 55% are classified as likely voters.

In practice that typically means all of the "7"s--given full weight--plus some proportion of those with lower scores (usually the "6"s), who are weighted down so that the size of the likely voter sample matches the projected turnout for the year (apparently 55 percent this year). All other voters are discarded from the sample.

What are the Gallup likely voter questions? Unfortunately, the exact questions and their wording are not released by Gallup along with their polling data, but the questions apparently involve past voting behavior, interest in the election, intention to vote in the election and knowledge of things like the location of the local polling place.

That's how Gallup does it. What about other organizations--do they select likely voters in the same way? Nope, they don't. CBS News doesn't use a cut-off model, where low-scoring respondents are thrown out altogether, but instead includes everyone in their RV sample, in some form, in their LV sample. They do this by asking respondents a series of voting-related questions and then assigning each respondent a weight based on their score on these questions, from very high weights for high-scoring respondents to very low weights for low-scoring respondents.

Finally, by far the most common way is simply to ask a few screening questions and then terminate the interview with those respondents who give the "wrong" answers. Or only one question; some likely voter screens are as simple as asking an RV how likely they are to vote in the upcoming election; if they don't say "almost certain" or "probably", out they go.

So that's how they get the likely voters in the polls you read about. How do they know that likely voters, months before the election, are actually the voters who will show up on election day? They don't.

Here's David Moore from Gallup again: "We simply do not know, nor can we know, which model is better during the campaign itself. " Exactly. So why does he think the Gallup LV model works so well months and months before the election. Because "if it is the most accurate model just before the election, it is probably the most accurate during the campaign as well".

But that doesn't follow at all. The Gallup LV model could work perfectly right before the election (not that it really does, but that's another discussion) and still be quite a biased instrument earlier in the campaign. Pretty much by definition, Gallup's LVs months before the election are not the same voters as Gallup's LVs right before the election, since voters answer the LV questions differently at different stages of the campaign. And if there is any kind of partisan dimension to "tune-in", so that, say, Democratic partisans or groups that lean strongly Democratic (like minorities) tend to tune in later, that means the LV model will have a systematic tendency to, on average, favor the party (the Republicans) whose partisans or groups tune in the earliest.

Of course, my hypothesis here about Gallup LV bias might be completely wrong. But to evaluate it, Gallup would have to make available the demographics and partisan breakdown of the both its RV and LV samples for the polls it releases plus, ideally, the results (including demographics and partisan breakdowns) of the various screening questions it uses. I'm not holding my breath.

Comments

Thank you so much. I might have to read this a couple of times to soak up the implications but I appreciate your effort (particularly your acknowlegement that this is _your_ take on the accuracy of the process.

At what point do personally begin to focus on the LV numbers? We are still lagging in those and I'd still like to hang my hopes on the RV polls.

I assume there have been studies that track the accuracy of LV models. Could you refer us to some you think are helpful. I understand it must be incredibly complex to judge whether it's the poll taken six months before the election that was inaccurate or if it was the intervening events.

like giving up the secret KFC formula.

Seems to me that Democratic turnout might be a bit higher this year than in prior years. I'm sure Gallup etc. are aware of this; the question is whether they are taking it into account, and to what extent.

Fingers are crossed that we will all be surprised (including the pollsters) when record Dem turnout shows polls to be pretty inaccurate this year.

I'm curious if you could explain why you don't think the RVs suffer the same late tune-in bias? I understand your explanation that LVs early on are questionable, but what makes RVs any better?

And what of states like mine ( WI ) where you can register to vote at the polling place on election day?

> I might have to read this a couple of times
> to soak up the implications

I can sum it up for you:

LV data is useless.

"LV data is useless."

I'm not sure that was the conclusion. It seems to me, once everyone is paying attention it is useful to see which voters are motivated (and can find their neighborhood polling place)

Interesting stuff. I could see how, if I'd been polled, Gallup might reject me as a "likely voter," although I've voted in every primary and general election since I've been eligible. I had no idea, for example, where my local polling place was until I got my registration card in the mail yesterday, since I just moved a few months ago. My wife isn't even registered at our new address yet, so she's even less "likely" than I am, even though there's no chance on earth that she won't vote.

Thanks for the analysis. Here in Iowa, the Democratic Party is pushing hard to get supporters to vote by absentee. People don't need to know the address of the polling place if they are mailing in the ballot. Also, in my precinct the polling place has moved since 2000. A bunch of hard-core Democrats have asked me where we will be voting this year. Gallup would code these people as non-LVs, but they will certainly turn out in force. As the election approaches they will figure out where they need to go.

It would seem that if their turnout prediciton of 55% is wrong , and the turnout is higher like 57-58% as others are predicting (though 55% is higher than the last election) than that would mean that their LV numbers should be closer to their RV numbers.

That could be better for us, but we would really have to know what and in what order the screens are and how the numbers were assigned in order to really know if it was our voters or their voters who are higher up in their screen.

Where do they get their pool of people to poll ?
What is the possibility of having a 'tainted' pool of respondents (where the pool doesnt have proper minority representation or maybe tends to favor the more affluent, or the south or whatnot) ?

I've always wondered, because I usually note, but I have never been polled (although I have taken many surveys for product / market research type stuff)

I know Zogby lets you sign up to be polled (but I think thats only for their interactive polls or something) but isnt there something wrong with asking to be polled ??

You make several really great points. The question is, in this election which is so polarized already, on which many more people plan to vote than normal, how are the polling companies and institutions factoring in those who fall in the RV column but not the LV column?

It occurs to me that

Sorry, hit the wrong button.

It occurs to me that the majority of these RVs who aren't LVs but plan to...V...will be the always-bad-at-voting 18-35 block. Since they trend Democratic, how many points does that translate to? Of course, not everyone who claims such a strong reaction - be it positive or negative - to this administration will, in fact, vote, but turnout will increase, and I believe there are few people who will disagree.

So the question becomes several-fold: How much will turnout increase? Where, specifically, will it increase (or, how effective are ACT and MoveOn's GOTV efforts)? And what kind of margin will that increase mean for (presumably) the Democrats?

I'm curious to know what other people think. Let me know via email or via www.politicsnation.com.

-Reid

A LV question that screed for prior presidential voting history could eliminate 18-22 year olds. A demographic that may not be likely to vote, but certainly vote in numbers high enough to swing elections.

Perhaps more significantly, undecided voters tend to break away from the encumbant. They generally have decided they don't want to votew for him, but are not yet convinced his opponent is any better.
Walking the line between sitting things out, and voting for the challenger, these voters also might get filtered out by a LV system

Probably about a month ago, I was looking at the book "Polling Matters" by Gallup's Frank Newport. I just leafed through the book in a bookstore and did not buy it. My recollection, though, is that he listed what the likely voter screening items were.

The formulas for determining LV are probably "proprietary;" like a trade secret, and therefore not for publication. That's neither unusual nor sinister: a polling org wouldn't want a rival polling org to use its formulas.

One big problem pollsters and consultants are having is cell phones. More and more people cannot be contacted by traditional means. This trend is increasing.... How does this affect things?

how come a pollster has never called me? i believe that the younger(pro kerry) voter isn`t being counted because of cell phone use, so bush better have a 3-5 point lead on election day or i would expect a kerry surprise.