« Whither the DLC? | Main | The "Values Voters" Debate Continues »

Rating the State Polls

By Alan Abramowitz

Here's a rating of 10 polling organizations that conducted polls in multiple battleground states during the 2004 campaign. The rating is based on the final polls released by each organization in 10 battleground states: Florida, Ohio, Pennsylvania, Michigan, New Mexico, New Hampshire, Nevada, Iowa, Minnesota, and Wisconsin. In order to take into account both the number of states polled and the accuracy of the results, the score of each polling organization is based on the number of states predicted correctly minus the number predicted incorrectly. Predictions of ties do not count. Closeness of predictions to the actual margins also does not count and some polls made correct predictions that were pretty far off the mark.

Here's the list, ranked from best to worst:

1. Rasmussen--10 states polled, 9 correct, 1 tie, +9
2. Mason-Dixon--10 states polled, 9 correct, 1 incorrect, +8
3. SurveyUSA--6 states polled, 5 correct, 1 incorrect, +4
3. Zogby--9 states polled, 6 correct, 2 incorrect, 1 tied, +4
5. Research 2000--5 states polled, 4 correct, 1 incorrect, +3
5. Strategic Vision--7 states polled, 4 correct, 1 incorrect, 2 tied, +3
7. FoxNews/Opinion Dynamics--4 states polled, 2 correct, 2 incorrect, 0
7. Los Angeles Times--2 states polled, 1 correct, 1 incorrect, 1 tied, 0
9. ARG--7 states polled, 3 correct, 4 incorrect, -1
10. Gallup--6 states polled, 2 correct, 4 incorrect, -2

Comments

I don't understand the point of these statistics. Wouldn't the pollsters' accuracy be better measured by something like the absolute difference between their prediction and the result, divided by the population of the state in question?

I also think margin should be taken into account. Take Iowa, for example. Bush won 50-49 (a margin of roughly 13,000 votes out of 1.5 million).


http://www.cnn.com/ELECTION/2004/pages/results/states/IA/P/00/

If we look at the polls for Iowa, yes, Mason-Dixon correctly predicted the winner, but it had Bush winning by 5 percent (49-44). Meanwhile, Survey USA called it 49-49. Any system that would award more credit to M-D than S-USA would be off, in my view.

http://www.race2004.net/states.php?state=ia


I want to note that I greatly admired Alan A.'s (aka "The Other Alan") analyses and commentaries during this election cycle. Even generally like-minded observers can have occasional differences of opinion, however.

A useful back-of-the-envelop calculation. Thanks, Dr. Abramowitz.

if hits to misses is the criteria, maybe rankings should be based on percentage correct. here, polling firsm get more points for doing more polls. I don't think it changes things too much, but research 2000 (80% correct) did better than zogby (75% excluding tie), but is lised as +3 to zogby's +4.