One More Time: Polling Averages Avoid Outlier Traps
In his WaPo post "Hey, reporters! Watch out for polling outliers," Jonathan Bernstein sounds a cautionary note, which should resonate with political writers across the spectrum. As Bernstein explains:
...A couple of high-profile polls last week showed drops in Obama's approval rating, including a New York Times/CBS survey that had him dropping to 41 percent approval. Yet several other polls showed Obama staying in the same general range as before, or even gaining; overall, it was clear that the NYT/CBS poll was an outlier.
Bernstein goes on to recount how a panel on "This Week" used the poll to launch into a discussion about President Obama tanking in the low forties, "ignoring that other polls last week had him at 46, 47, 47, 48, 50 and 50 percent approval." Bernstein responds:
This is really sloppy work. It's one thing to have overreacted to the Times poll when it was released on Monday -- that's bad enough -- but there's no excuse for missing all the other polls by the end of the week...Hey, reporters and pundits! Always, always, use the excellent polling averages provided by Pollster (currently 47.7 percent approval) and/or Real Clear Politics (currently 47.5 percent). Both of them have convenient charts showing current trends, and lists of recent polls so that one can see at a glance how a particular survey fits in with the overall pattern...We're going to be hit with a gazillion general-election polls in the coming months, which means that about one-twentieth of that gazillion is going to be an outlier even if everyone is doing their best work...
End of Political Journalism 101 refresher course. As Bernstein concludes, "Be prepared for those outliers, and be ready to discount them rapidly. Anything else is just bad reporting."