« Dems Must Lead Katrina Relief | Main | Labor Day, 2005: Workers Are Unhappy Campers »

Weighting by Party ID: Why to Do It and How to Do It

by Ruy Teixeira

Alan Abramowitz has waded, once again, into the party-weighting controversy with an excellent intervention, “Just Weight!: The Case for Dynamic Party Identification Weighting”, published in the Cook Political Report. In the article, Abramowitz looks at 2005 Gallup poll data (be sure to check out the charts in his article) and finds the following:

Between January 1 and August 7, 2005, the Gallup Poll conducted 24 separate national surveys in which respondents were questioned about their party identification....

...Across all 24 polls, there was an average Democratic advantage of 3 percentage points. This was identical to the average for all Gallup Polls conducted during 2004, indicating that there has been little or no change in the underlying party loyalties of the American electorate. Among these 24 polls, however, the party identification differential ranged from an 11 point Republican advantage on February 4-6 to a 14 point Democratic advantage on June 29-30, a 25 point swing. In some cases, moreover, there were dramatic shifts within just a few days. Between February 4-6 and February 7-10, an 11 point Republican advantage became a 6 point Democratic advantage. similarly, between March 18-20 and March 21-23, a 5 point Republican advantage became an 8 point Democratic advantage.

Hardly believable, eh? And here’s why it matters:

Because party identification is strongly related to political attitudes such as presidential approval, large swings in the proportions of Democrats and Republicans between surveys can produce large swings in estimates of these other attitudes. For example, between February 4-6 and February 7-10 there was a swing from an 11 point Republican advantage to a 6 point Democratic advantage in party identification. At the same time, President Bush’s approval rating fell from 57 percent to 49 percent. Similarly, between March 18-20 and March 21-23, there was a swing from a 5 point Republican advantage to an 8 point Democratic advantage in party identification and President Bush’s approval rating fell from 52 percent to 45 percent. Rather than reflecting any real change in the public’s evaluation of the President’s job performance, these shifts were probably caused by random variation in the partisan composition of the Gallup sample. Such random variation becomes even more problematic before a presidential election because it can affect estimates of voting intentions which, like presidential approval, are strongly related to party identification.

But what to do about this? Is there a way to fix the problem that still allows party ID to change some over time in response to changing political conditions (as it surely does)? Indeed there is: dynamic party ID weighting, a solution flogged by Abramowitz, myself, Charlie Cook and others. Here’s Abramowitz’ description:

A potential solution to the problem of excessive variation in the partisan composition of individual samples is to estimate the underlying proportions of Democrats and Republicans in the electorate by combing the results of surveys conducted over several weeks. This estimate can then be used to weight the proportions of Democrats and Republicans in each sample. By combining several surveys, random variation due to sampling error can be greatly reduced.

[For example, with a] 10-poll moving average....instead of estimating the proportions of Democrats and Republicans in the electorate based on individual samples of approximately 1000 respondents, we are estimating the proportions of Democrats and Republicans based on combined samples of approximately 10,000 respondents.

[This produces results].... much more consistent with the findings of political science research on the nature of party identification in the American electorate: party identification is a stable orientation that changes slowly in response to changes in the political environment. Over the past several months, for a variety of reasons, that political environment has become more favorable for Democrats and it appears that between April and July of 2005 there was a modest increase in the proportion of Democratic identifiers relative to the proportion of Republican identifiers. Because extraneous noise caused by sampling error has been largely removed, this trend is much more evident [with dynamic party ID weighting]....Overall, these results provide strong support for the use of dynamic party identification weighting in public opinion polling.

I agree. You may agree, too. But we don’t take the polls. Gallup and the various other survey organizations do. Let ‘em know what you think.