« Kerry, Bush in Statistical Tie in Pair of Iowa Polls | Main | Strategy Notes:
John Belisarius

What the Public Really Thinks About Iraq - And the Challenge Facing Kerry »

Race Still Tight in Swing States

Kerry leads Michigan LV’s by 4% in FOX News/Opinion Dynamics Poll (Sept. 21-22)

Kerry lags by 2% Nevada RV’s in CNN/USA Today/Gallup Poll (Sept. 18-21)

Bush leads by 3% Ohio LV’s in FOX News/Opinion Dynamics Poll (Sept. 21-22)

Kerry up 5% among Pennsylvania LV's in FOX News/Opinion Dynamics Poll (Sept. 21-22)

Bush ahead by 10% West Virginia RV’s in CNN/USA Today/Gallup Poll (Sept. 17-20)

Bush up by 6% among Wisconsin RV’s in ABC News Poll (Sept. 16-19)

Comments

What do we make of Wisconsin? This looks like a point of concern.

Rick-

I don't know what to make of Wisconsin. As with national polls, I'd point out variability. As best I can tell, there are 10 polls of Wisconsin released 9/12 or later:
ABC, Bush +6
Moore (GOP), Bush +3
Badger, Bush +14
TNS, Bush +10
Zogby, Kerry +2
Mason-Dixon, Bush +2
ARG, tied
Strategic vision (GOP), Bush +6
Gallup, Bush +8
Rasmussen, Bush +2

The median for those is Bush up by 4.5%. That's probably not a terribly bad guess. That's quite a recoverable lead, and then there's turnout.

Just keep saying the mantra...it's a close race. Let's work hard to get Kerry the victory. And say it if the polls show us ahead, tied, or behind.

Another note on Wisconsin. In the ABC poll, the breakdown was 35% R, 29% D, 36% I/Other. In the Moore (GOP) poll, it was 39% R, 39% D, 22% I/Other. In 2000 it was 32% R, 37% D, 31% I/Other.

So...it's back to that issue we keep debating. Has there really been a sizeable shift to the Republicans in registration, or is something else going on. I lean toward the latter, but I don't think we'll know till election day.

A bit OT, but I thought I'd post a version of a comment I made on the Mystery Pollster site, regarding his argument that Party ID was quite unstable, and therefore should not be used as a weighting factor. In fact, as I point out, the results he quotes from studies deal ONLY with increases in the number of INDEPENDENTS:

--------------------------------------------------------
One significant weakness in your [i.e., the Mystery Pollster's] logic, as you present it, is that you don't get at the nerve of Ruy's argument, which assumes that the DIFFERENCE between party identification for Republicans and that for Democrats has been stable.

The particular research you've quoted demonstrates something VERY different, namely that the number of INDEPENDENTS has varied a great deal. This is completely consistent with the idea that the difference between Republican PID and Democratic PID are stable. And, indeed, it stands to reason that a convention might polarize independents, drive them equally in both directions into the hands of the ideologically nearest party.

Indeed, one of the very studies you quote says that the largest amount of variation in Party ID comes from movement in and out of independence. It seems very plausible to believe that politically potent events or circumstances might in general simply polarize independents, leaving the difference essentially intact.

If you have any evidence that the DIFFERENCE between Rep and Dem Party ID varies as wildly as we've seen across these polls, that would be very good.

Otherwise, you've proved exactly zero.

Penn and Wisconsin are two battleground states with among the worst-maintained lists of registered voters, with many erroneously purged, according to a recent study by Scripps Howard News Service.

Some other items pulled as well from Joshua Kurlantzick's eye-popping piece, "2000, The Sequel", in the current (Oct) issue of The American Prospect, on how the Help America Vote Act (HAVA), passed by Congress in 2002, may make things even worse than 2000:

(Link is: http://www.prospect.org/web/page.ww?section=root&name=ViewPrint&articleId=8544)

*Rep. Rush Holt (D-NJ) introduced an amendment to HAVA in May '03 that would require touchscreen machines to have a paper record. Bob Ney, Chair of the House Administration Committee, has not allowed Holt's legislation out of committee. House and Senate Republicans have offered similar "smokescreen" legislation--that would take effect by the 2006 election.

*There has been major WH footdragging on getting up and running the Election Assistance Commission (EAC), authorized under HAVA to disburse money to states to upgrade voting systems, issue guidelines, and hold hearings to help make voting as fair as possible. EAC was supposed to be set up within 120 days of HAVA's passage but did not even have office space until April of this year. Congress gave the EAC $2 million out of a proposed $10 million budget for '04 and Bush asked for less than half of the $1 billion proposed in HAVA for overall election-reform efforts.

*Under HAVA all first-time voters have to have valid ID. NY PIRG says it is illegal for local election boards to tell poll workers not to accept a student ID as proof. U Wisc and Penn State Students for Kerry, do you copy? One study showed that in NY election officials in only 18 of 45 counties even understood voter-ID requirements.

*Some states simply do not count "provisional ballots", intended to give those wrongly denied the opportunity to cast a vote on Election Day via the usual process the chance to have their votes tallied if their wrongful exclusion is established. HAVA offers no national guidelines for counting these provisional votes. This raises just a few questions for state and local election officials and newspaper editorial boards, among others.

*After 2000, Jeb Bush rejected the recommendation of a bipartisan task force that he make state and county election supervisors nonpartisan. Shortly afterward, the Republican-controlled Florida legislature made all top election officials appointees of the governor.

I have not seen these specific items, several strongly suggesting the GOP does not want the sorts of problems that turned up in 2000 and since to be fixed, reported by any of the media bigfoots. Another shocker.

Well Rick, if you assume that Gallup and ABC oversample Republicans in their state polls (as they did in their national polls) then these Bush leads are probably not as large as they seem, or even extant at all.

Nice cherry picking of polls favorable to Kerry; I don't want to rain on the parade, but I'm waiting to see if you have synopses of the AP Poll showing GWB over Kerry by 7 (RVs) and the CBS poll showing GWB over Kerry by 8 (RVs).

Fyi, An analysis of the various polls:

Poll Methodology - A 2004 Guide


There has been intense interest in the polls this year, and the recent disagreement about the range of position has only highlighted discussion. Some people like to support a poll with results they like, without any sort of examination about why that poll is different from others. And some reject polls on a charge of outright bias or prejudice, which I can understand, given the partisan comments from supposedly objective people like John Zogby and Larry Sabato, but I must caution the readers to be careful to consider the evidence before accepting or rejecting a poll.

Let’s start with the obvious; more information is better, especially if it is relevant to how the numbers were driven. By relevant, I mean two things: The information should show valid evidence to support the poll’s main conclusion, and the information should be consistent with past polls, so that trends and historical benchmarks may be seen. To that end, I discovered that in terms of methodology, we can separate the polls into three broad types – the polls which provide demographic internal data, the polls whose questions show mood in the main issues, and those polls which refuse to provide internal data.

The best way to find out how the polls developed their methodologies, is to look for that information. Some publish their methodologies at the bottom of their poll releases, others are so proud of their methodologies, they wrote up special articles to explain their process. Others did not have their methodologies handy, but responded when I asked them how they did their polling. And others, well, they were neither forthcoming nor cooperative, and that speaks for itself. This article allows you to get to know the polls all over again, this time starting form the inside. I figure, this guide will help you figure out for yourself, whose word is worth listening to, and who is nothing but hooey. I am listing the polls in alphabetical order. All telephone polls referenced employ Random-Digit-Dialing (RDD); RDD is used to pre-select Area codes and exchanges, then use a randomizer to select the last 3 or 4 digits, depending on the poll. When I say ‘pure’ RDD, I mean that the respondent poll is new; some polls appear to use an initial pool of respondents for future polling, and I will note this where it shows up. All references to “Margin of Error” reflect a standard 95% confidence level by the polls. When I reference ‘NCPP’, I mean the National Council on Public Polls, who published guidelines for demographic weighting and internal responsibility, which they expect their members to follow. Another national group for pollers is the American Association of Public Opinion Research (AAPOR), but they appear to be much smaller, and have looser standards than the NCPP. It’s worth noting, though, that neither the NCPP nor AAPOR appears to have any deterrent in their policies; there is no specified penalty for not meeting their standards, nor any formal auditing authority. That, of course, is one reason I’m doing this review.

ABC News/Washington Post: This agency uses a call center for its polling. The subcontractor at present is TNS of Horsham, Pa. The poll is performed by telephone, calling roughly 1,200 “randomly selected adults nationwide”, from which self-identified registered voters are polled for the report’s information. The respondent poll is pure RDD for each poll. ABC/WP says their Margin of Error is +/- 3 points. The ABC/WP poll cites results by gender, race, age, education, and sometimes also by income groups. Regarding the weighting of their poll data, ABC says, “Final data are weighted using demographic information from the Census to adjust for sampling and non-sampling deviations from population values. Respondents customarily are classified into one of 48 cells based on age, race, gender and education. Weights are assigned so the proportion in each of these 48 cells matches the actual population proportion according to the Census Bureau’s most recent Current Population Survey” The weighting appears to be in line with NCPP guidelines.

American Research Group: ARG’s methodology for national polling was not published, but from their primary polling in the spring, ARG stated “The results for this tracking survey are based on completed telephone interviews among a statewide random sample of likely primary voters in [the state]. Telephone households are selected by adding random digits to telephone exchanges selected in a random probability-proportionate-to-size sample of the state. Respondents in the telephone households are randomly selected and then screened for voter registration (including intent to register on or before primary day) and likelihood of voting in the primary.” On September 22, ARG released a nationwide compilation of state polls, which revealed they use a 53-47 weighting of women to men, and a party weighting of 41.4% Democrats, 35.5% Republicans, and 23.1% Independents. These do not conform to 2000 or 2002 exit polling, nor the 2000 Census, and are not in line with NCPP guidelines. ARG’s respondent poll may sometimes be pure RDD, but at other times appears to be a reserved poll from previous polls, in order to track possible opinion shifts in the same pool. ARG does not explain whether this is the case, and did not respond to a request for clarification.

Associated Press/Ipsos : The Associated Press Poll is conducted by Ipsos-Public Affairs. The poll is a telephone poll of randomly selected numbers, with a sample of roughly 1,500 adults nationwide, produced between 1,200 and 1,300 registered voters ,whose responses produce the poll’s information, along with a smaller number of self-described ‘likely’ voters, who are defined as voters who voted in 2000 and are 8-10 on a likelihood of voting (1-10 scale), or who did not vote in 2000, but are ‘10’ this year. The respondent poll is pure RDD. Ipsos weights its poll, but does not detail the breakdown in its Press Release or Questionnaire, though some demographic information was released in their latest poll. Instead, Ipsos concentrates on the trend of questions measuring a degree of support on key issues, such as Overall Leadership, Foreign Policy, the Economy, Domestic Issues, and Terrorism. Ipsos’ reported Margin of Error is +/- 2.5 points for adults, +/- 2.7 points for registered voters.

Ayres McHenry: “Ayres, McHenry, & Associates belongs to the American Association of Public Opinion Research, and the American Association of Political Consultants, where Ayres serves as a member of the Board of Directors.”
That’s all they have. Nothing about weighting or breakdown of samples, which is contrary to the AAPOR’s written Code of Professional Ethics and Practice. Ayres McHenry did not respond to a request for more information. As they are a Republican-sponsored firm, this agency does not provide any supporting evidence for its statements, and should be not generally be considered a reliable indicator of voters’ true opinion.

Battleground Poll : The Battleground Poll uses two firms for its interviews and analysis; the Tarrance Group and Lake, Snell, & Perry. The Battleground Polls started in 1991, so they have some history to track. The Battleground Poll did not publish its methodology, but the Tarrance Group was kind enough to answer a request for more information (hat tip to Brian Nienaber); Lake, Snell, & Perry did not respond to a request for information. Overall, the Battleground Poll uses a “stratified” sample design, and pure RDD for respondent pooling. Battleground explains their weighting thusly; “quotas are set by state and by gender based on voter registration statistics or voting age population, where applicable. Prior election turnout statistics are also used to set these quotas. For the 2004 Battlegrounds, we have been applying a weight to the demographics of race and party identification. Race is weighted to White=80%, African Americans=10%, Hispanics=6%, and Other races=4%. Party identification is weighted to even with Republican=42.3%, Democrat=42.3%, and Independent=15.4%.” Note that the demographics are consistent with 2000 Census, and the party weighting presumes parity. Battleground release the demographic breakdown of their respondents, but does not publish polling results by demographic groups. Like the Associated Press, most questions reflect a trend of national mood on the major issues. The sample used is for 1,000 registered voters who self-describe as “likely”. Tarrance estimates their Margin of Error to be +/- 3.1 points.

CBS News, and CBS News/NY Times : Telephone interviews with adults in the continental United States. Phone calls are randomly generated within system-selected area codes and exchanges. CBS goes to some length to brag about their methodology, and you know what? They should. While CBS and the NY Times tend to over-weight the poll in favor of Democrats, their demographics not only follow NPCC guidelines by matching the 2000 Census, they also publish their demographics regularly, and have for the last five years. If you don’t like their numbers, at least you can take them apart to see where they came from, and this with no subscription fee or doubletalk to hide the trends. Obviously, the “60 Minutes” guys and Dan Rather have nothing to do with the polling at CBS. The polls are consistent and complete, and frankly, very impressive in their detail and history. CBS/NYT generally calls about 1,000 adults in each survey, with around 78-80% as registered voters. The respondent poll is pure RDD. Their cited Margin of Error is +/- 3 points

CNN/USA Today/Gallup: This poll uses random telephone interviews, with around 1,000 adults on average, around 76-80% registered voters responding. Announced Margin of Error is +/- 4 points. Demographics details are available, but generally only to Gallup subscribers. The weighting matches NPCC guidelines. The respondent poll is pure RDD.

Democracy Corps : This Democrat-sponsored polling agency (James Carville is one of the owners, that should tell you a lot) uses Greenberg, Quinlan, Rosler (GQR) for it’s interview sampling. They do not respond to queries, and do not explain their methodology. Note that NPR uses the same sub-contractor for their polling. This agency should be recognized as partisan and biased by design.

Fox News/Opinion Dynamics : Opinion Dynamics Corporation conducts a national telephone poll 1,000 self-described ‘likely voters’ from random contacts. Fox includes internal details by gender and party affiliation, but not race. Their website says “Generally, Fox News poll results are not weighted. The national probability sample, if conducted properly, should accurately reflect national attitudes. However, particularly because the survey is often conducted over only two nights (limiting the opportunity for callbacks), some demographic deviation is possible. Opinion Dynamics Corporation has a constantly updated database of demographic information about the national samples and, if one should deviate significantly from past averages, weighting is used to bring the sample into conformity with other samples”. In English, that suggests that Fox will weight some polls, but not others, which is a strike against consistency. There is no information to determine whether or not the respondent poll is pure RDD or pre-selected. The same website admits that Fox weights their polls by gender, 47% Men and 53% Women, even though this is not in line with NPCC guidelines, Census data, or consistent with Exit Polls from past elections. Neither Fox News nor Opinion Dynamics responded to a request for clarification.

Gallup: The gold standard of opinion polling. Gallup presents demographic and trend data for every poll they have anything to do with. Whether on their own or in combination with other groups (the CNN/USA Today/Gallup poll, for example), Gallup insists on consistent procedures to insure consistency. Their respondent poll is pure RDD for the Presidential Trial Heats. Gallup weights their polls in line with NPCC guidelines, and releases internal data on race, gender, party affiliation, age, region, education, economic strata, union/non-union, veteran/non-veteran, religious preference, and sexual orientation. Gallup polls are random telephone interviews, with around 1,000 adults on average, around 76-80% registered voters responding. Announced Margin of Error is +/- 4 points. The down side to the demographics details, is that they are generally only available to Gallup subscribers. With a 69-year track record, Gallup is able to show an impressive record for their predictions and tracking.

Harris: The Harris Poll is one of the oldest polls in the nation, after Gallup. For some reason, though, Harris not nearly as successful as their older sibling, and I think I know why. They like to ask questions, but they don’t answer them. The Harris Poll is a random telephone poll, as most of the polls are, interviewing roughly 1,000 adults nationwide in each poll, and producing around 80% registered voters from that pool. The respondent poll for their telephone interviews is pure RDD. Harris also has an Interactive Poll, but there is no established benchmark for the accuracy of the Interactive poll, nor do they explain their methodology for the Interactive poll; I suspect it is similar to their telephone poll, since they produce similar results, but cannot confirm this possibility. Harris weights their responses by the NCPP guidelines, for age, gender, race, education, number of adults, number of voice/telephone lines in the household, region and size of place, in order to “align them with their actual proportions in the population”. Harris cites a +/- 3 point Margin of Error. Unfortunately, when it comes to releasing their information, well, they don’t. I’ve been part of the Interactive polling as a respondent, and even then, they are parsimonious with hard data. From the lack of response I’ve had from them, I get the strong impression they are all about chasing the corporate patrons, and only put out the occasional public poll to keep their name in the press. OK, that’s their right, but other polls can chase sponsors, without looking like the Information Age version of Ebenezer Scrooge (pre-Ghost Visit). My advice? Ignore these guys, unless they start putting some hard data behind the headlines in their releases.

Investor’s Business Daily/Christian Science Monitor: The Christian Science Monitor is a long-established, well-respected name, but they have no experience in polling. Investor’s Business Daily is a publication I’d never heard of, until they showed up with their releases. They began polling in February 2001, using something they called “Indices” for various factors they considered important. The Indices are developed using random nationwide telephone interviews with approximately 900 adults each month. The respondent pool appears to be pure RDD, but there is no confirmation. They seem very impressed with themselves. I’m not impressed. Since they don’t release much hard data at all, and pretty much diddly to support their claims, and their ‘indices’ don’t seem to follow any established method for determining public opinion, my opinion of IBD is rather like the old Monty Python skit about an especially bad brand of wine: “This is not a poll for enjoying, it’s a poll for laying down, and avoiding”.

Investor’s Business Daily/TIPP : See Investor’s Business Daily/Christian Science Monitor, above.

LA Times : The Los Angeles Times wants to be a big-time newspaper. I write it that way ,because while they want the glory, they don’t seem to feel as though they should have to earn it. The LA Times uses telephone interviews nationwide, of at least 1,500 adults, using pure RDD sampling. They produce a subset of registered voters, at around 77-80% of the adult number. The LA Times says the samples are “weighted slightly to conform with their respective census figures for gender, race, age, and education”, which may or may not be in alignment with NCPP guidelines. The Times’ announced Margin of Error is +/- 3 points. The Times releases details by party alignment and gender, and appears to over-weight Democrats.

Marist College Institute for Public Opinion: Marist is a college up in New York, who produces polls on the Presidency. Marist does not release a lot of details, however, including their methodology. Their website notes that “MIPO adheres to the current Code of the National Council on Public Polls and the AAPOR Code of Professional Ethics and Practices”, which at least suggests they use the 2000 Census for their weights, although this does not speak to party alignment or sampling methodology. They haven’t put anything out for a long time, so it may not matter, but if they pop up again, the fact that they don’t back up their statements with supporting data should be a warning sign.

NBC News: NBC News uses a sub-contractor for its polling. Princeton Survey Research Associates (PSRA), whom NBC contracts for some of their polls, was kind enough to provide specific details by email at my request (hat tip to Evans Witt). The July poll where Kerry picked Edwards for his running mate, NBC used Braun Research, Inc. for the interviews, using a sample designed by Survey Sampling International, LLC. The sample was relatively small (504 registered voters), but used pure RDD. NBC says that “statistical results are weighted to correct known demographic discrepancies”, which “parameters came from a special analysis of the Census Bureau’s 2003 Annual Social and Economic Supplement (ASEC) that included all households in the continental United States that had a telephone”. NBC estimates their Margin of Error at +/- 5 points, due to the smaller sample size compared to normal poll pools. NBC does not release demographic breakdowns of votes in their polls.

NBC also examined their response rate, which is an often overlooked factor in poll analysis. NBC states “the response rate estimates the fraction of all eligible respondents in the sample that were ultimately interviewed. At PSRAI it is calculated by taking the product of three component rates:
Contact rate – the proportion of working numbers where a request for interview was made – of 47 percent
Cooperation rate – the proportion of contacted numbers where a consent for interview was at least initially obtained, versus those refused – of 32 percent
Completion rate – the proportion of initially cooperating and eligible interviews that were completed – of 99 percent
Thus the response rate for this survey was 15 percent.”

Newsweek : Like NBC, Newsweek has also used Princeton Survey Research Associates International (PSRA) to do their polls, and again, they are pure RDD telephone interviews, of roughly 1,000 registered voters nationally. They seem to weight by NCPP guidelines. Newsweek did not publish the response rates from respondents, but they are very good about including the demographic response in their releases, including party support, gender, non-whites, geography, and by age groups. Newsweek reports their Margin of Error to be +/- 4 points.

NPR-POS/GQR : NPR uses a sub-contractor for their polls, Greenberg Quinlan Rosner Research (GQR) and Public Opinion Strategies (POS) for National Public Radio (NPR). GQR also does work for the Democracy Corps firm, indicating a loose possible relation between the two polls. NPR presents its results for “likely voters”, defined as “registered voters, voted in the 2000 presidential election or the 2002 congressional elections (or were not eligible) and indicated they were almost certain or certain to vote in 2004”. Pure RDD was used for the pool selection. GQR interviews around 800 voters, and reports a Margin of Error at +/- 3.5 points. NPR does not release demographic responses, and did not respond to a request for further information.

Pew Research Center: Pew conducts its research using the same sub-contractor as NBC News and Newsweek, Princeton Survey Research Associates International. PSRA performs a pure RDD pool of respondents, interviewing a national sample of adults by telephone, for roughly 1,000 or 2,000 respondents, of which 78-80% are registered voters. As with other PSRA work, it appears NCPP guidelines are followed for weighting. Pew publishes an extensive report, reflecting not only national mood on key issues, but demographic response by party response, gender, age, and regional groups. Like the CBS News poll, I find the political weighting a little bit off, but I can’t complain about their work ethic or standards. Pew is very consistent, and is particularly useful for measuring shifts in demographic trends. Pew estimates their overall response Margin of Error is +/- 2.5 points, and +/- 3.5 points for registered voters.

Quinnipiac University: This school in Connecticut performs polls on “politics and public policy in New York, New Jersey, and Connecticut”, as well as the occasional national poll, such as the Presidential Election. They use pure RDD with their on-campus Institute to contact roughly 1,000 registered voters or more nationally by telephone, over a five or six day period. They release results by overall weighted response, party affiliation, by gender, and by black/white racial group responses. Quinnipiac does not detail their methodology for weighting demographics, and did not respond to a request for more information. Quinnipiac estimates their Margin of Error at +/- 3.1 points, less than that if the sample size is larger.

Rasmussen: Rasmussen Research performs more national polls than anyone else right now, with a poll taken every day. Unfortunately there is no methodology released to the general public. Not the size of the respondent poll, nor whether the sample is randomly developed, there is no weighting method cited, and there is no breakdown of respondents’ responses, which might allow analysts to compare Rasmussen’s results with anyone else. Scott was kind enough to respond to a request for more information, but only to say that he is very busy right now, and will answer later. By the time this article went to publishing, no information on his methodology had been provided, so I must regard this poll to be unsupported in its claims. There is no evidence to confirm whether or not the response is weighted, and if so how, or whether any standardized methods are employed in this poll.

Survey USA: Survey USA is a unique polling agency. On the one hand, they do not perform national polls on the Presidential race, yet they poll in almost every state on the Presidential race. Survey USA has been around since 1992, and they love to punch out state polls Survey USA and Zogby are in a horse race for who will put out the most state polls this year. I also included Survey USA in this list, because they have strong opinions about polling methodology, and they printed an extensive article, far too long for me to copy here, so read it here.

Survey USA uses pure RDD for their telephone polls,usually between 500 and 1,000 self-identified “likely voters”. While Survey USA does not define “Likely Voter” for their methodology, they do take pains to emphasize that their polls “are conducted in the voice of a professional announcer. SurveyUSA is the first research company to appreciate that opinion research can be made more affordable, more consistent and in some ways more accurate by eliminating the single largest cost of conducting research, and a possible source of bias: the human interviewer.”. This suggests they use an automated voice, which is certainly original. Survey USA opines that human error in pronunciation, diction and unintended inflection leads to flaws in the voter response. Survey USA uses weighting in line with NCPP guidelines. The reports are specific to states, but lack demographic breakdowns or votes by demographic group. Survey USA estimates their Margin of Error to be +/- 4.5 points.

It’s interesting to note three additional comments made by Survey USA. First, Survey USA makes a point of the need to verify results, disparaging “call-in” polls as unscientific, and strongly suggests Internet polling is about as useless as the “call-in” polls. Survey USA notes that “only a few large research companies employ their own telephone interviewers. Almost all small pollsters, and even some of the nation’s most prominent firms, outsource all of their interviewing to a third party.” This appears to imply that contractors are not as valid as independent firms, but from my review of the polls, a number of the contractors are equal or superior to established polls which are better known, Princeton Survey Research Associates International in particular.

Finally, Survey USA mentioned a practice I had heard about, but which is impossible to prove: “curbstoning”. This is, as Survey USA explains, where a pollster “may not interview a respondent at all, but just make up the answers to questions”. It’s rare, says Survey USA, but the problem is, unless you check your pollsters carefully, you really don’t know if they are putting down the real response or not, or whether they are putting down what they think their boss wants to hear. I agree that I think this practice is not very common, as I believe poll clients really do want honest numbers, so they can see where they stand, but it is important to recognize that this problem exists.

TIME : TIME magazine hires a contractor, Schulman, Ronca, & Bucuvalas (SRBI) to perform their polling, including the interviews. They average roughly 1,000 registered voters, and 850-900 self-described “likely voters”. Pure RDD is used for the contact. SRBI follows NCPP guidelines for demographics and weights the part affiliation, as follows: Likely voters, 34% Republican, 35% Democrat, 22% Independent. Registered voters 31% Republican, 32% Democrat, 26% Independent, which shows a rough parity. TIME does not publish results by demographic group response, but to measure the mood in key questions, and reflects trends by asking the same consistent questions. SRBI estimates their Margin of Error is +/- 3 points for registered voters, +/- 4 points for likely voters.

Wall Street Journal: The Wall Street Journal does not do its own polling, but co-sponsors polls with other groups. Earlier this year, the WSJ was partnered with NBC News, but is now partnered with Zogby. No additional information was available from the Journal.

Zogby: Back in 1996, pollster Zogby hit the bullseye in predicting the results of the Presidential election. In 2000, they were close again, though their aggregate error tied them with 5 other national polls. In 2002, Zogby appeared to show a lean in favor of the Democrats, and he was way off in his mid-term election predictions. This year, at the end of the spring, John Zogby actually came out and predicted John Kerry would win the election, which appeared to indicate his bias had reached the point of full-blown partisanship against the President, reflected in a growing number of opinions made out of personal preference, rather than on the evidence. Zogby’s refusal to show his work, only magnifies the apparent distortion of his results.

Zogby runs two polls; a telephone poll and an Interactive Internet poll. Unlike almost every other poll, Zogby’s telephone poll is not RDD. Zogby describes his list as follows: “The majority of telephone lists for polls and surveys are produced in the IT department at Zogby International. Vendor-supplied lists are used for regions with complicated specifications, e.g., some Congressional Districts. Customer-supplied lists are used for special projects like customer satisfaction surveys and organization membership surveys.
Telephone lists generated in our IT department are called from the 2002 version of a nationally published set of phone CDs of listed households, ordered by telephone number. Residential (or business) addresses are selected and then coded by region, where applicable. An appropriate replicate1 is generated from this parent list, applying the replicate algorithm repeatedly with a very large parent list, e.g., all of the US.
Acquired lists are tested for duplicates, coded for region, tested for regional coverage, and ordered by telephone, as needed.” Zogby notes that “regional quotas are employed to ensure adequate coverage nationwide.” That is, Zogby takes pains to insure that his respondent poll is not random.

As for his weighting, Zogby states “Reported frequencies and crosstabs are weighted using the appropriate demographic profile to provide a sample that best represents the targeted population from which the sample is drawn from. The proportions comprising the demographic profile are compiled from historical exit poll data, census data, and from Zogby International survey data.”

In other words, Zogby uses his own polls to drive some of his demographic parameters, a practice not approved, much less recommended, by either the NCPP or the AAPOR.

All in all, Zogby’s habit of confusing his personal opinion with data-driven conclusions, his admitted practice of manipulating the respondent pool and his demographic weights, by standards not accepted anywhere else, along with mixing Internet polls with telephone interview results, forces me to reject his polls as unacceptable; they simply cannot be verified, and I strongly warn the reader that there is no established benchmark for the Zogby reports, even using previous Zogby polls, because he has changed his practices from his own history.

Except for some specific polls whose practices earned remarks for their excellence or a distinct lack of it, I have tried not to rank or grade the polls. I would also recommend the reader read through the polls himself, to determine which is most thorough in its work and results. But hopefully, this guide will help sort through who is chasing the money, and who is serious about their work.

The last 2 times Zogby polled Wisconsin (Sep 17th and Sep 20th), they found Kerry ahead by 2.

Part of the problem in Wisconsin results from the poor governance by Democrats as Mayor of Milwaukee and Milwaukee County Executive, alienating goo-goos and giving suburban Republicans stories to frighten their children. We're not seeing the movement toward Democrats in the Milwaukee suburbs that shows up very clearly in the Chicago, Philadelphia, and New York City suburbs over the past 8 years.

I've lived in Madison for 27 years, and I would not be surprised to see Bush win here if the national election is very close. But if Kerry winds up winning nationally by 3 or 4 points, Wisconsin will very likely remain Democratic.

ABC and Gallup put Bush ahead in Wisconsin and West Virginia? That means it's a dead heat.

Loooking over the post above from Smooth Jazz:

Interesting that so many polls conducted with such similar methodologies should be coming up with such wildy different results. Although, I do note that a lot of the apparent discrepencies between polls that we have seen lately tend to be more in the "LV" subsamples than the unadulterated RV segments and the article really doesn't get into LV screens to any meaningful extent.

So I would say this article tends to back up Mr. Tiexera's assertion that we should give more credence to the RV data wherever available and poke the LV data with a long stick before going anywhere near it.

Smooth Jazz-

You do not give the source for that long article you posted. I gather it's from someone on the right given (a) pointing out Democratic-related firms, but not Republican-related firms; and (b) various snarky comments about Democrats.

Makes me wonder about the validity of the analysis overall.

I will note one thing that stood out to me. There's a 15% response rate for the NBC survey. I'd guess that's fairly typical. If so, it really makes me wonder about these surveys. Some of the 85% non-response is doubtless because of random numbers that are not in fact residences. But with such a high non-response rate, small variations in people choosing not to respond would seem able to produce large variations in poll results.

I read some negative comment on RDD in an academic piece sometime in the last week or two, which also makes me wonder whether that is, in fact, a sufficient methodology for getting a random sample. It's 9 pm here on a Friday night, so I don't think I'm going to go searching right now. Perhaps over the weekend.

And who's cherry-picking? I'd note that I used all the polls I could find about Wisconsin. It's the sort of argumentation that you engage in that continually reminds me why I can't vote Republican. And I came pretty close in '00, when I thought about going for McCain, till Bush decided to trash him.

The interesting thing about the ABC News poll is that it shows union household figures for Bush and Kerry, and something else I've not seen before: "Contact by the campaign" numbers.

The ABC News poll discusses the degree of partisanship in Wisconsin itself, as opposed to general arguments based on the 39D-35R numbers usually discussed on this site. The ABC News memo (see the section titled "Party" on p. 6) discusses how partisanship on election day has varied widely in Wisconsin -- from +6 for Dems in 2000 to only +1 for Dems in 1996. Average it out (I wish ABC News had provided partisan numbers for 1992 and 1988 also, but oh well...) and that's only +3.5% for the Dems.

Not much of a cushion. Look at the numbers: It makes Bush's lead in the highly Republican Milwaukee suburbs +25, and Bush's lead in the Northeast (Green Bay) +22. It's heartening to see Kerry polling extremely strongy in the Southwest (Racine over to Walworth county). But it's shocking to see Kerry polling LOWER in Milwaukee proper than Kerry is polling in the Southwest.

The ABC News poll says Kerry is up four points over Bush among union households. I'm hearing numbers that say that in south and west Milwaukee, as much as a quarter of all union households remain "undecided." In some heavily union precincts, as many as 30%-35% of union households remained "undecided"! In an election with a virulently anti-union president (as Bush is), it is astonishing that union households would be anywhere this ambivalent about John Kerry as an alternative to George W. Bush.

Worse: Union turnout for Al Gore in 2000 proved decisive in pushing Wisconsin into the Democratic category. The ABC News poll points out that Gore won union voters by 16 points. John Kerry's numbers are nowhere near that.

What makes me worried about Kerry's campaign in Wisconsin is the following: "Registered voters are six points more apt to have been contacted by his campaign than by John Kerry's, 25 to 19 percent. And six in 10 of those reached by Bush's campaign support him, while Kerry's supported by fewer than half of the Wisconsin voters his campaign has personally contacted."

I don't know what "personally contacted" means. Telephone? Face-to-face? In on Labor Day, many voters were saying that they had already burned out on telephone contacts, and had received almost no face-to-face contact (canvassing). Indeed, only progressive groups (League of Conservation votes, unions, etc.) had done face-to-face canvassing.

In one way, this result can be taken to mean that the Bush campaign is contacting random votes and finding massive support for Bush, while the Kerry campaign does the same and finds little support.

But, another interpretation could be that the Bush campaign is contacting its base. Meanwhile, the Kerry campaign is contacting its base but also attacking the Bush people. We're given no analysis of whether the Kerry contacts are changing minds.

But the idea that the Bush campaign is more active in Wisconsin than Kerry's makes me very nervous. (Will the Dems blame former Kerry campaign co-chair Matt Flynn for abandoning the Kerry ship for a Quixotic run at Congress, if Kerry loses?)