Prospective political polling has become less and less effective over the last two decades, in large part because the polling is conducted by whores who violate every scientific convention that social scientists developed to make sure that polling would indeed produce high quality results. None of the statistics are legitimate because the basic assumption of having a random sample of the population in question has been violated. We’re not just talking about the tip-of -a-finger in an orafice, no, this is more like a jackhammer-up-the-ass type violation. It’s hilarious to hear reporters with 85 IQ’s talk about these shit polls having a 5% or 6% “margin of error”, as if they passed undergraduate statistics or something. We don’t know what the margin of error is, because we don’t have a random sample. Even adjusting the standard error to reflect the piss-poor response rates wouldn’t help us. This could be possible, using techniques similar to what is done for complex sampling design, but since we don’t know the composition of the non-respondents, we can’t be sure of our initial estimates of the mean or its standard deviation. There is no technical fix for bad data.
Most of the political opinion polls being conducted have response rates of under 10%, probably less the 5%. Nobody reports this. That means that 90%-95% of the randomly selected targets for interview are not interviewed. Usually, this is because of laziness on the part of the firms conducting the interview. What do they care? They’re getting paid to have “results” not to have quality findings. Opinion polling is no longer a cottage industry, it’s big business. Also, many of the principals, almost uniformly media conglomerates (but also politicians and parties–who should have an interest in getting high quality data), want results NOW! Not tomorrow. Not next week. Now. As if people change their political choices on a whimsy every day. You can’t get accurate results NOW. A hardworking team may be able to get good results in 1.5 weeks, but even that is pushing it. It’s better to wait for good data, than to collect shit. It’s like collecting mushrooms. Some people may not want to wait for that perfect fungus to emerge, but instead decide they’ll just cook up the cow pattie.
Worse yet, indifference towards high quality data is infiltrating the social sciences. Gee, if we can get a $250,000 grant and do a survey with 1,500 respondents, that’s good, right? No. Not if your study has a response rate of 21%! I was horrified to open up a recent issue of a journal and see a feature series on a “study” which only managed to get cooperation from 21% of the targeted subjects. Gee, do you think the 21% who did cooperate are like the rest of the population, or do you think the 79% who didn’t cooperate are more normal. Several of the authors had the audacity to compare their findings to higher quality studies with 70-80% response rates, and claim that because they made some minor shift in question wording or coding that the results of the shit study were more meaningful than those produced by the high quality source! Wow. I’m impressed. Is it hard to be that full of shit? Or, maybe if they’re stupid it just comes naturally. Or, perhaps they’re just whores, feeding off of grant money from agencies that want quick answers in line with their interests…..
Earlier this year I published a paper on this topic in Sociology of Religion, where I was looking at potential religious biases created by differential response rates in very high quality opinion polls. Even when the response rate is 70% (as it is in the General Social Surveys and National Election Surveys), if some groups are more likely to be in the 30% than in the 70% our statistics will not converge to the population parameters no matter what the sample size (that, of course, is the definition of bias in samples). What I found is that fundamentalist Christians are underrepresented in sample surveys, which is consistent with some findings from prospective sample prediction failures. My favorite occurred when I was in graduate school. We all knew that Harvey Gantt was going to kick Jesse Helms ass! There was an 8% gap in the polls! High quality stuff, done at UNC! Of course, Helms trounced Gantt by about that same margin. Media sources have consistently framed that election as one which turned on Helms’ last minute racist advertisements, as if somehow racist rednecks in North Carolina were going to vote for Gantt (who was African American), but then changed their minds in response to these vile ads. Yeah, right. The real answer is that even the highest quality polls had a 60% response rate (the UNC polls), and the Helms supporters hung up the phone as soon as they knew that it was some liberal, communist, homosexual from Chapel Hill wanting to know if they were in support of turning the nation over to the Africans.
Still, I’m bummed that moving between ex-wives and ex-fiance’s after graduate school I lost my “Another Liberal Communist Homosexual from Chapel Hill for Gantt” T-shirt. I never found my “Impeach Reagan” T-shirt, either….