Monday, January 28, 2008

Ask Not for Whom the Opinion Polls...

Dr. Christopher Blunt earned his PhD in political science from UCLA before working as a study director and analyst with Market Strategies, Inc. for a dozen years. In 2003, he started his own public opinion consulting practice, Overbrook Research. His analysis has helped shape Republican campaign strategies nationally and in many individual states.

In a very interesting interview with MercatorNet, Dr. Blunt gives readers a view of how polls work, how they can be abused, and how they can be more effectively interpreted. Here's a few excerpts:

...MercatorNet: After decades of polling, how is it that pollsters still get things wrong? Is it more an art than a science?

Blunt: There is a science to drawing a sample that is representative of the population, which allows us to extrapolate from that sample to the electorate as a whole. However, a survey is not a census; there is a margin of error associated with every poll. A larger sample can reduce this margin of error, but nothing can eliminate it. Professional pollsters are usually very accurate in gauging public opinion, as we have developed excellent standards for sampling and balanced question wording.

One of the big challenges we’re facing going forward, however, is the difficulty of interviewing people who have "cut the cord" and are using only cellular phones. Because cell phones usually have charges for incoming calls, pollsters do not include these numbers in their samples. As increasing numbers of people are leaving their landlines behind, we’re finding it increasingly difficult to get younger voters in our samples. We have ways of compensating for it, but this is going to be a serious issue in the future...

MercatorNet: People often like to score points in policy debates by invoking a poll that shows that a majority of American approve of, say, stem cell research or more restrictions on immigration. But opponents can often throw back contradictory figures. Can polls on social mores be trusted?

Blunt: Polls on nearly any subject can be trusted if they are conducted fairly, but polls on contentious social mores perhaps need to be examined particularly closely in this regard. When a major player in such a debate (for example, a biotech company that would profit from embryonic stem cell research, or a business group that profits from cheap immigrant labor) releases a poll purporting to show that that public agrees with their side of the debate, readers should take a very hard look at the question wording. Were arguments or messages given for one side but not the other? Was each side in the debate fully and fairly represented, or was one side more of a straw man than the other? Were respondents simply asked if they agree with favorable statements, or were they given the choice of two sides on the issue?

MercatorNet: What are the three most important questions voters should ask to assess whether a poll is meaningful?

Blunt: First, who is the sponsor of the poll and what is that organization’s motive? Second, were the questions worded and presented in a fair and balanced manner? Finally, how consistent are the results with other polls on the same subject?...

(By the way, Dr. Blunt and his wife are Catholic homeschooling parents of three kids. They also own and operate a five-acre sustainable agriculture operation which raises pastured poultry, dairy goats, purebred Icelandic sheep, heritage turkeys, geese, ducks, and free range laying hens. An intriguing fellow, huh?)