Many public sector marketers and communicators are very reliant on POR (public opinion research). Many marketing and communication decisions are based on this type of research. But what happens when POR become unreliable? What alternatives do marketers and communicators have to POR? Have we become too addicted to POR?
Now we recently witnessed polling inaccuracies in elections in BC. Alberta, and most recently in Ontario, also we have seen some significant inaccurate polls in the USA.
IAN McGUGAN points out in the Globe that the most notable losers in Ontario’s election were the legions of researchers who failed to foresee the Liberal romp. There is no question that mistakes were made e.g. Ipsos Reid and CTV polling indicated that the 3 parties were in a neck and neck race. Well the results show that they could not be more wrong. The final results show that the Liberals took 38.6 per cent of the popular vote to 31.3 per cent for the PCs and 23.8 per cent for the NDP. Hardly a neck and neck race!
The Canadian prognosticators have company in their embarrassment. American political leader Eric Cantor’s pollster is reported to have told him only days before the recent Virginia Republican primary that he had a 34-percentage-point lead. In fact, the House majority leader wound up losing by 11 points. Yikes!!!
Clearly something is awry in the state of research. I know that this is political polling but is this part of a slippery slope that may have implications for all public opinion research. Will this not have impact on how trustworthy POR is for marketers and communicators?
Clearly the surveying techniques that worked so well in the past are stumbling, and there’s no shortage of explanations as to why.
Frank Lutz, a Republican pollster, writes in a New York Times op-ed that Mr. Cantor’s pollster was guilty of “quantitative malpractice”. Mr. Lutz’s unusual modesty about polls is actually a bridge to his larger point – that polling is both art and science.
As the article in the Globe points out public opinion is too subtle and nuanced to be gauged by simple “yes” or “no” questions. That is no doubt true, but it sidesteps the question of why so much public opinion polling have had so many high-profile pratfalls recently.
Public opinion has been sophisticated and subtle for decades. It’s only recently, though, that pollsters seem to be messing up with regularity.
Ian Mcgugan points out that one possibility is that polling itself has become more difficult. As more and more people become cell phone-only users, it is harder for pollsters to get a representative sample of the population through the time-honoured practice of calling up people’s homes. Add in call-blocking technology and caller-display options, and the number of people who are willing to offer up their opinions to a survey taker shrinks even further. This is worrying!
Stuart Soroka, a professor of political science at McGill University, says the cost of traditional phone polling is now too expensive for many purposes. “Four out of five phone calls don’t work” in a typical survey, he estimates, “but each one of those non-responders involves a cost” to the polling company in the form of staff time and salary.
Pollsters are responding by turning to online surveys. The polling company recruits people by promising money or points if they agree to be part of a continuing panel of respondents. It can then conduct endless online surveys among those panel members at next to no incremental cost. The potential problem with this approach is that the pollster still has to take the results from the panel and adjust them, using sophisticated statistical techniques, to reflect trends in a wider population. That is no simple matter.
There are other challenges to researching public opinion especially when it comes to political polling but I think that public sector marketers and communicators need to start relying less on POR. It has become too convenient to make decisions on public opinion research but obviously we need to find alternatives.
I certainly hope the research industry in Canada and the USA come up with solutions to the challenges of inaccurate polling but in the interim “buyer beware”.