Statistics are used much like a drunk uses a lamppost: for support, not illumination. Vin Scully
Four years ago I wrote a blog regarding my concern with government’s decision to cancel the mandatory long-form census questionnaire. Now actually I always felt that the government should rethink the census. Although the information we get from the census is great, for marketers and business much of the data comes to us too late. There has to be a way in this modern age of technology to speed up the process. I also felt that there were too many questions (and yes some very intrusive and probably should have been omitted.)
But what I would have expected the government to do when making changes to the census is consult the users of the census. The data generated by the long-form census questionnaire provide decision-makers in the public and private sectors with a deep and rich set of facts about Canadians, facts that are reliable at the local, regional and national levels. There is no question that this change in the census would have a major impact to those of us who work in the field of marketing. The Marketing Research and Intelligence Association (MRIA) wrote a letter to the government at the time of the decision which highlights the concerns. here is what they said.
“The concern about the Government’s decision to cancel the mandatory long-form and introduce a voluntary new National Household Survey (NHS) is that the response rate will likely be substantially lower and the resulting data less robust, given that hard-to reach segments of the population will not likely be included among respondents. The experience of survey researchers and social scientists is that those in lower-income groups, ethnic minorities, and the wealthiest citizens are least likely to answer questions voluntarily.
This they claimed would lead to skewed data and doubts about the accuracy of information that is relied upon by public policy and business decision-makers. Without robust census data, it will be exceedingly difficult for governments to respond effectively to shifting patterns of need in the populace or to introduce changes that provide the greatest value for money. One particularly problematic outcome of the elimination of the mandatory long-form questionnaire would be the eradication of the only reliable, national source of information on aboriginal educational achievement.
The long-form data are also combined with other survey data to compute and extrapolate rates for key social and economic indicators. For example, local health authorities can use their own survey data combined with census data to calculate rates of health service utilization and many other vital statistics.
The new National Household Survey – may be biased on important dimensions such as income, education, housing status, and many others. Researchers across the country, working on projects in all areas of public policy and business decision-making, will have no data with which to correct for these biases. It will also not be possible for researchers to compare numbers from census to census, and analyze trends.
So here we are in November 2014 and what has been the results of replacing the long form with a voluntary questionnaire?
A recent Globe editorial provides some valuable information regarding the elimination of the mandatory long-form census in 2010 and replacing it with the voluntary National Household Survey.
The government claimed the compulsory form was an unwarranted intrusion into Canadians’ privacy. It was a bizarre and unsupported explanation. Statistics Canada warned from the get-go that replacing the long form with a voluntary questionnaire, even one containing the same questions, would undermine the quality of the data. The chief statistician, Munir Sheikh, resigned over the issue.
The warnings were prophetic. The compulsory long-form census in 2006 had a 93.5 per cent response rate. The voluntary one in 2011 had a 68.6 per cent response rate, even though more surveys were sent to more homes. When the 2011 data were released, they came with prominent warnings about contamination due to “higher non-response error.” Information gathered about more than one quarter of all Canadian communities wasn’t released because too few people in those places filled out the voluntary form. Aboriginal communities were particularly underrepresented.
Think-tanks, economists, scientists and academics in Canada and around the world have dismissed the 2011 data as fatally flawed. It can’t be compared in a meaningful way with the 2006 data, because they were gathered using different methodologies. Vital research projects on issues like income, unemployment and poverty that require long-term data have been compromised. And Statistics Canada can’t provide an accurate picture of how Canadians are faring, relative to 2006, since the 2008 economic crash.
Do politicians have any idea how valuable, and essential, the information in the former long-form census was for planning and delivery of health and social services across the country? Let alone the enormous array of marketers who need data to make business/marketing decisions.
Hopefully one day we will see the return of the mandatory long-form census. It only makes sense and marketers like me will rejoice.
Let me know what you think.
There are lies,damned lies and statistics – Mark Twain