Has Stats Canada Screwed Up job Numbers?

#ldnont has never had a month where 4000 people left the labour force. This month's data? 5000 gone. It's an error. @LFPress @KevinLabonte

— Mike Moffatt (@MikePMoffatt) May 8, 2015

Same day we got new warnings about bad data in the census.

How To Use The New Census Data

It involves holding your nose:

The only way Statscan was able to publish the 2011 NHS data was by remarkably dropping its quality standard by doubling the acceptable global non-response rate.

And accepting incomplete questionnaires:

What exactly did Statscan consider an acceptable response to the 2011 NHS questionnaire? Apparently, enumerators had been instructed to accept the long form with as few as 10 of 84 questions answered.  What the standard was for the 2011 NHS and how it differed from previous long-form Census are important questions that, to date, have not been answered. Given that StatsCan lowered the data quality standard by doubling the acceptable non-response rate, one suspects a similarly dramatic change to the acceptable questionnaire completeness rate for the 2011 NHS relative to prior year long-form Census.

And, as far as I know, we are still talking about data at the level of  Census subdivisions (CSDs). That is, municipalities.  At this level:

One way to demonstrate the dramatic difference this change in data quality’s had is by looking at the published 2011 NHS data for Census subdivisions (CSDs). Of 5252 CSDs in the 2011 NHS, data for 3439 was fit for release, while data for 1813 was suppressed, almost all for non-response, using the post-2011 NHS data quality standard.  However, using the pre-2011 NHS data quality standard, only 994 of those 5252 CSDs, or 19%, would have been fit for release, while 4258 would have been suppressed. By comparison, of 5418 CSDs in the 2006 long-form Census, data for 4534, or 84%, was fit for release, while data for only 884 was suppressed.

Below that, down to the tract/dissemination area level, data has still not been released.  

Stats Canada Regrets...

OTTAWA – In a highly unusual move, Statistics Canada has delayed the release of a major set of data from the 2011 National Household Survey by a month.

Data on Canadian income, earnings, housing and shelter costs were to be released Wednesday morning, but instead will now be unveiled by the federal statistical agency on Sept. 11. The National Household Survey replaced the mandatory long-form census.

Census manager Marc Hamel said that the release involved complex calculations based on hundreds of formulas. But over the weekend, he said the agency’s experts found a problem while doing some last-minute checks.

The folks at StatsCan insist that this flub-up has nothing to do with the switch to a voluntary long-form (now called the National Household Survey).  Nevertheless, economist Stephen Gordon brings the thunder o' disapprobation:

It's entirely reasonable to be skeptical about numbers generated by an untried and untested methodology in the absence of technical documentation. Many people - me included - were prepared to give StatsCan the benefit of the doubt. They no longer deserve it.

Meanwhile, as far as I know, Statscan has still not released data down to the census tract/dissemination area for any of its NHS-based reports, and last time I called had not given any date for so doing.

National Housing Survey: News From The Senate

In yesterday's QP, Leader of the Government Marjory  LeBreton admits that the voluntary NHS with replaced Stats Canad's mandatory long form census survey requires improvement, and suggests a possible way forward:

This was the first time, honourable senators, that there has been a voluntary long-form census. Clearly, there are some areas that require improvement and, obviously, by the time it is necessary to conduct the next census, the areas of concern will have been addressed.

And:

We have the mandatory short form as well. There have been suggestions by experts that a few more questions could be added to the short form.

This idea, that in future iterations some questions on the voluntary NHS might be transferred to the mandatory short-form, is developed a little more fully later on:

I did indicate that this was the first time there has been a voluntary long-form census. Some groups have indicated that they would like to see the data be more conclusive. These considerations will be taken into account when Statistics Canada and the government prepare the questions for the next long-form census. People will then be more used to the long form. Whatever questions they put on the mandatory form will address the concerns of the groups that feel this does not produce the data they require.

I don't know how much insider knowledge LeBreton has with respect to the thinking within the government's HOC caucus, but shuffling questions to the short form is an interesting option.  However, it is counterproductive according to the reasoning that has been employed on this issue to date.  Move a question to the short form and you are now invading the privacy of 100% of the population rather than the 25% required for the old long form. Which is to say that, in so far as the government wants accurate data it can't protect privacy, and vice versa.

Canada In The News! Canada Long Form Census Makes New York Times

Canada recently replaced its mandatory long-form census with a voluntary survey — and now lives with the sorry results. To try to get an adequate level of response, the voluntary survey was sent to one in three Canadians instead of one in five, which increased costs. The response rate plunged anyway, from 94 percent to 68 percent. In a staggering one-fourth of Canadian communities, not enough people responded to make the data usable

More stuff I've written on the long form census scandal here.

The Census And Your Privacy: The Courts Speak

When Sandra Finely received her long-form census papers in 2006, she refused to fill it out because she felt that the requirement violated her right to privacy, and was therefore unconstitutional   She was taken to court for violating the Statistics Act, lost, and requested an appeal.

Her arguments were heard by the Saskatchewan Court of appeals and last week they issued their ruling.  She lost again.  To get a flavor of the court's decision, a fairly representative passage:

[39]   Thus, the question is not whether Ms. Finley had an expectation of privacy or even a reasonable expectation of privacy in dictionary terms.   The question must be linked to the overall context of the case.   In this case, the question must be cast in these terms:  whether a reasonable person would expect to have privacy in the information requested by the 2006 Long Form Census, which the government wishes to collect exclusively for statistical purposes to aid it in implementing sound and effective public policy, with no criminal or quasi-criminal repercussions flowing from the disclosure of such information, and with the specific information collected being ultimately generalized and “delinked” from the individuals being required to so disclose.  The trial judge answered this critical question negatively and the summary conviction appeal court judge found no error of law, mixed fact and law or fact in her conclusion.

Sandra Finley was the poster girl for that tiny minority of Canadians who support the federal government's elimination of the long-form census.  I think the court ruling makes clear why their concerns, where not driven by simple paranoia, are off-track.  We are supposed to be living in a knowledge-based economy, and it is essential that the government gather accurately and disseminate widely/cheaply such data as is necessary for that economy to function.  Period.  End of story.  If it takes an hour of Ms. Finley's day every few years...well, that's the price we pay for living in the modern world.

Community Level (Census Tract/Dissemination Area) Data In NHS--There Or Not: UPDATE UPDATED

*Sigh*.  A 2nd call to Stats-Can makes the quote below seem somewhat plausible

Today’s first release of the 2011 National Household Survey (NHS) data confirmed what we had previously written. It appears the data quality was so poor that Statscan decided to release neither data at the dissemination area (DA) nor the census tract (CT) levels. These are more commonly referred to as ‘community-level’ data.

At the moment Stats-Can is offering community level materials from the census as custom requests; they plan to release the data more widely later on, though have not given a precise date for this.  Stats-Can staff have also informed me that  census tract/dissemination level data from the the NHS is not yet available, and they have not been given a date upon which it will be made available.

Stats Canada NHS Survey: The First Big OOPS

I spent months arguing that the changes to Canada's long form census would degrade the data from it.  Today we are starting to see proof.  Although most of the damage will come at the lower geographic levels, there appears to be evidence that some of the head-line numbers will be effected as well. From the analytic document "Immigration and Ethnocultural Diversity in Canada"

Top source countries for newcomers

 The NHS results showed that the Philippines was the leading country of birth among people who immigrated to Canada between 2006 and 2011.(3) 

However, go to footnote (3).  It reads:

(3) This result was not in line with administrative data from the Department of Citizenship and Immigration Canada which provides the number of recent immigrants by their country of birth settling in Canada each year. A number of factors could explain this difference, such as the effects of sampling, response patterns, and under or over estimation of certain groups of recent immigrants in the NHS.

So in effect Stats Canada is saying that they cannot reliably tell us which country provided Canada the largest share of immigrats during the 2006/2011 period.  A rather glaring flaw, if you ask  me.

The Tory War On Brains

The cancellation of the  long form census has already degraded the nation's primary knowledge product:


OTTAWA -- Prime Minister Stephen Harper's cancellation of the long-form census has started to take a toll on Statistics Canada's data.

The agency released its final segment of the 2011 census last week, focusing on languages. However it included a big warning that cautions data-users about comparing key facts against censuses of the past. That's because the cancellation of the long-form census forced Statscan to move language questions on to the mandatory short-form.

Statscan's lead analyst says repositioning the questions likely prompted people to answer them differently, especially when it came to questions about mother tongue and languages used at home.

As a result, Jean-Pierre Corbeil says the census is probably not accurately capturing the mother tongue of large numbers of immigrants who came to Canada over the past few years.He also says it's likely overstating the number of people who have English as their mother tongue.

Note that the problem with the data  noted here is not even among those most often flagged by critics of the government change.  Therefore expect unforeseen problems to crop up along side of the foreseen ones.

In any case, not content, the Tories have decided to go after the NRC:


OTTAWA - Nearly 100 of Canada's top researchers and scientists are facing layoffs in what the NDP science and technology critic says is just the "tip of the iceberg" in what appears to be the Conservative government's plan to slash funding for basic research and focus instead on research that's industry-driven.

"There's a much larger game afoot but it's being rolled out in a really stealthy way," said Kennedy Stewart, the NDP critic for science and technology. "When we look back in a couple of years we'll see that it really is part of a larger plan and it will probably have an impact on our international standing."

Which is to say that the indices that measure international performance in basic research will all show Canada ranking lower.  A plot to render the nation incapable of doing anything but hewing wood and hauling bitumen?

Fool Me Once, Shame On You; Fool Me Twice, And I Must be Lorne Gunter

The latest tranche  of census information, like the material released back in February, is from the still mandatory short form.  It is the long form that was compromised by the government's decision.  Lorne Gunter, who also covers the global warming controversy and would, you would think, possess at least a single smidgen of  statistical knowledge, still cannot tell the difference between long and short forms.  Naturally, he writes for the Toronto Sun.

Today's Census Numbers Are ALL From The Short Form

...which is still mandatory, meaning that Lorne Gunter's column about the voluntary long form is entirely irrelevant, and you don't need to read it.

Census Info Already Degraded

Stats Can will take any piece of crap you're willing to send them:

OTTAWA - Census officials are settling for incomplete long questionnaires in the final push of the 2011 collection period.

A longer version of the story here.

Gov. Radio Ads On The Census

I've heard the one they're running on Q107 a dozen times now.  It refers to a "mandatory census form", but make no distinction between the long and short form.  Which means that the gov's not stupid enough at least to encourage non-compliance with the (now voluntary) long-form. 

Too little too late, but a sign somebody realises the original decision (back-story here, though a bit scrambled) was a dangerous fuck-up.

Out Of The Mouthes Of Berniers

Mad Max, in a 2006 letter to Liberal MP Bryon Wilfert:

These questions continue to be essential for providing the information needed by governments, businesses, researchers and individual Canadians to shed light on issues of concern to all of us — employment, education, training, transportation, housing, immigration, income support, pensions for seniors, transfer payments, aboriginal issues and many more

[...]

All of the questions are designed to meet important information requirements that would be extremely difficult to satisfy efficiently from other sources. In fact, every question goes through a rigorous consultation, testing, review and approval process, including that by Cabinet

It's almost embarrassing to link to Maxime's blog post of July 23rd, when all the folks mentioned in the first paragraph have become "special interests". Its almost embarrassing to note his Radio-Canada appearance where he cavalierly states that the information requirements--which he formerly thought were so difficult to satisfy efficiently from other sources--could be created --SHAZAM!!--through research paid for by the private sector.

Take Note, Nervous Nellies

In Canada, if you argue about putting abortion into your maternal rights plan, or if you argue for a science-based policy in regards to the long-form census, or if you argue for maintaining the long-gun registry, the polls all turn your way eventually.

Progressivism is in our blood, is part of our DNA. All you need to do is persevere.

Census Deja Vu

The U.K. Census debate looks quite a bit like ours, minus the Libertarian crazy talk. In any case, a good overview of the issues involved should Canada go the U.K route (although, to be fair, the U.K. is a long way from deciding just how their new, modernized census will look in the end).

A Few Average Men, Redux

The Tories have scared up a few, in fact precisely two, "regular guys" to come talk up their changes to the census at the INDU meeting this morning. Or at least I am assuming that talk radio ranter and Fox News North wannabe David Ruthorford will be on-side. In addition, Lawrie McFarlane, editorial writer for the Times-Colonist, has made his feelings clear. And that seems to be it from the schedule that I can see. Even the gal from B.C. civil liberties seems pro-long form.

Update: Looks like the Tories have found another talk show guy; Lowell Green will be appearing.

Update: Tories also found James Henderson, whose bitches turned out to be with the still mandatory agricultural census, and Joseph Lam, the vice-president of the Canada First Community Organization, to speak on behalf of regular folk.

Tom Flanagan On The Census Long-Form

In which he walks back some of his previous criticism of the government's policy, but not really that much:

We need some rational debate, maybe even a government task force, on the proper shape of the census in the electronic age. Statistics Canada still relies on 20th-century technology (mailed questionnaires) as well as 19th-century technology (interviewers at the door) to gather information. How about some 21st-century technology? If we can create an electronic voters list, can we not carry out at least part of the census electronically?

Fine. We do need a rational debate. Even maybe a task force. But until we get all that, can we keep an unemasculated version of the long form? Oh, and lets have that task force look at the cost of census products. How come when I visit U.S. statistics sites I encounter data, but at Statscan I immediately start running into price lists?

Several European countries have abolished the traditional census. They have systems of compulsory registration and personal ID cards that can effectively substitute for the census but are politically unacceptable in North America. Yet, Britain’s coalition government plans to abolish its registry and ID cards immediately, and the census in 2021; at the very least, we should be watching to learn from this experiment.

Yeah. Watch the British experiment closely. They were going to a European style system, but rejected a European style ID card. Plans for the new regime were to have been ratified almost a month ago. But so far...nothing. I'm speculating, but the U.K. coalition may be finding it a bit hard to square that circle.

A Few Average Men...

Mr. Clement has already dismissed the controversy as one that only occupies “some of the elites in our country.” His party appears ready to continue this theme in committee, with a string of appearances from average Canadians.

Actually, if you look at the latest polling data on the census--it tells you what a wonderful country this is: 50% of Canadians are in the elite!

Because There Is Such A Thing As Truth

George Jonas' latest on the census long-form plumbs the depths of innumeracy:

At least, that’s the mantra. Self-selection skews data. No one explains why coercion doesn’t skew data the other way, or why doesn’t it bother statisticians if it does, or why data can’t be adjusted for self-selection when it can for most other things.

The answer to the last bit, about adjusting for self-selection, is of course that data can be adjusted for self-selection where you have census data to correct it with.

But its the first part that's most interesting. Jonas is suggesting, obliquely, that there is nothing but skew--skew this way, skew that way, the further implication being that these directions are "left" and "right", politically, and that statisticians tend towards socialism.

But of course that's not accurate: to any question about the demographics of an area, there is a correct answer, and you get that answer when you have managed to extract (via the mandatory long-form) a truly random sample from within that area (ie a sample that is unskewed). Underlying Jonas' argument, then, is an appeal to truth relativism.

Which, you would think, is weird coming from a Conservative, but perhaps more common than you might think.