Yukon’s Liberal Opposition is denouncing a recent poll that shows them straggling far behind the Yukon Party and NDP in public opinion.
At the heart of the dispute is DataPath Systems’ use of an online pool of respondents, which is used alongside randomized telephone calls.
Mike Travill, the Liberals’ campaign manager, worries the online respondents mean the survey sample is no longer randomly selected. “That invalidates the poll,” he said.
Travill compared DataPath’s methods to polling shoppers as they enter a mall - convenient, but statistically unreliable.
Nonsense, says pollster Donna Larsen. “They’re totally wrong. They don’t understand their methodology at all.”
The company contacts all respondents with a randomly dialled telephone call. Once a survey is complete, the respondent is asked if he or she would like to receive an email alert for future surveys.
If they agree, respondents join a pool of more than 1,000 Yukoners - only some of whom are contacted during each survey, said Larsen. That’s at odds with Travill’s assertion the company uses “the same 300” respondents “every time.”
As old respondents drop out of the pool, new ones are added. Wednesday’s political poll was roughly split “50-50” between previous and new respondents, said Larsen.
DataPath puts quotas on the number of respondents from Yukon’s rural communities to ensure they’re properly represented in each survey.
Travill suggests the polling firm is slanted against the Liberals. As evidence, he notes that DataPath’s poll preceding the last federal election gave the Liberal incumbent a 20-point lead. In the end, Bagnell lost to the Conservatives’ Ryan Leef.
But if the company’s sample had an anti-Liberal bias, it doesn’t make sense that it would wildly overshoot with one survey, then undershoot in the next, said Larsen.
“That rules out sample bias.”
She insists the federal poll was an accurate reflection of public opinion at the time it was collected, a week and a half before the election. But, during the intervening period, Liberal support collapsed across the country.
Larsen also noted another inconsistency with the Liberals’ line of argument. They blame DataPath for handing the Yukon’s federal seat to the Conservatives, by making left-leaning voters, who may have backed Bagnell had they known it was a tight race, feel at ease to vote their conscience for the NDP or Greens.
Yet Liberal Leader Arthur Mitchell declared this week he wasn’t worried about the latest DataPath survey - he said polls don’t sway public opinion.
“Make up your mind,” said Larsen.
Major polling companies around the world use online panels, said Larsen. And she’s right.
But their use is controversial.
Big Canadian pollsters such as Ipsos Reid and Leger Marketing use online panels to conduct their surveys. Yet others, such as Nanos Research, stick strictly with telephone calls.
Polling companies have struggled over the past decade with declining response rates. More Canadians are giving up landlines in favour of tougher-to-reach cellphones. Refusal rates are rising.
Online polling is meant to help fill that gap.
But the question of whether self-selecting, online respondents muck-up a poll’s randomized selection has also concerned national pollsters, following the federal election’s result of a Conservative majority, which few predicted.
But the principals behind Ipsos Reid called this concern “completely ridiculous” in a public letter issued in February.
“Nobody in the world does true random samples for political research. And, nobody has ever conducted a random sample for a political survey in Canada, ever. True random samples take too long, are too expensive, and are overkill for the task at hand.”
Likewise, Larsen says her surveys have almost always proven accurate within several percentage points.
Canada’s industry group for pollsters, the Market Research and Intelligence Association, recommends that its members refrain from reporting margins of error on polls that use internet samples.
That hasn’t stopped companies big and small - including DataPath - from doing so. The association is currently reviewing this policy.
It’s important to remember the limitations of a poll, said Larsen. They’re best considered “a snapshot” of public opinion, and are by no means a prediction.
“Is this the way an election will be two months from now? Not a chance. This is today. This is how people feel right now.”
“Smart” parties use polling data to develop strategy, said Larsen. Insecure ones shoot the messenger.
“If you’re putting out false information, what does that say about who you are? I’m not going out and lying about who you are. It’s very frustrating.”
Contact John Thompson at
johnt@yukon-news.com.