Revenge of the Nerd

Do you go with your gut? Or dive into the data? One of the most interesting contests during the 2012 U.S. election was not between politicians.

Do you go with your gut? Or dive into the data?

One of the most interesting contests during the 2012 U.S. election was not between politicians. It featured a Who’s Who of big-name political pundits versus a data wonk you’ve probably never heard of, Nate Silver.

The breathless clips on most newscasts told us the race would be “tight” and “too close to call.” High-profile commentators like Peggy Noonan of the Wall Street Journal called for “a Romney win” based on her gut feel for the intense support she felt at recent Romney rallies. Joe Scarborough of MSNBC said “after practicing politics for 20 years, I suppose I would rather be in Mitt Romney’s shoes.” Former Clinton advisor turned commentator Dick Morris predicted a big Romney win, with 325 electoral college votes and just 213 for Obama.

Even former Canadian Prime Minister Brian Mulroney’s gut got into the act. “Romney, by a walk,” he told media yesterday. “You can call me tomorrow morning.”

But these intense intestinal feelings turned out not to be prescient instincts, just bad cases of “pundit tummy” (the political equivalent of Delhi Belly or Montezuma’s Revenge).

Nate Silver and his statistical model turned out to be right. In fact, Silver called the presidential election correctly in all 50 states (assuming Florida stays in the Obama column where it was when this column was written on Wednesday morning).

On the morning of election day, Silver said there was a 90.9 per cent chance of an Obama win. He also estimated the likelihood of Obama winning certain numbers of Electoral College votes, and his most likely scenario was the 332 votes that Obama will get if his win in Florida is confirmed.

How did he do it? The secret is data; more specifically, a technique called “poll aggregation.” Remember that U.S. presidents are elected by the Electoral College. The simple version of how this works is that each state has a number of votes based roughly on population, and the candidate who wins the state gets all of those votes. A 51 per cent win in California, with 55 Electoral College votes, is better than a 99 per cent win in Alaska, which has just three.

Silver’s technique aggregates dozens of polls at the state level to create a deep, data-driven view on which candidate will win each state’s Electoral College votes. This technique is so powerful because it addresses some of the key challenges with traditional polling.

First, it focuses on the state level, not the national polls that make the nightly news. This makes sense since the states are where the Electoral College votes are decided. You’ll remember seeing weird Canadian news stories saying the Bloc Quebecois had 8 per cent support nationally, which is a misleading way of saying the Bloc has 50 per cent in Quebec and zero per cent everywhere else.

The second is margin of error. Because Silver combines literally dozens of polls in a swing state like Ohio, he gets a much larger and more reliable sample size. On Monday, he reported the results of 22 new polls from swing states, with 19 showing leads for Obama. Of six Ohio polls that day alone, five showed Obama in the lead by one to five points, and one was a tie.

Compare that tidal wave of data to a Yukon election, where we only have one or two polls in an entire campaign and the sample sizes are small.

Third is bias. In a world where perhaps only 10 per cent of people actually pick up the phone and answer pollster questions, it is hard for polling firms to accurately survey a true random sample. If you call during the day you get more retired and unemployed people. If you call landlines, you miss the cellphone-only crowd. And if your method for estimating which poll question answerers are “likely voters” is wrong, that can throw you off too.

Polling companies use algorithms to correct for all of this, but some are more accurate than others. When you read about an individual poll in the paper, it’s hard to tell how this effect is playing out. But Silver uses polls from many different pollsters. He can average them, and also apply correcting factors to polls that appear to be biased a point or two in either direction.

Again, it’s something that’s hard to do in a place where we have few polls and fewer polling companies.

Silver’s predictions attracted increasing attention as the campaign went on. Newscasters were annoyed that Silver’s blog mocked the way they reported national polls as evidence of a “tight race.” Republicans were alarmed by his steady predictions of an Obama win. Pundits were horrified that a young math whiz was making them look bad.

The tone of the debate reminded me of Moneyball, where Brad Pitt plays a statistics-obsessed baseball manager under attack by traditionalists. Or maybe a scene from a teen movie involving nerds being bullied by jocks in the high school cafeteria.

On one side was Nate Silver, skinny, young and with a nasally voice and nervous giggle. On the other, a crowd of popular, self-confident loud mouths with good hair.

Scarborough in particular singled out Silver for abuse: “Anybody that thinks that this race is anything but a toss-up right now is such an ideologue, they should be kept away from typewriters, computers, laptops, and microphones for the next 10 days, because they’re jokes.”

That shows you what 20 years of gut feel can look like if the owner of the gut ignores the facts.

There were a lot of victory parties in the United States last night, and I’m sure they whooped it up at the White House. But few were likely as sweet as the one in Nate Silver’s cubicle at the New York Times.

Keith Halliday is a Yukon economist and author of the Aurore of the Yukon series of historical children’s adventure novels.

Just Posted

No vacancy: Whitehorse family spends five months seeking housing

‘I didn’t think it would be this hard’

Bedbug situation in Whitehorse building becoming intolerable, resident says

Gabriel Smarch said he’s been dealing with bedbugs since he moved into his apartment 15 years ago

Yukon government transfers responsibility for Native Language Centre to CYFN

‘At the end of the day the importance is that First Nations have control of the language’

New operator applies for licence at shuttered Whitehorse daycare

Application has listed a proposed program start date of Feb. 1.

The week in Yukon mining

Goldcorp re-submits Coffee plans, Mount Nansen sale looms, Kudz Ze Kayah comments open

Ice, ice, baby: scaling a frozen Yukon waterfall

‘There’s a really transformative affect with adventure’

Says Marwell is problematic, requests council further hash it out

You can buy alcohol and tobacco on Main Street in Whitehorse —… Continue reading

Yukon history is picture post card perfect

The most interesting gift I received at Christmas this year was the… Continue reading

Contentious Whitehorse quarry proposal raises city hackles

‘We’ve had concerns from the get-go on this one’

Whitehorse time machine

Yukon’s capital added 10,000 people over the last three decades, no YESAB application needed

How to make sure your car starts in the cold

It’s about more than just making sure your plug works

Whitehorse fuel delivery company fined $1,100 for Rancheria crash

The Crown stayed two other charges against the company related to the Aug. 7, 2017, crash

Most Read