Do you go with your gut? Or dive into the data?
One of the most interesting contests during the 2012 U.S. election was not between politicians. It featured a Who’s Who of big-name political pundits versus a data wonk you’ve probably never heard of, Nate Silver.
The breathless clips on most newscasts told us the race would be “tight” and “too close to call.” High-profile commentators like Peggy Noonan of the Wall Street Journal called for “a Romney win” based on her gut feel for the intense support she felt at recent Romney rallies. Joe Scarborough of MSNBC said “after practicing politics for 20 years, I suppose I would rather be in Mitt Romney’s shoes.” Former Clinton advisor turned commentator Dick Morris predicted a big Romney win, with 325 electoral college votes and just 213 for Obama.
Even former Canadian Prime Minister Brian Mulroney’s gut got into the act. “Romney, by a walk,” he told media yesterday. “You can call me tomorrow morning.”
But these intense intestinal feelings turned out not to be prescient instincts, just bad cases of “pundit tummy” (the political equivalent of Delhi Belly or Montezuma’s Revenge).
Nate Silver and his statistical model turned out to be right. In fact, Silver called the presidential election correctly in all 50 states (assuming Florida stays in the Obama column where it was when this column was written on Wednesday morning).
On the morning of election day, Silver said there was a 90.9 per cent chance of an Obama win. He also estimated the likelihood of Obama winning certain numbers of Electoral College votes, and his most likely scenario was the 332 votes that Obama will get if his win in Florida is confirmed.
How did he do it? The secret is data; more specifically, a technique called “poll aggregation.” Remember that U.S. presidents are elected by the Electoral College. The simple version of how this works is that each state has a number of votes based roughly on population, and the candidate who wins the state gets all of those votes. A 51 per cent win in California, with 55 Electoral College votes, is better than a 99 per cent win in Alaska, which has just three.
Silver’s technique aggregates dozens of polls at the state level to create a deep, data-driven view on which candidate will win each state’s Electoral College votes. This technique is so powerful because it addresses some of the key challenges with traditional polling.
First, it focuses on the state level, not the national polls that make the nightly news. This makes sense since the states are where the Electoral College votes are decided. You’ll remember seeing weird Canadian news stories saying the Bloc Quebecois had 8 per cent support nationally, which is a misleading way of saying the Bloc has 50 per cent in Quebec and zero per cent everywhere else.
The second is margin of error. Because Silver combines literally dozens of polls in a swing state like Ohio, he gets a much larger and more reliable sample size. On Monday, he reported the results of 22 new polls from swing states, with 19 showing leads for Obama. Of six Ohio polls that day alone, five showed Obama in the lead by one to five points, and one was a tie.
Compare that tidal wave of data to a Yukon election, where we only have one or two polls in an entire campaign and the sample sizes are small.
Third is bias. In a world where perhaps only 10 per cent of people actually pick up the phone and answer pollster questions, it is hard for polling firms to accurately survey a true random sample. If you call during the day you get more retired and unemployed people. If you call landlines, you miss the cellphone-only crowd. And if your method for estimating which poll question answerers are “likely voters” is wrong, that can throw you off too.
Polling companies use algorithms to correct for all of this, but some are more accurate than others. When you read about an individual poll in the paper, it’s hard to tell how this effect is playing out. But Silver uses polls from many different pollsters. He can average them, and also apply correcting factors to polls that appear to be biased a point or two in either direction.
Again, it’s something that’s hard to do in a place where we have few polls and fewer polling companies.
Silver’s predictions attracted increasing attention as the campaign went on. Newscasters were annoyed that Silver’s blog mocked the way they reported national polls as evidence of a “tight race.” Republicans were alarmed by his steady predictions of an Obama win. Pundits were horrified that a young math whiz was making them look bad.
The tone of the debate reminded me of Moneyball, where Brad Pitt plays a statistics-obsessed baseball manager under attack by traditionalists. Or maybe a scene from a teen movie involving nerds being bullied by jocks in the high school cafeteria.
On one side was Nate Silver, skinny, young and with a nasally voice and nervous giggle. On the other, a crowd of popular, self-confident loud mouths with good hair.
Scarborough in particular singled out Silver for abuse: “Anybody that thinks that this race is anything but a toss-up right now is such an ideologue, they should be kept away from typewriters, computers, laptops, and microphones for the next 10 days, because they’re jokes.”
That shows you what 20 years of gut feel can look like if the owner of the gut ignores the facts.
There were a lot of victory parties in the United States last night, and I’m sure they whooped it up at the White House. But few were likely as sweet as the one in Nate Silver’s cubicle at the New York Times.
Keith Halliday is a Yukon economist and author of the Aurore of the Yukon series of historical children’s adventure novels.