Revenge of the Nerd

Do you go with your gut? Or dive into the data? One of the most interesting contests during the 2012 U.S. election was not between politicians.

Do you go with your gut? Or dive into the data?

One of the most interesting contests during the 2012 U.S. election was not between politicians. It featured a Who’s Who of big-name political pundits versus a data wonk you’ve probably never heard of, Nate Silver.

The breathless clips on most newscasts told us the race would be “tight” and “too close to call.” High-profile commentators like Peggy Noonan of the Wall Street Journal called for “a Romney win” based on her gut feel for the intense support she felt at recent Romney rallies. Joe Scarborough of MSNBC said “after practicing politics for 20 years, I suppose I would rather be in Mitt Romney’s shoes.” Former Clinton advisor turned commentator Dick Morris predicted a big Romney win, with 325 electoral college votes and just 213 for Obama.

Even former Canadian Prime Minister Brian Mulroney’s gut got into the act. “Romney, by a walk,” he told media yesterday. “You can call me tomorrow morning.”

But these intense intestinal feelings turned out not to be prescient instincts, just bad cases of “pundit tummy” (the political equivalent of Delhi Belly or Montezuma’s Revenge).

Nate Silver and his statistical model turned out to be right. In fact, Silver called the presidential election correctly in all 50 states (assuming Florida stays in the Obama column where it was when this column was written on Wednesday morning).

On the morning of election day, Silver said there was a 90.9 per cent chance of an Obama win. He also estimated the likelihood of Obama winning certain numbers of Electoral College votes, and his most likely scenario was the 332 votes that Obama will get if his win in Florida is confirmed.

How did he do it? The secret is data; more specifically, a technique called “poll aggregation.” Remember that U.S. presidents are elected by the Electoral College. The simple version of how this works is that each state has a number of votes based roughly on population, and the candidate who wins the state gets all of those votes. A 51 per cent win in California, with 55 Electoral College votes, is better than a 99 per cent win in Alaska, which has just three.

Silver’s technique aggregates dozens of polls at the state level to create a deep, data-driven view on which candidate will win each state’s Electoral College votes. This technique is so powerful because it addresses some of the key challenges with traditional polling.

First, it focuses on the state level, not the national polls that make the nightly news. This makes sense since the states are where the Electoral College votes are decided. You’ll remember seeing weird Canadian news stories saying the Bloc Quebecois had 8 per cent support nationally, which is a misleading way of saying the Bloc has 50 per cent in Quebec and zero per cent everywhere else.

The second is margin of error. Because Silver combines literally dozens of polls in a swing state like Ohio, he gets a much larger and more reliable sample size. On Monday, he reported the results of 22 new polls from swing states, with 19 showing leads for Obama. Of six Ohio polls that day alone, five showed Obama in the lead by one to five points, and one was a tie.

Compare that tidal wave of data to a Yukon election, where we only have one or two polls in an entire campaign and the sample sizes are small.

Third is bias. In a world where perhaps only 10 per cent of people actually pick up the phone and answer pollster questions, it is hard for polling firms to accurately survey a true random sample. If you call during the day you get more retired and unemployed people. If you call landlines, you miss the cellphone-only crowd. And if your method for estimating which poll question answerers are “likely voters” is wrong, that can throw you off too.

Polling companies use algorithms to correct for all of this, but some are more accurate than others. When you read about an individual poll in the paper, it’s hard to tell how this effect is playing out. But Silver uses polls from many different pollsters. He can average them, and also apply correcting factors to polls that appear to be biased a point or two in either direction.

Again, it’s something that’s hard to do in a place where we have few polls and fewer polling companies.

Silver’s predictions attracted increasing attention as the campaign went on. Newscasters were annoyed that Silver’s blog mocked the way they reported national polls as evidence of a “tight race.” Republicans were alarmed by his steady predictions of an Obama win. Pundits were horrified that a young math whiz was making them look bad.

The tone of the debate reminded me of Moneyball, where Brad Pitt plays a statistics-obsessed baseball manager under attack by traditionalists. Or maybe a scene from a teen movie involving nerds being bullied by jocks in the high school cafeteria.

On one side was Nate Silver, skinny, young and with a nasally voice and nervous giggle. On the other, a crowd of popular, self-confident loud mouths with good hair.

Scarborough in particular singled out Silver for abuse: “Anybody that thinks that this race is anything but a toss-up right now is such an ideologue, they should be kept away from typewriters, computers, laptops, and microphones for the next 10 days, because they’re jokes.”

That shows you what 20 years of gut feel can look like if the owner of the gut ignores the facts.

There were a lot of victory parties in the United States last night, and I’m sure they whooped it up at the White House. But few were likely as sweet as the one in Nate Silver’s cubicle at the New York Times.

Keith Halliday is a Yukon economist and author of the Aurore of the Yukon series of historical children’s adventure novels.

Get local stories you won't find anywhere else right to your inbox.
Sign up here

Just Posted

A proposed Official Community Plan amendment would designate a 56.3-hectare piece of land in Whistle Bend currently designated as green space, as urban residential use. Whitehorse city council will vote on the second reading of the Official Community Plan amendment on Dec. 7. (Courtesy City of Whitehorse)
Future area of Whistle Bend considered by council

Members set to vote on second reading for OCP change

The City of Whitehorse’s projected deficit could be $100,000 more than originally predicted earlier this year. (Crystal Schick/Yukon News file)
City deficit could be just over $640,000 this year

Third quarter financial reports presented to council

Chief Medical Officer of Health Dr. Brendan Hanley speaks during a COVID-19 press conference in Whitehorse on Oct. 30. Masks became mandatory in the Yukon for anyone five years old and older as of Dec. 1 while in public spaces. (Crystal Schick/Yukon News file)
As mask law comes into effect, premier says $500 fines will be last resort

The territory currently has 17 active cases of COVID-19

Crystal Schick/Yukon News file
Ranj Pillai, minister of economic development, during a press conference on April 1.
Government rejects ATAC mining road proposal north of Keno City

Concerns from the First Nation of Na-Cho Nyäk Dun were cited as the main reason for the decision

asdf
WYATT’S WORLD

Wyatt’s World for Dec. 2, 2020

The new Little Salmon Carmacks First Nation council elected Dec. 1. (Submitted)
Little Salmon Carmacks elects new chief, council

Nicole Tom elected chief of Little Salmon Carcmacks First Nation

Submitted/Yukon News file
Yukon RCMP’s Historical Case Unit is seeking information related to the unsolved homicide of Allan Donald Waugh, 69, who was found deceased in his house on May 30, 2014.
Yukon RCMP investigating unsolved Allan Waugh homicide

Yukon RCMP’s Historical Case Unit is seeking information related to an unsolved… Continue reading

A jogger runs along Millenium Trail as the sun rises over the trees around 11 a.m. in Whitehorse on Dec. 12, 2018. The City of Whitehorse could soon have a new trail plan in place to serve as a guide in managing the more than 233 kilometres of trails the city manages. (Crystal Schick/Yukon News file)
2020 trail plan comes forward

Policies and bylaws would look at e-mobility devices

Snow-making machines are pushed and pulled uphill at Mount Sima in 2015. The ski hill will be converting snow-making to electric power with more than $5 million in funding from the territorial and federal governments. (Joel Krahn/Yukon News file)
Mount Sima funded to cut diesel reliance

Mount Sima ski hill is converting its snowmaking to electric power with… Continue reading

Colin McDowell, the director of land management for the Yukon government, pulls lottery tickets at random during a Whistle Bend property lottery in Whitehorse on Sept. 9, 2019. A large amount of lots are becoming available via lottery in Whistle Bend as the neighbourhood enters phase five of development. (Crystal Schick/Yukon News file)
Lottery for more than 250 new Whistle Bend lots planned for January 2021

Eight commercial lots are being tendered in additional to residential plots

The Government of Yukon Main Administration Building in Whitehorse on Aug. 21. The Canada Border Services Agency announced Nov. 26 that they have laid charges against six people, including one Government of Yukon employee, connected to immigration fraud that involved forged Yukon government documents. (Crystal Schick/Yukon News file)
Charges laid in immigration fraud scheme, warrant out for former Yukon government employee

Permanent residency applications were submitted with fake Yukon government documents

Black Press Media and BraveFace have come together to support children facing life-threatening conditions. Net proceeds from these washable, reusable, three-layer masks go to Make-A-Wish Foundation BC & Yukon.
Put on a BraveFace: Mask fundraiser helps make children’s wishes come true

From Black Press Media + BraveFace – adult, youth and kid masks support Make-A-Wish Foundation

Whitehorse City Hall. (Joel Krahn/Yukon News file)
City hall, briefly

A look at decisions made by Whitehorse city council this week

Most Read