Learning from the evidence

A wag once said that economists liked to look at something that worked in practice, and then study it to see if it worked in theory. But a new generation of economists is turning this stereotype on its head.

A wag once said that economists liked to look at something that worked in practice, and then study it to see if it worked in theory.

But a new generation of economists is turning this stereotype on its head. Instead of theorizing about why people make the economic decisions they do, these economists use randomized trials to test how real-world people behave.

Esther Duflo and Abhijit Banerjee have recently published a fascinating book on the topic: Poor Economics. In it, they look at why so many well-meaning development aid projects fail. And they convincingly answer some tricky questions, including a remarkable case study on the best way to teach African teenagers about how to avoid getting HIV/AIDS.

While the book’s focus is development, it raises plenty of questions about why some social and economic problems in rich countries like Canada seem impervious to the government programs we throw at them.

The heart of their approach is the randomized trial. Perhaps best known for testing new drugs, this is where you take a group of people, divide them randomly into groups, and give each group a different treatment. There is the “control group,” where you give them a placebo, and then one or more treatment groups who get the drug being tested.

Since the subjects have been selected randomly, a properly executed randomized trial with a large enough sample removes all the noise from the experiment. The control group and the treatment group subjects share the same characteristics; the only difference is whether they got a placebo or the drug being tested.

So you can be pretty confident that if the treatment group gets better and the control group doesn’t, it’s because of the drug. Not because they have different lifestyles, live in a part of town with better access to health care, or some other reason.

An example shows how effective this can be in public policy. Consider HIV/AIDS, which infects nearly two million Africans every year with horrific social and economic consequences. An economist named Pascaline Dupas from UCLA asked herself what the best way was to educate African teenagers to avoid this scourge.

The control group was teenagers who got the usual national curriculum focused on abstinence. The treatment group instead got “risk reduction” information, including the critical fact for young women that sex with older men is much riskier since they have a much higher infection rate than boys their own age.

Dupas measured teen pregnancy in the two groups, as a good proxy for unsafe sexual behaviour. After the study, the control group following the official curriculum showed no change. But teen pregnancy among young women given the risk reduction information went down 28 per cent.

Read the details in Dupas’s paper. They are hard to argue with. Her study has huge implications for the millions of dollars spent on HIV/AIDS programs in Africa every year, suggesting that much of the money is being spent ineffectively. Governments now have evidence to rethink their focus on politically popular but less effective abstinence training. With millions of people being infected every year, this is enormously important.

Randomized trials are increasingly common in business. Online retailers test different page layouts or even the shape and colour of “Proceed to check out!” buttons to see what works best. Barack Obama’s campaign team famously used randomized trials to see which images on the candidate’s website elicited the most donations, netting millions in extra cash.

Gary Loveman, a Harvard economist turned casino CEO, uses randomized trials to coax more money out of visitors’ pockets at Caesar’s. Since many gamblers use swipe cards at the casino, he can track how they behave. He can test offering free dinners or hotel rooms to first-time visitors having a bad day, and see if it makes them more likely to return a second time (it does, by the way). I once heard Loveman on Planet Money saying that there are three things that will get you fired from the casino: stealing, sexual harassment and running an experiment without a control group.

Governments are slower to adopt the practice. We identify a problem, spend a lot of money designing and implementing programs, and then can’t tell if it worked or not.

Say we are paying for more training for the unemployed. If more get jobs, is that because of the training or because the economy got better? Or we spend a lot of money on police and the crime rate went down. Was that because of the extra police, or the fall in the number of 18-25 year-old males as the population aged?

What is something in the Yukon where we might experiment with these techniques? The high school drop-out rate is one idea. Victoria Gold has recently donated a bunch of money for stay-in-school activities.

In a traditional government program, officials would think of some ideas on how to spend the money effectively. Then a few years later, the program would be evaluated and the results would be inconclusive. Was it the stay-in-school programming, improvements in course offerings at school, a slow economy or changes in socio-economic characteristics in the school population?

If the government was clever, it would have a control group and test two or three different stay-in-school strategies against it. Perhaps one-on-one mentoring works best? Or cash bonuses for attendance? Or better information on job opportunities?

It’s hard to know for sure without hard randomized trial data.

Some might say that it’s wrong to do randomized trials on teenagers like this. But the sad fact is that the government already experiments on them with new policies every few years. It just doesn’t have much evidence whether its experiments work or not.

Keith Halliday is a Yukon economist and author of the MacBride Museum’s Aurore of the Yukon series of historical children’s adventure novels.

Just Posted

Yukon government reveals proposed pot rules

‘We’re under a tight timeline, everybody is Canada is, so we’re doing this in stages’

Michael Nehass released from custody in B.C.

Yukon man who spent years in WCC awaiting peace bond application, faces no charges

Phase 5 of Whistle Bend a go

Next phase of subdivision will eventually be home to around 750 people

Silver rules out HST, layoffs and royalty changes

Yukon’s financial advisory panel has released its final report

City of Whitehorse budgets $30M for infrastructure over four years

‘I think we’re concentrating on the most important things’

Straight and true: the story of the Yukon colours

Michael Gates | History Hunter Last week, I participated in the 150th… Continue reading

Get ready to tumble: Whitehorse’s Polarettes to flip out at fundraiser

‘There’s a mandatory five-minute break at the end, just so people don’t fall over’

Alaska’s governor goes to China

There are very different rules for resource projects depending on which side of the border you’re on

Yukon survey shows broad support for legal pot

But there’s no consensus on retail and distribution models

Yukon government releases survey on the territory’s liquor laws

Changes could include allowing sale of booze in grocery stores

Get family consent before moving patients to other hospitals: NDP critic

‘Where is the respect and where is the dignity?’

Bill C-17 passes third reading in House of Commons

The bill, which will repeal controversial amendments made to YESAA by Bill S-6, will now go to Senate

White Pass and Yukon Route musical chugs on without director

The cast and crew of Stonecliff are pushing forward without Conrad Boyce, who went on medical leave

Most Read