Learning from the evidence

A wag once said that economists liked to look at something that worked in practice, and then study it to see if it worked in theory. But a new generation of economists is turning this stereotype on its head.

A wag once said that economists liked to look at something that worked in practice, and then study it to see if it worked in theory.

But a new generation of economists is turning this stereotype on its head. Instead of theorizing about why people make the economic decisions they do, these economists use randomized trials to test how real-world people behave.

Esther Duflo and Abhijit Banerjee have recently published a fascinating book on the topic: Poor Economics. In it, they look at why so many well-meaning development aid projects fail. And they convincingly answer some tricky questions, including a remarkable case study on the best way to teach African teenagers about how to avoid getting HIV/AIDS.

While the book’s focus is development, it raises plenty of questions about why some social and economic problems in rich countries like Canada seem impervious to the government programs we throw at them.

The heart of their approach is the randomized trial. Perhaps best known for testing new drugs, this is where you take a group of people, divide them randomly into groups, and give each group a different treatment. There is the “control group,” where you give them a placebo, and then one or more treatment groups who get the drug being tested.

Since the subjects have been selected randomly, a properly executed randomized trial with a large enough sample removes all the noise from the experiment. The control group and the treatment group subjects share the same characteristics; the only difference is whether they got a placebo or the drug being tested.

So you can be pretty confident that if the treatment group gets better and the control group doesn’t, it’s because of the drug. Not because they have different lifestyles, live in a part of town with better access to health care, or some other reason.

An example shows how effective this can be in public policy. Consider HIV/AIDS, which infects nearly two million Africans every year with horrific social and economic consequences. An economist named Pascaline Dupas from UCLA asked herself what the best way was to educate African teenagers to avoid this scourge.

The control group was teenagers who got the usual national curriculum focused on abstinence. The treatment group instead got “risk reduction” information, including the critical fact for young women that sex with older men is much riskier since they have a much higher infection rate than boys their own age.

Dupas measured teen pregnancy in the two groups, as a good proxy for unsafe sexual behaviour. After the study, the control group following the official curriculum showed no change. But teen pregnancy among young women given the risk reduction information went down 28 per cent.

Read the details in Dupas’s paper. They are hard to argue with. Her study has huge implications for the millions of dollars spent on HIV/AIDS programs in Africa every year, suggesting that much of the money is being spent ineffectively. Governments now have evidence to rethink their focus on politically popular but less effective abstinence training. With millions of people being infected every year, this is enormously important.

Randomized trials are increasingly common in business. Online retailers test different page layouts or even the shape and colour of “Proceed to check out!” buttons to see what works best. Barack Obama’s campaign team famously used randomized trials to see which images on the candidate’s website elicited the most donations, netting millions in extra cash.

Gary Loveman, a Harvard economist turned casino CEO, uses randomized trials to coax more money out of visitors’ pockets at Caesar’s. Since many gamblers use swipe cards at the casino, he can track how they behave. He can test offering free dinners or hotel rooms to first-time visitors having a bad day, and see if it makes them more likely to return a second time (it does, by the way). I once heard Loveman on Planet Money saying that there are three things that will get you fired from the casino: stealing, sexual harassment and running an experiment without a control group.

Governments are slower to adopt the practice. We identify a problem, spend a lot of money designing and implementing programs, and then can’t tell if it worked or not.

Say we are paying for more training for the unemployed. If more get jobs, is that because of the training or because the economy got better? Or we spend a lot of money on police and the crime rate went down. Was that because of the extra police, or the fall in the number of 18-25 year-old males as the population aged?

What is something in the Yukon where we might experiment with these techniques? The high school drop-out rate is one idea. Victoria Gold has recently donated a bunch of money for stay-in-school activities.

In a traditional government program, officials would think of some ideas on how to spend the money effectively. Then a few years later, the program would be evaluated and the results would be inconclusive. Was it the stay-in-school programming, improvements in course offerings at school, a slow economy or changes in socio-economic characteristics in the school population?

If the government was clever, it would have a control group and test two or three different stay-in-school strategies against it. Perhaps one-on-one mentoring works best? Or cash bonuses for attendance? Or better information on job opportunities?

It’s hard to know for sure without hard randomized trial data.

Some might say that it’s wrong to do randomized trials on teenagers like this. But the sad fact is that the government already experiments on them with new policies every few years. It just doesn’t have much evidence whether its experiments work or not.

Keith Halliday is a Yukon economist and author of the MacBride Museum’s Aurore of the Yukon series of historical children’s adventure novels.

Get local stories you won't find anywhere else right to your inbox.
Sign up here

Just Posted

Yukon gets sixth case of COVID-19, three have recovered

Dr. Brendan Hanley said there is a sixth case in Whitehorse but three have recovered

White River First Nation calls on premier to stop mining activity

An influx of miners in the area is causing concern

Whitehorse city council holds off on purchasing Seventh Avenue property

Would be a “costly endeavour for the city”, says councillor

VGFN passing emergency legislation to better screen for travellers entering the community

The move follows the unwelcome arrival of two people from Quebec last week

Victoria Gold still operating Eagle Gold mine with COVID-19 precautions in place

The mine is still in operation but with precautions, including social distancing, in place

YTA, Yukon government reach agreement on hiring dispute out of court

YTA’s petition was set to be heard March 25 but was called off after the parties reached an agreement

City hall, briefly

Here’s a look at issues discussed by Whitehorse city council at its… Continue reading

Skagway has resolve in the COVID-19 struggle, mayor says

Skagway mayor said border access is important for residents.

Yukonomist: Steering your business through COVID-19

While “proofing” your business against the impacts might not be possible, being prepared is.

History Hunter: How the Yukon was spared the influenza pandemic of 1918

The isolation of the Yukon then afford the territory some protection that it doesn’t have today

Whitehorse city council contemplates OCP change for section of the tank farm

Change would allow for commercial industrial use instead of current residential classification

Truck slides off Dempster Highway

The truck left the road around 4 p.m. on March 19. The highway was closed until March 21 for cleanup.

Most Read