A wag once said that economists liked to look at something that worked in practice, and then study it to see if it worked in theory.
But a new generation of economists is turning this stereotype on its head. Instead of theorizing about why people make the economic decisions they do, these economists use randomized trials to test how real-world people behave.
Esther Duflo and Abhijit Banerjee have recently published a fascinating book on the topic: Poor Economics. In it, they look at why so many well-meaning development aid projects fail. And they convincingly answer some tricky questions, including a remarkable case study on the best way to teach African teenagers about how to avoid getting HIV/AIDS.
While the book’s focus is development, it raises plenty of questions about why some social and economic problems in rich countries like Canada seem impervious to the government programs we throw at them.
The heart of their approach is the randomized trial. Perhaps best known for testing new drugs, this is where you take a group of people, divide them randomly into groups, and give each group a different treatment. There is the “control group,” where you give them a placebo, and then one or more treatment groups who get the drug being tested.
Since the subjects have been selected randomly, a properly executed randomized trial with a large enough sample removes all the noise from the experiment. The control group and the treatment group subjects share the same characteristics; the only difference is whether they got a placebo or the drug being tested.
So you can be pretty confident that if the treatment group gets better and the control group doesn’t, it’s because of the drug. Not because they have different lifestyles, live in a part of town with better access to health care, or some other reason.
An example shows how effective this can be in public policy. Consider HIV/AIDS, which infects nearly two million Africans every year with horrific social and economic consequences. An economist named Pascaline Dupas from UCLA asked herself what the best way was to educate African teenagers to avoid this scourge.
The control group was teenagers who got the usual national curriculum focused on abstinence. The treatment group instead got “risk reduction” information, including the critical fact for young women that sex with older men is much riskier since they have a much higher infection rate than boys their own age.
Dupas measured teen pregnancy in the two groups, as a good proxy for unsafe sexual behaviour. After the study, the control group following the official curriculum showed no change. But teen pregnancy among young women given the risk reduction information went down 28 per cent.
Read the details in Dupas’s paper. They are hard to argue with. Her study has huge implications for the millions of dollars spent on HIV/AIDS programs in Africa every year, suggesting that much of the money is being spent ineffectively. Governments now have evidence to rethink their focus on politically popular but less effective abstinence training. With millions of people being infected every year, this is enormously important.
Randomized trials are increasingly common in business. Online retailers test different page layouts or even the shape and colour of “Proceed to check out!” buttons to see what works best. Barack Obama’s campaign team famously used randomized trials to see which images on the candidate’s website elicited the most donations, netting millions in extra cash.
Gary Loveman, a Harvard economist turned casino CEO, uses randomized trials to coax more money out of visitors’ pockets at Caesar’s. Since many gamblers use swipe cards at the casino, he can track how they behave. He can test offering free dinners or hotel rooms to first-time visitors having a bad day, and see if it makes them more likely to return a second time (it does, by the way). I once heard Loveman on Planet Money saying that there are three things that will get you fired from the casino: stealing, sexual harassment and running an experiment without a control group.
Governments are slower to adopt the practice. We identify a problem, spend a lot of money designing and implementing programs, and then can’t tell if it worked or not.
Say we are paying for more training for the unemployed. If more get jobs, is that because of the training or because the economy got better? Or we spend a lot of money on police and the crime rate went down. Was that because of the extra police, or the fall in the number of 18-25 year-old males as the population aged?
What is something in the Yukon where we might experiment with these techniques? The high school drop-out rate is one idea. Victoria Gold has recently donated a bunch of money for stay-in-school activities.
In a traditional government program, officials would think of some ideas on how to spend the money effectively. Then a few years later, the program would be evaluated and the results would be inconclusive. Was it the stay-in-school programming, improvements in course offerings at school, a slow economy or changes in socio-economic characteristics in the school population?
If the government was clever, it would have a control group and test two or three different stay-in-school strategies against it. Perhaps one-on-one mentoring works best? Or cash bonuses for attendance? Or better information on job opportunities?
It’s hard to know for sure without hard randomized trial data.
Some might say that it’s wrong to do randomized trials on teenagers like this. But the sad fact is that the government already experiments on them with new policies every few years. It just doesn’t have much evidence whether its experiments work or not.
Keith Halliday is a Yukon economist and author of the MacBride Museum’s Aurore of the Yukon series of historical children’s adventure novels.