Why economists need bureaucracy
05 Jun 2018
Ask an economist “how should the world be run?” and their first reaction will be one of surprise. They’ve written several decades-worth of papers on how the world should be run, but no one’s ever actually asked them before. After recomposing themselves they’ll probably start talking about interest rates and externality-internalizing market mechanisms, but they’ll be lying. An economist’s true utopia is a world of arbitrary and rigidly enforced bureaucracy.
Economists need bureaucracy because it can accidentally generate so much data for natural experiments. If economists truly had things their way, they’d be allowed to test all of their theories using Randomly Controlled Trials. RCTs are as close to the gold-standard scientific method as the social sciences can get. You take a big group of people and randomly split them into a control and a treatment group. You give the treatment group something nice, like extra training, more money, lower taxes, a cool hat, and you give the control group nothing. Then you step back and measure how everyone gets on. A well-designed RCT is therefore mostly free of sampling bias, survivorship bias, and all the other things that stop economists from getting Nobel Prizes.
However, RCTs are expensive, and in many cases impossible or evil or both. I don’t want to run a drug cartel, the subject says, I want to be a neurosurgeon. Sorry pal, replies the economist, we need to see how your community is impacted by your life of crime, now get in the van. Whilst university boards of ethics might be comfortable with giving one group of unemployed steelworkers training as accountants and comparing their earnings to a separate control group, they get quite squeamish when the interventions get more exotic.
Medical science gets to make its own experiments all the time. Patient A gets the sugar pill, patient B gets the potentially (we’re not sure yet) life saving cancer drug. Patient C gets the experimental new surgery, Patient D gets some anesthetic, a fake scar, and whatever placebo effect that gives you. Experimental physicists have it easy too. They get to smash together the exact same particles in the exact same particle accelerator and see if anything changes when they fiddle with the velocity. If you believe the Pauli Exclusion Principle (and you should), many of these particles are literally physically indistinguishable from each other. Economists won’t get true counterfactuals until the invention of the time machine or socially-acceptable human cloning, at which point no one will care whether the Fed should hike rates by a quarter point next July anymore.
Instead of running expensive RCTs, many pragmatic cheapskate economists turn to arbitrary and rigidly enforced bureaucracy to create “natural experiments”. For example, suppose that you want to know the effect of smaller class sizes on pupil performance. You could try naively calculating whether kids in smaller classes get higher marks, but this would be subject to all kinds of biases. For example, it’s likely that schools in wealthy areas tend to have smaller classes than those in poorer areas. It’s also likely that kids from wealthy families tend to get higher marks anyway, regardless of their class size. And even within a single school, classes may not have been assigned with experimental validity in mind. Perhaps there’s a special group of all the high-scoring kids, or a different special group of all the low-scoring ones. You could try to control for this with some careful regression analysis, but it’s very difficult to know if you’re really controlling for everything that matters. You’d be better off doing what Angrist and Lavy did in their 1999 study and paying a visit to the delightfully bureaucratic Israeli school system.
In Israel, school classes are filled up with kids until they reach a size of 40. When, and only when, the 41st child enrolls, the class is split into two classes of 20 and 21. These smaller classes fill up until they contain a total of 81 kids, at which point they are split up into 3 classes, and so on. Angrist and Lavy exploited this “regression discontinuity” to rigorously compare the effect of class size on a child’s performance. Thanks to the Israel bureaucracy, they could be weirdly confident that the main driver of class size was whether or not the number of children in a grade reached these rigid and arbitrary boundaries. It seems very safe to assume that whether a grade has exactly 40 or 41 children is essentially independent of family wealth, kids’ natural ability, teacher ability, or anything else that might interfere with their study of the effect of class size on achievement.
An economist’s ideal web of red tape affords its subjects no wiggle room. Angrist and Lavy don’t want Israeli parents lobbying for their children to be moved to a smaller class, or worse, moving them to another school and taking them out of the natural experiment altogether. It might turn out that parents of more able children are more likely to lobby for them to be moved to smaller classes, or that rich parents are more likely to put their children into private school if they get stuck in a 40 kid mega-class. These types of mid-experiment changes upset the balance of the control and treatment groups, and are liable to reintroduce the kinds of sample bias that random assignment is meant to avoid.
Many natural experimentalists therefore make use of the most heartless and undiscerning bureaucracy on the planet, or as some people like to call it, nature. In their 2009 study, Maccini and Yang investigated the effects of early life poverty on a range of adult outcomes in Indonesia. They find that women born in a village at a time of above-average rainfall have significantly better health in adulthood, complete more years of schooling, and are more wealthy. They provide evidence that this is because the increased rainfall led to better crop yields, which led to families being able to better feed their children and help them develop, with long-term positive effects that do not simply dissipate after childhood. In this study, variable rainfall does the work of a committed but deranged experimentalist who ruins the lives of a treatment group, does nothing to a control group, and runs the numbers on their depraved but bias-free study at the end of the year. If there’s going to be humanitarian tragedy in the world then we might as well get some good data on it so we know what to do in the future.
A final example of the benefits of bureaucracy: many people have attempted to study the effect of military service on future earnings. A naive analysis might compare the earnings of random samples of veterans and non-veterans, see which number is bigger and wait for the Nobel Prize notification to arrive. But such an analysis would be vulnerable to the same types of sample bias as the naive classroom size analysis. Perhaps people who are more likely to serve in the military tend to earn less anyway, even without the effect of their service. For a truly bias-free experiment, we would need to randomly force people to join the army, and this is unfortunately one of the many things that society and university boards of ethics won’t let economists do. However, randomly forcing people to join to army was the exact aim of the Vietnam War Draft. Again, the sample groups were not perfect - plenty of people joined up voluntarily without being drafted, and plenty of draftees fled to Canada. But, as any experimentalist will tell you, you’ll always have some subjects fleeing to Canada, and these fuzzy edges can be dealt with via a cunning approach known as “instrumental variables”. The important part is that you get a big dose of random assignment to give your study its core power, and indeed some of the most statistically trustworthy analysis of veterans’ earnings has come from data from the Vietnam War Draft.
So the next time you are tripped up by some frivolous-seeming bureaucracy, whilst your identical twin sibling experiences receives free training as a database administrator and a year-long exemption from all sales taxes, you can be comforted by the fact that you are making an economist somewhere very happy.
Examples and inspiration taken from Mostly Harmless Econometrics by Angrist and Pischke.
Ask an economist “how should the world be run?” and their first reaction will be one of surprise. They’ve written several decades-worth of papers on how the world should be run, but no one’s ever actually asked them before. After recomposing themselves they’ll probably start talking about interest rates and externality-internalizing market mechanisms, but they’ll be lying. An economist’s true utopia is a world of arbitrary and rigidly enforced bureaucracy.
Economists need bureaucracy because it can accidentally generate so much data for natural experiments. If economists truly had things their way, they’d be allowed to test all of their theories using Randomly Controlled Trials. RCTs are as close to the gold-standard scientific method as the social sciences can get. You take a big group of people and randomly split them into a control and a treatment group. You give the treatment group something nice, like extra training, more money, lower taxes, a cool hat, and you give the control group nothing. Then you step back and measure how everyone gets on. A well-designed RCT is therefore mostly free of sampling bias, survivorship bias, and all the other things that stop economists from getting Nobel Prizes.
However, RCTs are expensive, and in many cases impossible or evil or both. I don’t want to run a drug cartel, the subject says, I want to be a neurosurgeon. Sorry pal, replies the economist, we need to see how your community is impacted by your life of crime, now get in the van. Whilst university boards of ethics might be comfortable with giving one group of unemployed steelworkers training as accountants and comparing their earnings to a separate control group, they get quite squeamish when the interventions get more exotic.
Medical science gets to make its own experiments all the time. Patient A gets the sugar pill, patient B gets the potentially (we’re not sure yet) life saving cancer drug. Patient C gets the experimental new surgery, Patient D gets some anesthetic, a fake scar, and whatever placebo effect that gives you. Experimental physicists have it easy too. They get to smash together the exact same particles in the exact same particle accelerator and see if anything changes when they fiddle with the velocity. If you believe the Pauli Exclusion Principle (and you should), many of these particles are literally physically indistinguishable from each other. Economists won’t get true counterfactuals until the invention of the time machine or socially-acceptable human cloning, at which point no one will care whether the Fed should hike rates by a quarter point next July anymore.
Instead of running expensive RCTs, many pragmatic cheapskate economists turn to arbitrary and rigidly enforced bureaucracy to create “natural experiments”. For example, suppose that you want to know the effect of smaller class sizes on pupil performance. You could try naively calculating whether kids in smaller classes get higher marks, but this would be subject to all kinds of biases. For example, it’s likely that schools in wealthy areas tend to have smaller classes than those in poorer areas. It’s also likely that kids from wealthy families tend to get higher marks anyway, regardless of their class size. And even within a single school, classes may not have been assigned with experimental validity in mind. Perhaps there’s a special group of all the high-scoring kids, or a different special group of all the low-scoring ones. You could try to control for this with some careful regression analysis, but it’s very difficult to know if you’re really controlling for everything that matters. You’d be better off doing what Angrist and Lavy did in their 1999 study and paying a visit to the delightfully bureaucratic Israeli school system.
In Israel, school classes are filled up with kids until they reach a size of 40. When, and only when, the 41st child enrolls, the class is split into two classes of 20 and 21. These smaller classes fill up until they contain a total of 81 kids, at which point they are split up into 3 classes, and so on. Angrist and Lavy exploited this “regression discontinuity” to rigorously compare the effect of class size on a child’s performance. Thanks to the Israel bureaucracy, they could be weirdly confident that the main driver of class size was whether or not the number of children in a grade reached these rigid and arbitrary boundaries. It seems very safe to assume that whether a grade has exactly 40 or 41 children is essentially independent of family wealth, kids’ natural ability, teacher ability, or anything else that might interfere with their study of the effect of class size on achievement.
An economist’s ideal web of red tape affords its subjects no wiggle room. Angrist and Lavy don’t want Israeli parents lobbying for their children to be moved to a smaller class, or worse, moving them to another school and taking them out of the natural experiment altogether. It might turn out that parents of more able children are more likely to lobby for them to be moved to smaller classes, or that rich parents are more likely to put their children into private school if they get stuck in a 40 kid mega-class. These types of mid-experiment changes upset the balance of the control and treatment groups, and are liable to reintroduce the kinds of sample bias that random assignment is meant to avoid.
Many natural experimentalists therefore make use of the most heartless and undiscerning bureaucracy on the planet, or as some people like to call it, nature. In their 2009 study, Maccini and Yang investigated the effects of early life poverty on a range of adult outcomes in Indonesia. They find that women born in a village at a time of above-average rainfall have significantly better health in adulthood, complete more years of schooling, and are more wealthy. They provide evidence that this is because the increased rainfall led to better crop yields, which led to families being able to better feed their children and help them develop, with long-term positive effects that do not simply dissipate after childhood. In this study, variable rainfall does the work of a committed but deranged experimentalist who ruins the lives of a treatment group, does nothing to a control group, and runs the numbers on their depraved but bias-free study at the end of the year. If there’s going to be humanitarian tragedy in the world then we might as well get some good data on it so we know what to do in the future.
A final example of the benefits of bureaucracy: many people have attempted to study the effect of military service on future earnings. A naive analysis might compare the earnings of random samples of veterans and non-veterans, see which number is bigger and wait for the Nobel Prize notification to arrive. But such an analysis would be vulnerable to the same types of sample bias as the naive classroom size analysis. Perhaps people who are more likely to serve in the military tend to earn less anyway, even without the effect of their service. For a truly bias-free experiment, we would need to randomly force people to join the army, and this is unfortunately one of the many things that society and university boards of ethics won’t let economists do. However, randomly forcing people to join to army was the exact aim of the Vietnam War Draft. Again, the sample groups were not perfect - plenty of people joined up voluntarily without being drafted, and plenty of draftees fled to Canada. But, as any experimentalist will tell you, you’ll always have some subjects fleeing to Canada, and these fuzzy edges can be dealt with via a cunning approach known as “instrumental variables”. The important part is that you get a big dose of random assignment to give your study its core power, and indeed some of the most statistically trustworthy analysis of veterans’ earnings has come from data from the Vietnam War Draft.
So the next time you are tripped up by some frivolous-seeming bureaucracy, whilst your identical twin sibling experiences receives free training as a database administrator and a year-long exemption from all sales taxes, you can be comforted by the fact that you are making an economist somewhere very happy.
Examples and inspiration taken from Mostly Harmless Econometrics by Angrist and Pischke.