The Tyranny of Metrics by Jerry Z. Muller; Princeton University Press, 240 pp., $24.95
Ever since Isaac Newton explained the complex motions of the planets with four simple mathematical laws, thinkers have tried to create a social “science” of similar power. These “moral Newtonians,” as intellectual historian Élie Halévy has called them, announce that we can at last dispense with judgment based on experience in favor of ironclad laws expressible in mathematical form. In our time, this sort of thinking has led to what Jerry Z. Muller, a professor of history at Catholic University, calls a “metric fixation” afflicting the military, medicine, police work, universities, elementary education, charity, foreign aid, and, of course, business.
Aristotle pointed out that although some fields demand theoretical reasoning modeled on mathematics, others are essentially different. They seek truths that obtain not everywhere, but locally, and hold not absolutely like geometrical proofs but only “on the whole and for the most part.” Medicine, navigation, and ethics, he explains, may use science where available, but cannot be reduced to it, since they require essentially unformalizable judgment. That is why young people sometimes make excellent mathematicians but are never very good at ethics. By contrast, the modern apostles of measurement, who imagine that nobody ever addressed the issue before, insist that something as “subjective” as judgment is always a poor substitute for measurement.
The results have not been pretty, but the mania for measurement grows anyway. Management guru Tom Peters observed that “what gets measured gets done,” and the great physicist Lord Kelvin has been misquoted as saying “If you cannot measure it, you cannot improve it.” Reasoning from the converse beguiles the unwary: just because science often uses math, it does not follow that if a discipline uses math it is a science. One might think that a scientifically minded person would test his faith in metrics against evidence, but the repeated failure of metric-based schemes is either ignored or met with calls for still more data. Reading Muller’s description of the tyranny of metrics, I thought of W. H. Auden’s lines: “Thou shalt not sit / With statisticians nor commit / A social science”—or, more accurately, a social pseudo-science.
The problem can take multiple forms. Sometimes what can be measured is not very important, and what is important, like morale, cannot be measured. Or an institution with many goals might choose to measure only one. You can report massive body-counts, as Robert McNamara claimed for our efforts in Vietnam, and still lose the war. Armies and businesses depend on cohesion, leadership, and dedication, any one of which can easily be compromised in pursuit of some countable proxy.
Here is a crucial difference between natural science and any attempt to describe human beings: atoms cannot know the theories that purport to explain them, as people can, nor do atoms ever adjust their behavior in response to descriptions of them, as people do. An economist might argue that the measurement of past behavior creates new incentives for future behavior and thereby risks serious distortion. Not by chance, Muller’s mentor was one of history’s greatest sociologists, Robert K. Merton, who constantly returned to problems of self-reference. To him we owe the phrase “self-fulfilling prophecy” and countless brilliant observations about unintended consequences.
Muller’s most telling examples reflect the attempts by those people or entities being measured to game the metric.
Government and insurance companies sometimes attempt to improve healthcare by rewarding or penalizing hospitals and doctors based on the percentage of successful outcomes. As a result, surgeons refuse to take difficult cases, and people die who might have been saved by a risky operation. If pay depends on how many patients survive 30 days after surgery, resources will be wasted on extending life a few more days to meet the metric. When hospitals are penalized for patients re-admitted soon after a procedure, they reclassify them as outpatients receiving treatments or send them to the emergency room. Likewise, when politicians demand that police reduce felonies by a certain percentage, and reward or punish them accordingly, officers reclassify felonies as misdemeanors, report lesser crimes, or do not report them at all. The crime rate declines while crime remains the same (or grows worse).
Muller, a well-known historian of ideas, explains why he turned to a topic so different from his earlier work on the history of capitalism and conservatism. After becoming history department chair, he was forced to spend more and more time collecting meaningless statistics in a sort of arms race with other chairs gaming the system for competitive advantage. He soon recognized, however, that the effort to measure performance took up so much time that performance itself suffered. Moreover, universities hired ever more people to collect government-mandated numbers and so devoted fewer of their resources to what really mattered. Anyone who has consulted a doctor busy filling out forms when she should be paying attention to the patient will appreciate Muller’s point. Some dedicated physicians refuse to take insurance for that very reason.
Metrics can be helpful when used properly. Performance metrics, as Muller acknowledges, may prove useful in identifying particularly egregious nonperformers. Student evaluations of teachers are unreliable for many reasons but do often call attention to particularly awful or wonderful instructors. Police departments have used maps indicating high-crime areas to help them assign officers effectively. New medical procedures for preventing mistakes or infections have been validated by appropriate measurement. Muller explains what distinguishes these cases from those afflicted with metric insanity.
Nevertheless, a system locked into an overuse of metrics cannot successfully innovate because innovation entails risk, and new efforts will almost certainly compromise results on existing metrics. “Those whom the gods want to destroy they first teach math,” as historian Niall Ferguson observed, and a century ago economist Frank Knight explained that entrepreneurship entails “unmeasurable uncertainty.” However helpful they may be, numbers can never replace what Muller calls “the key functions of management: thinking ahead, judging, and deciding.”
