# On the Fashionable Error of Diagrams ## *Being Observations on Causal Knowledge and the Diagram-Makers* I have lately spent some hours in a coffeehouse much frequented by men of learning—natural philosophers, statisticians, and what they call "causal inference researchers"—and I observed among them a curious species of confidence. They speak much of transparency, of laying bare their assumptions in diagrams before all eyes. Yet I noticed that the laying bare of an assumption is not the same as its correction. Indeed, I have found that a man who states his error plainly is often more dangerous than one who conceals it, for the plain statement persuades us of its honesty. This began when a young scholar showed me a diagram. It contained boxes connected by arrows—a model of the world, he assured me, rendered explicit and therefore rigorous. "You see," said he, "we have expelled the vague language of cause, and returned it only when the diagram permits. We are scientific." I could not forbear to smile. For a century, I said, natural philosophy had indeed expelled causal language as imprecise. But what was purchased by this exile? The scholar seemed puzzled. I explained: the appearance of rigor, perhaps, but not rigor itself. For when Professor Pearl and others restored causal language to respectability, they did not restore it naked. They dressed it in diagrams. And now the diagram has become the tyrant—more absolute than the language it replaced. --- **On Transparency Without Truth** Here is what troubles me most. The scholar told me his diagram was "transparent"—that any man might read it and judge its assumptions. This seemed to him the height of virtue. Yet I observed that transparency and validity are not married to one another, though they are often mistaken for twins. Consider a gentleman of my acquaintance, a man of considerable learning, who published a study of whether a certain medicine cures a certain disease. His diagram was exquisitely drawn. He showed how the medicine affected the body, how the body affected recovery, how age affected both. Every assumption was visible, as though written in the clearest hand. Readers praised him for his honesty. Yet the diagram was wrong. Not in its transparency—that was genuine. But in its substance. He had omitted a factor he did not know to include. Or rather, he had *assumed* he need not include it. The diagram was explicit about what it contained; it was silent about what it did not. And his silence was mistaken for knowledge. This is the paradox of the diagram-maker: the more clearly he states what he assumes, the more his readers mistake clarity of statement for correctness of assumption. The diagram becomes a confidence trick, performed in good faith. --- **On Who Decides the Diagram** But there is a second trouble, deeper still, and here I must speak plainly of a matter that touches on the social order. The diagram, you see, does not draw itself. A man must decide what goes into it. And this is not a question that data can answer. The data speaks *after* the diagram is made, not before. The researcher must *assume* the diagram before the data can confirm anything. Now, who makes this assumption? And more importantly—on what authority? I observed that in the coffeehouse, the researchers spoke as though the diagram were a discovery, like finding a new star. But it is not. It is a *choice*. And choices made by learned men often reflect their positions, their interests, their invisible assumptions about how the world works. Let me be concrete. Suppose we wish to understand whether poverty causes crime. A researcher draws a diagram. She includes certain factors as causes, others as effects, still others as irrelevant. But what she includes depends on her theory of human nature, her beliefs about society, her convictions about what matters. A man of conservative disposition might draw the diagram so that poverty appears as a cause of crime, but also so that moral weakness appears as a *prior* cause of both poverty and crime. Thus the diagram "discovers" what he already believed. Another researcher, of different sympathies, might draw the diagram differently. He might include systemic oppression as a cause, or desperation as a cause, arranging the arrows so that poverty's effect appears even more dire. Both diagrams are transparent. Both are explicit. Both can be rigorously analyzed once drawn. But they are not equally true, for they are not equally chosen. The choice precedes the rigor. --- **On the Social Dimension** And here the matter becomes properly social, which is to say, it becomes a question of power. For who has the authority to decide what the diagram shall contain? In our universities, this authority rests with those who have already succeeded in drawing diagrams—the established researchers, the holders of grants, the men (I note it is still mostly men) whose diagrams have been published and praised. A young scholar wishes to study whether a policy helps or harms. She proposes a diagram. But the diagram must be approved by those already in authority. They look at her assumptions. If her diagram conflicts with their diagrams—if her arrangement of boxes and arrows suggests a different truth than the one they have already published—they may reject it as "unscientific," or "lacking in rigor." Yet the rigor, as I have observed, lies not in the diagram itself but in what one does *after* drawing it. The diagram is an assumption. The rigor is in the analysis of that assumption's consequences. I have seen this in the coffeehouse debates. When a researcher's diagram is approved by the powerful, it is called "transparent" and "rigorous." When a researcher of less authority proposes a different diagram, it is called "lacking in justification." But the justification for any diagram lies outside the diagram itself—it lies in the world, in the evidence, in the theory that preceded it. This is why I say that transparency is not validity. A diagram can be perfectly clear and still perfectly wrong. And once a diagram is accepted, it becomes difficult to challenge, because it is now "explicit," and any challenge appears to be a challenge to honesty itself. --- **On What It Means to Know a Cause** So what, then, does it mean to know the cause of something? I confess that after much conversation in the coffeehouse, I am less certain than before. It seems to mean something like this: to have a diagram that correctly represents the world's structure, and to have analyzed that diagram rigorously so that one may predict what will happen when one pulls a lever. But the diagram comes first. And the diagram cannot be derived from data alone. It must be assumed. This is not a failure of Professor Pearl's restoration of causal language. His tools are indeed provably correct *given* a diagram. The fault lies not with the tools but with the assumption that the diagram is innocent—that it merely represents what is, rather than what the diagram-maker believes to be. The honest researcher, it seems to me, must do three things: First, he must state his diagram clearly—this Pearl's methods encourage, and it is good. Second, he must analyze it rigorously—this too Pearl's methods permit, and it is good. But third—and this is what I find most often neglected—he must acknowledge that his diagram is a *choice*, made before the data could speak. He must ask: Who chose this diagram? On what authority? What other diagrams might have been drawn? Whose interests does this diagram serve? For it is in this third act that the social dimension reveals itself. The diagram is not innocent. It is a claim about power: the power to say which things cause which other things, and therefore which interventions matter and which do not. --- **A Concluding Observation** I remarked to the young scholar that the coffeehouse itself is a kind of diagram. We have arranged ourselves here with certain assumptions: that conversation is good, that merchants and lords and scholars may sit together, that truth is better discovered in company than in solitude. These assumptions are not derived from data. They precede it. They shape what data we gather and how we interpret it. He seemed troubled by this. "Then nothing is truly scientific?" he asked. "On the contrary," I replied. "Everything can be. But only if we remember that science begins with assumptions, not with data. The rigor comes in testing those assumptions honestly, and in being willing to revise the diagram when the world resists it. The vice comes in pretending the diagram was discovered rather than chosen, and in using transparency as a shield against doubt." The scholar left the coffeehouse thoughtful. I ordered another dish of coffee. It is my hope that others, reading this, might do the same—not order coffee, though I recommend it, but rather think carefully about the diagrams they are shown, and about who drew them, and why. For that, I believe, is what an educated person owes to himself and to society. *Your humble servant,* **The Spectator**