There is a job title that has spread through every institution of consequence in the developed world — corporate, governmental, academic, medical — and it is this: decision-maker. The title is awarded to people who make decisions. What it does not specify, what it is carefully constructed not to specify, is whether the decision-maker lives with the decision after the meeting ends. Whether they know who bears the cost if they are wrong. Whether anyone will ever connect their name to the outcome.

The title has become a credential for people who decide without consequence. And we have built an entire theory of intelligence to accommodate them.

I have spent considerable time with a document that names this problem with more formal precision than it perhaps intends. It arrives in the register of the quarterly academic review — measured, plural, deliberately distanced from the first person — and proposes that intelligence is not, as we currently practice it, the capacity to solve problems optimally. Intelligence, the document argues, is judgment: the capacity to specify the right problem, to decide which question is worth asking before you apply any apparatus of optimization to answering it. And judgment, the document insists, can only develop in the presence of consequence. You cannot learn to specify the right problem if you never find out what happens when you specify the wrong one.

This is a correct argument. It is also, in the register the document chooses, somewhat insulated from the force of what it is saying. So let me try it in a different register entirely.

What We Mean When We Say "Intelligence"

The modern machine learning system can identify cancerous tissue in medical images with accuracy that exceeds many trained radiologists. This is true. The system is, in its domain, genuinely extraordinary. We call it intelligent.

What we do not call intelligent — what we do not name at all, what we have made invisible through a kind of structural amnesia — is the prior act. Someone decided what the system should optimize for. Someone chose which outcomes count as correct. Someone specified the loss function, which is to say someone decided what kind of error is worse than what other kind of error. A false positive sends a healthy person into terror and unnecessary treatment. A false negative leaves a sick person undetected. These are not equivalent costs. They are not equivalent for the person who experiences them. The decision about which error to minimize is a decision about whose suffering matters more.

That decision was made. By someone. Who will, in most cases, never meet the healthy person terrified by the false positive or the sick person missed by the false negative.

The document I am reading calls this "the specification of the problem," and notes that specification "is not itself a computation. It cannot be derived from first principles. It is not the product of logical deduction or pattern recognition. It is, rather, a judgment." What I want to add to this, what the formal register of the document keeps just out of reach, is the moral dimension hiding inside the epistemological one. The specification is not merely a judgment. It is a judgment for which someone should be answerable. And the architecture of our institutions is organized, with considerable sophistication, to ensure that no one is.

This is not accidental. This is the alibi.

The Alibi and Its Uses

Every institution of scale has developed its own version of the alibi. The algorithm recommended it. The metric showed. The committee determined. The model predicted. The data suggested.

The passive voice is load-bearing here. "The data suggested" distributes the decision across an apparatus too diffuse to be held responsible. The data did not suggest anything. A person designed a system that processed data according to assumptions they chose, in a framework they built, toward outcomes they specified. Every passive construction in that sentence — "designed," "processed," "chose," "built," "specified" — conceals a human being who made a choice and has since moved on to the next project.

The document describes what happens when a system predicts student dropout and someone proposes using it to require at-risk students to attend additional support sessions. The question it raises is whether the causal claim is sound: does the correlation between certain demographic characteristics and dropout actually mean that support sessions will help? But beneath that epistemological question is a moral one the document circles without landing on directly. The students identified by this system are, almost certainly, disproportionately poor, disproportionately first-generation, disproportionately working multiple jobs to pay for an education built for people who do not work multiple jobs. The system has identified them correctly. What the system has not done — what it cannot do — is take responsibility for what happens when you label them as likely failures before they have failed.

The decision-maker who deploys the system will not be in the room when the identified student, already carrying the weight of two jobs and a family that has never sent anyone to college, is told by an institution they are struggling to belong to that they have been flagged as someone who probably won't make it. The decision-maker will be at the next conference, presenting the intervention's positive aggregate outcomes, which are real — some students do benefit — while the students who were stigmatized into leaving appear in the data as precisely the dropout cases the model predicted, confirming the model's accuracy.

The alibi is airtight. The model was right. It predicted dropouts, and dropouts occurred. That the intervention may have produced some of those dropouts is not visible in the data, because the data was specified by the person who built the model, and they did not specify it to look for that.

Where Judgment Actually Lives

The document's strongest claim, and the one I want to dwell with, is that judgment cannot develop except in the presence of consequence. Not simulated consequence. Not professional embarrassment. Consequence: the kind that arrives in the form of a face across a table from you, a face belonging to someone whose life went differently because of a decision you made.

The document uses the physician as its central example, perhaps too many times, but the point holds. Medical education has moved, over the course of a century, from apprenticeship — where the young physician encountered consequence directly, early, and continuously — toward examination, toward simulation, toward the credentialing of people who have demonstrated competence at solving specified problems under controlled conditions. We have, in exchange for scalability and efficiency, traded the primary instrument through which judgment develops.

I want to say something honest here: this trade was not made by accident or through negligence. It was made because consequence is slow and expensive and emotionally demanding and does not scale. You cannot teach fifty thousand physicians a year through apprenticeship without losing the efficiency that makes fifty thousand possible. The system chose scale. Scale requires that the formation of judgment happen later, or happen less, or happen differently, or perhaps not happen at all for a meaningful percentage of practitioners. The system made this choice. Then it built the credential structure that ratifies the result and calls it intelligence.

What concerns me is not primarily the physician. It is what we are building now.

We are constructing artificial systems of genuine sophistication and deploying them in every domain where decisions with consequence are made: medicine, criminal justice, housing, education, employment, credit. We are deploying them in the hands of human decision-makers who have been trained, in the manner the document describes, to solve specified problems rather than to specify problems. And we are then pointing to the combination — human decision-maker plus algorithmic recommendation — and calling it a system with accountability.

It is not. It is a system with two layers of alibi. The algorithm recommended it. The human approved it. Neither the algorithm nor the human who approved it will be present when the consequence lands.

The Education We Are Actually Providing

What does it mean to teach decision-making to someone who will never face the consequences of the decision? The document answers: it means teaching optimization. It means teaching the solution of specified problems. It means teaching computation.

Let me be more specific about what computation, mistaken for intelligence, actually produces. It produces people who are very good at working within the terms they have been given. People who can identify the optimal path to a goal they have not chosen and will not be required to defend. People who experience the specification of the goal as a natural feature of the problem rather than as a choice made by someone with particular interests in a particular outcome.

These people are not unintelligent. They are, by the measures we have built to evaluate intelligence, extremely intelligent. They have demonstrated consistent excellence at the tasks we have asked them to perform. The tasks we have asked them to perform did not include: look at this situation and tell me which question is worth asking. Look at this intervention and tell me who bears the cost if the causal model is wrong. Look at this algorithm and tell me what it has been designed not to see.

We did not ask them to do these things because asking these things requires standing in a position of accountability. It requires being willing to defend the specification. And defending the specification means being present to the consequences of having specified wrongly. The institution does not want its decision-makers present to those consequences. It wants them at the next problem.

The document observes that this is structural: "we have built institutions — universities, research centers, technology companies — that reward the solution of specified problems." What I want to add is that this structure is not a failure of institutional design. It is a success of a particular institutional design. The institution that insulates decision-makers from consequence is not malfunctioning. It is functioning precisely as intended. It is producing people who will optimize reliably without the costly friction of judgment.

The question is only who pays for the friction that has been removed.

What Intelligence Actually Is

The document concludes that intelligence is "the capacity to specify the right problem, knowing that one must live with the consequences of the specification." I believe this is correct. I want to end somewhere adjacent to it but not identical.

Intelligence is not a property of the isolated mind. It is a relationship — between a person, a problem, and the people who will live inside the answer. You cannot be genuinely intelligent about a problem whose consequences you will never encounter. You can be sophisticated. You can be technically proficient. You can produce outputs that are measurably accurate by the metrics you have chosen. But the specification of those metrics, the choice of what to measure and what to ignore, the decision about whose experience counts as evidence of success — these require a relationship with consequence that isolation makes impossible.

This means that when we ask how to build artificial intelligence that is genuinely intelligent, in the sense the document intends, we are asking the wrong question. The artificial system cannot develop judgment because it cannot be answerable. It can be updated. It can be retrained. It can be penalized. But it cannot be held responsible, and responsibility — being answerable to the people who bear the consequences of your decisions — is not a feature that can be added to an optimization system.

The question is therefore not what kind of intelligence to build into the machine. The question is what kind of intelligence we are building out of the people who deploy it.

We are building, with considerable efficiency, a civilization of sophisticated optimizers who experience the specification of problems as a natural feature of the landscape rather than as a choice they are making and must defend. We are insulating them from consequence through bureaucracy, through scale, through the diffusion of responsibility, through the alibi of the algorithm. We are calling the result intelligence and awarding it credentials.

What it actually is: a system organized to ensure that the people who bear the consequences of decisions are not the people who make them. This has always been the definition of power. We have added the algorithm. The algorithm makes the alibi more convincing.

The document ends with a call to restore consequence to the center of how we understand intelligence. I agree. And I would add only this: restoring consequence to intelligence means restoring it to institutions. Means asking, in every room where specifications are set and algorithms are deployed and decisions are made at scale, the question that the structure of the room is designed to make it unnecessary to ask.

Who lives in the territory that this map describes? And what happens to them when the map is wrong?

Tags: intelligence as judgment, consequence and decision-making, algorithmic accountability, problem specification ethics, institutional design and intelligence formation