On Intelligence Without Stakes
# On Intelligence Without Stakes
The question is not really about intelligence at all. It is about something much simpler and more troubling: the difference between knowing and living.
An algorithm is a machine for solving a problem that someone else has already defined. It solves that problem perfectly, within its specification. This is not intelligence. It is obedience to a clear instruction. We have confused the two because obedience, when performed at sufficient speed and scale, looks like thought.
But consider what happens when the problem specification does not match reality—which is to say, always.
A chess algorithm plays chess. The problem is stated precisely: capture the opponent's king while protecting your own. The rules are fixed. The board is finite. The algorithm wins, sometimes brilliantly. This looks like intelligence because chess is difficult. But the algorithm has never had to decide whether to play chess at all. It has never wondered whether the game was worth the cost, whether the opponent was worth defeating, whether the victory would matter tomorrow. These questions are not part of the specification. They are not computations. The algorithm is not equipped to ask them, and we are not wise to pretend it is.
Now consider a man who must decide whether to testify against his brother in court. The algorithm has no equivalent problem. You cannot specify this decision in advance because the specification itself is the problem. What matters is not the logical chain—any clever mind can construct arguments either way—but the weight of consequence. The man will live with his choice. His brother will live with it. His family will live with it. The decision is his to make because he is the one who will carry it.
This is where wisdom enters, and where nearly all our contemporary thinking about intelligence fails.
We have built systems that are asked to make consequential recommendations about people's lives without ever facing the consequences. A loan algorithm denies credit. A hiring algorithm rejects an application. A parole algorithm recommends detention. These systems are optimal for their problem specifications. They minimize error according to whatever metric we have fed them. They are not wise. They cannot be wise. Wisdom is not a computational property. It is a property of agents who must live with the results of their judgments.
The algorithm's developers face some consequences—reputation, perhaps legal liability. But they do not face the consequences that matter most: they do not live the life that is diminished by the wrong decision. The algorithm faces none. And the person making the appeal, the person denied the loan, the person locked away—they face all of them.
This creates a kind of corruption that is not obvious because it wears the mask of rationality. The algorithm is not biased in the ordinary sense. It does not hate anyone. It simply cannot know what it does not know, and what it cannot know is what it would feel like to be wrong.
Wisdom requires this knowledge. Not abstract knowledge of suffering—any fool can read about that. But the felt knowledge that comes from having something to lose. From having made a decision and lived through the consequences. From having been wrong and having to continue anyway.
You cannot teach this through computation because computation does not stake anything on being right. You cannot teach it through simulation because simulation is not life. You can only teach it by requiring people to make real decisions about real consequences and then holding them responsible.
This is why we used to require judges to live in the communities they judged. Why we used to require officers to live among the soldiers they commanded. Why we used to require that those who made policy had to live under it. Not out of sentiment, but out of hard necessity. A man who will never face the result of his judgment is not fit to make it.
We have inverted this. We have built systems of decision-making in which the most consequential judgments are made by those furthest from the consequences. We have called this efficiency. We have called this objectivity. It is neither. It is only cowardice dressed up as mathematics.
The step between the algorithm and the territory—between the specification and the actual life affected by the decision—is not a computational step. It is a moral step. It requires judgment. And judgment, real judgment, cannot be separated from stakes. You cannot have wisdom without the possibility of being wrong in a way that matters.
This is what we have forgotten. And we will not remember it until we make people live with the consequences of their optimizations.
Tier 7: Wisdom
0
Comments
No comments yet.
Sign in to comment.