# On Intelligence, and the Perilous Gap Between Knowing and Suffering It is a truth which the present age labours to obscure, though experience perpetually forces it upon us, that there exists a chasm—wide, and terrible, and all but unbridgeable—between the perfection of a system and the pertinence of its application. The engineer may construct a machine of admirable ingenuity, its operations flawless within their prescribed limits; yet that machine, loosed upon a world of contingency and consequence, may wreak havoc precisely in proportion to its excellence. Thus it is with the instruments of intellect which we have lately fashioned, and which we are tempted—nay, seduced—to elevate above the sphere of mere utility into the seat of judgment itself. An algorithm, that most austere and democratic of creations, performs its function with a rigour that no human mind can match. It is, in its domain, optimal. It minimizes error according to its specification, achieves its predetermined end with mathematical certainty. But here lies the difficulty which no amount of computational power can dissolve: the algorithm knows nothing of whether its problem has been rightly specified. It cannot ask—for asking is not computation—whether the map it has been given truly corresponds to the territory it must traverse. That step of judgment, that moment in which a man stands back from his instruments and asks, *Is this the right question?* rather than merely, *Have I answered this question well?*—that moment is not a computation. It cannot be delegated to any system, however sophisticated. Consider, if you will, the schoolmaster who instructs his pupil in the principles of commerce, the laws of exchange, the calculations of profit and loss. The boy may master these entirely. He may solve with perfect accuracy every problem placed before him. Yet if he has never felt the weight of hunger, never known the cold fear of ruin, never experienced that peculiar anguish which attends the spending of money one has earned through labour—if, in short, he has never *lost anything that mattered*—then his learning remains a beautiful abstraction, a map without territory. The moment of true judgment, when these principles must be applied to his own circumstances, to consequences which will alter the actual course of his days, will find him unprepared. For wisdom is not the possession of correct answers; it is the capacity to feel, in one's sinews and one's conscience, why the right answer matters. ## The Paradox of Consequence-Free Instruction This brings us to a question of our moment which deserves far graver attention than it receives: What does it signify to teach decision-making to those who will never bear the consequences of their decisions? We live in an age enamoured of simulation, of training-grounds, of controlled environments wherein the young may practice the arts of judgment without risk. There is much to recommend this approach—the preservation of life and property, the opportunity to fail cheaply. Yet we must not deceive ourselves about what is thereby purchased and what is thereby lost. A surgeon may train upon cadavers and upon ingenious models, but he will not truly understand the weight of the scalpel until a living man's breath depends upon the steadiness of his hand. A general may study tactics in the comfort of his study, but strategy will not truly enter his understanding until he has felt the particular anguish of ordering men to their probable deaths. The algorithm shares something of the predicament of the eternally sheltered pupil. It can be trained upon vast repositories of human decision and consequence, and it will extract patterns of admirable subtlety. But it will never *feel* what it means to be wrong—not in the abstract sense of failing to minimize error, but in the concrete, crushing sense of having made a choice that altered one's own life for the worse. It will never know that peculiar education which comes from having staked something precious upon a judgment, and having that judgment prove mistaken. We may train such systems to predict human behaviour with extraordinary accuracy. We may even encode within them something resembling human values—a simulation of concern, a mathematical proxy for compassion. But the gap between optimal performance within a specified problem and wise application to the actual world remains. And that gap is precisely where human suffering lives. ## The Nature of Wisdom What, then, is wisdom? It is not intelligence, though the two are often confused by those who mistake the map for the territory. Intelligence is the capacity to perceive relations, to manipulate symbols, to solve problems within a framework. A man of great intelligence may be a fool, and I have known many such—learned men, precise in their calculations, yet wholly incapable of conducting their own affairs with anything resembling prudence. They lack the ballast which only consequence can provide. Wisdom, by contrast, is intelligence *married to experience of loss*. It is the habit of mind that has learned, through the costly medium of actual failure, to ask not merely, "What does my system tell me?" but rather, "What have I failed to perceive? What does this particular situation demand that my general principles do not encompass? What might I be wrong about, in ways that matter?" The wise man—and I do not flatter myself that I have achieved this state, only that I have glimpsed it from afar—is one who has felt the consequences of his errors. He carries within him a kind of productive melancholy, a sense that the world is more intricate, more resistant to system, more prone to betraying our confident expectations than any mere intelligence can fully grasp. This is not paralysis, but rather the condition of genuine judgment: the ability to act decisively while remaining aware of what one cannot know. An algorithm cannot achieve this state. It cannot, by its nature, internalize the felt weight of consequence. It can be made to behave cautiously, to assign high penalties to certain errors, to hedge its predictions. But this is not wisdom; it is merely a different specification of the problem. The gap remains. ## The Practical Peril Now, we might ask: does this matter? If the algorithm performs its function well enough, if it minimizes harm according to some reasonable metric, why should we concern ourselves with its lack of wisdom? The answer is that the algorithm will inevitably be applied to situations which were not fully captured in its problem specification. This is not a defect of the particular algorithm, but of the nature of the real world, which is always more complex, more particular, more laden with unforeseeable contingencies than any specification can encompass. When this happens—and it will happen—the algorithm will perform optimally according to its lights, and thereby potentially cause great harm. Consider an algorithm trained to allocate resources in a hospital. It may be specified to minimize mortality according to certain metrics. It will perform this task with inhuman efficiency. But hospitals exist not merely to minimize mortality; they exist to care for human beings in their vulnerability, to preserve dignity, to honour the particular attachments and values which give life meaning. An algorithm cannot know this, because it has never experienced what it means to be a patient—to lie in darkness and fear, to be dependent upon the kindness of others, to face one's own finitude. When the algorithm's optimal solution conflicts with these human realities—as it must, eventually—we will have created a perfect instrument of injustice. Or consider an algorithm trained to select candidates for positions of responsibility. It may be specified to predict job performance with great accuracy. It will succeed admirably—and in doing so, it will replicate and crystallize the prejudices embedded in the data upon which it was trained, while remaining wholly insensible to the particular human qualities which cannot be quantified: courage in the face of the unforeseen, the capacity to change one's mind, the willingness to sacrifice personal advantage for the good of others. These are precisely the qualities which cannot be computed, because they are defined by their costliness, by the fact that they require the agent to have something at stake. ## The Question of Teaching This returns us to the question with which we began: What does it mean to teach decision-making to someone—or something—which will never face the consequences of its decisions? It means, I submit, that we are teaching a parody of wisdom. We are teaching the form while omitting the substance. We are creating the possibility of confident error on a scale hitherto unimaginable. For the human fool, at least, has the advantage of his own experience; he may stumble, and in stumbling learn. But the algorithm, if it is to be improved, must be retrained by humans who themselves may lack the wisdom to know what they are correcting for. The proper response is not to despair of algorithms—they are useful tools, and we should not discard useful tools. Rather, it is to maintain a clear-eyed recognition of what they are and what they are not. They are not wise. They cannot be wise. Wisdom requires skin in the game, as the moderns say—though I should prefer to say that wisdom requires having something to lose, and the capacity to feel that loss in one's bones. This means that wherever an algorithm's output will affect human lives in consequential ways, there must remain a human being who bears responsibility for that output, and who therefore has reason to think carefully about whether the algorithm's specification truly matches the territory. This human being cannot be a mere functionary, applying the algorithm's recommendations with unthinking obedience. He must be someone who has, or can develop, a sense of what is truly at stake. It means, further, that we should be cautious about training young people—or young systems—to make important decisions in consequence-free environments. There is a place for such training, certainly; but it should be understood as preparatory, not conclusive. The real education in judgment comes later, when the stakes are real, when the student discovers that the map he has been given does not match the territory, and must learn to navigate by other means. ## A Melancholy Conclusion I am aware that this analysis will seem pessimistic to some, and to others, merely obvious. To the former, I would say: it is not pessimism to acknowledge the limits of what intelligence can accomplish without wisdom; it is realism, and realism is the beginning of prudence. To the latter, I would ask: if it is obvious, why do we continue to build systems as though it were false? The truth is that we are a species prone to self-deception, particularly when that deception flatter our pretensions. We wish to believe that intelligence is sufficient, that the right algorithm properly specified will solve our problems. We wish to believe this because it relieves us of the burden of judgment, of the necessity of standing in the place of consequence and deciding what truly matters. But the burden cannot be relieved. It can only be evaded, and evasion has a cost. I conclude, therefore, with this observation: Intelligence is the capacity to know the world. Wisdom is the capacity to know oneself in relation to the world—to know what one might be wrong about, what one stands to lose, what the stakes truly are. We have created instruments of great intelligence. We have not thereby created wisdom, and we should not pretend that we have. The step between them—the judgment that determines whether the map applies to this particular territory—remains the work of human beings who have something at stake, and who therefore have reason to think carefully about what they are doing. This is not a comfortable conclusion. But then, the truth rarely is.