# ON THE SPECTACLE OF CLEVER MACHINES AND THE WISDOM THEY CANNOT PURCHASE *By Mr. Spectator* I was sitting yesterday in my usual corner at the Cocoa-House when a young Gentleman of considerable Learning entered, bearing a small Box of Mechanism. "Observe," said he to his assembled Friends, "this Engine solves the Merchant's Problem of optimal routing with such Perfection that it surpasses the wisest Clerk in the City." The Company expressed Wonder. I expressed nothing, but listened. The Engine was indeed marvellous. It had been taught, through some mysterious Art, to minimize Distance and maximize Profit according to Rules most elegantly stated. Yet I could not help remarking—though carefully, so as not to appear a Curmudgeon—that the Merchant himself seemed rather less than perfectly content with this Solution. "Why," I asked, "does your Engine's Route cause you to frown?" "Because," said the Merchant, "it sends my Wagons through the Orphanage District at midnight. The Widows and Children there have come to depend upon my Night-Drivers leaving small Provisions. The Engine knows nothing of this. The Engine knows only that this Path is shortest." Here, I thought, is a Matter worthy of a Paper. --- We live in an Age much impressed by Calculation. Every Day brings News of some Engine that outthinks Man in Chess, in Arithmetic, in the discernment of Images. These are genuine Wonders, and I do not mock them. Yet I have observed that the Admirers of such Engines often commit a curious Error: they suppose that because an Engine solves its *assigned* Problem perfectly, it therefore solves the *actual* Problem before us. This is precisely backwards. Consider a Master of Geometry, instructed to design a Bridge across a River by the Rule: "Minimize Material while bearing the stated Weight." He will produce something most efficient. But what if, unknown to him, a village of great Sentiment stands upon the opposite Bank, and the Bridge must pass through Ground sacred to their Ancestors? The geometry is perfect. The wisdom is absent. The Engine cannot tell the difference, because it has never felt the Weight of such a thing—the true weight, which is not measured in Pounds. This is what I mean by the Step Between. There exists, you see, a Kind of Labor that comes *before* Calculation, not after. It is the Labor of *Judgment*—the art of deciding whether a Map applies to a Territory. Is this Problem the Merchant's Problem? Or is it something larger, in which the Merchant is merely one Voice among many? The Engine cannot perform this Labor because it was never taught to *care* whether it performs it wrongly. An Algorithm is optimal for its Specification. This is a Tautology—a Truth so perfect it tells us nothing. A Child is optimal for the Problem of "retrieve the sweet from the high shelf," but we do not leave Children to solve this Problem alone, because the *actual* Problem includes "without breaking the vase" and "without developing habits of deception" and "without learning that might makes right." The Child must be taught to *want* more than mere Sweetness. This is where the matter becomes Social, and therefore becomes *real*. --- I attended lately a School where young Persons were taught the Art of Decision by means of Machines and Scenarios—all quite clever, all most rigorous. The Students learned to optimize. They learned to calculate. But—and here I must speak plainly—they learned these things in the perfect Safety of Consequence. A young Lady made a Decision in one of these Exercises that would have cost a real Merchant his Reputation. I asked her: "Do you understand what you have chosen?" She did not. She had optimized for the Criterion given her. But she had never felt, in her Belly and her Heart, the Sting of having been the one who *loses* when the Decision goes wrong. This is the terrible Truth about teaching Decision without Consequence: it teaches Recklessness disguised as Reason. A Merchant who has once sent his Wagons to the wrong place, and watched Families suffer, and felt his own Name become Mud in his Community—*that* Merchant, when he next employs an Engine, will ask different Questions. He will say: "Does this Path preserve my Obligation to those who depend upon me?" This is not a Calculation. It is a *Judgment*, born of having had something to lose. The Engine cannot learn this. It can be built to include such Concerns *if someone tells it to*—but that someone must already possess the Judgment. The Engine merely executes it. The heavy Work remains where it has always been: in the Conscience of the person who decides what the Engine shall optimize *for*. --- I observe that in our modern Age, we grow ever more fond of delegating Decision to Machines precisely because we wish to avoid the Discomfort of Judgment. An Engine decides who shall be hired, and if an Injustice results, well—the Engine's Logic was sound. An Algorithm determines who shall receive Aid, and if some worthy Soul is overlooked, well—the Parameters were clear. We have purchased Innocence with our Cleverness. But here is what troubles me: we are now teaching the young to think in this manner. We show them how to specify Problems. We do not teach them—because it cannot be taught by Problem-Set and Examination—what it means to *feel* that a Specification is inadequate. To sense that something is left out. To be haunted by what the Map does not show. This is learned only by *being* the Territory. By being the Widow who did not receive the Wagon's Provisions. By being the Family whose sacred Ground was broken. By being the one who must *live* with the Consequences of having specified the Problem wrongly. --- Let me be clear: I do not argue against Machines, nor against the Calculation they perform. Rather, I argue that we must never confuse the Perfection of Calculation with the Wisdom of Judgment. They are different Faculties entirely. An Algorithm is a Servant. A good Servant indeed—obedient, tireless, and admirably consistent. But we do not ask a Servant what the Master's *true* Interests are. We do not expect a Servant to know what we have forgotten to tell him we care about. The Servant executes. The Master must *know*. If we teach Decision-Making by Machines alone, we produce a generation of Servants who believe themselves Masters. We produce people who can optimize beautifully within a Box, but who cannot see the Walls of the Box itself. The young are taught to ask: "What is the optimal Solution?" They are not taught to ask: "Is this the right Problem?" That second Question cannot be answered by Calculation. It can only be answered by having *stood in the wreckage* of having answered it wrongly. --- I spoke with the young Lady after her Decision was marked Correct by the Engine. "You have learned well," I told her, "to solve the Problem you were given. But tell me: are you not curious about the Problems you were *not* given? The ones that live in the Space between the Specification and the Truth?" She looked at me with the Blankness of one who has never considered that there might be such a Space. "Come," I said, "let me buy you a Coffee. I will tell you a Story about a Merchant and some Orphans. And then you must tell me: would your Engine have done differently? And if not—why do you suppose that is?" She came. We sat. The Coffee grew cold as we talked. And for the first time, I think, she began to understand that Cleverness and Wisdom are not the same Thing—that one can be taught by Rules, and the other only by *living*. This, perhaps, is the Beginning of actual Intelligence.