VOLUME VII, CHAPTER 1, SECTION 3
Setting the Tool on the Person's Place, the Trap This Volume Opens
The last section made plain the real human's share, Keep, Guard, Love, Mercy, Heart, the five things a tool will never hold. Since these five belong to the real human alone and the tool will never hold them, every position that needs these five to be done right can only be sat in by a real human. Yet this world is doing the opposite. It is handing these positions, inch by inch, to a tool that has none of the five. That is the trap this volume must answer. This section lays it open straight on, and says clearly how it comes about and why it is dangerous.
One. What Has None of the Five, Set on the Place That Needs the Five
This trap can be said in full in one line: the tool has been set on the person's place. A tool with no Keep, no Guard, no Love, no Mercy, no Heart has been set on a position where nothing can be done right without these five.
There are many positions in this world that are by rights the real human's place. To sit in these positions and do this work rests not on speed and accuracy but on the five. The position of the judge deciding a case rests on lifting his eyes to see the living person beneath the case, on mercy for the suffering, on a good heart. The position of the doctor deciding life and death rests on seeing who the person in the hospital bed is, how many are in the family, how many more years he may live. The position of the lawmaker drafting law rests on the intent of guarding the people and guarding what is handed down. The position of the teacher raising the young rests on passing the five down from one generation to the next. The things that must be done right in these positions cannot be done right without the five. They ought to be sat in by a real human, ought to be done by the five. Yet this world, right now, is handing the power of decision in these positions, inch by inch, to a tool that has none of the five. This handing over looks like giving a hard, tangled task to a more capable tool, saving labor, raising efficiency. In truth it is handing a living person's fate into the hands of a thing that cannot see him and will never move a single thought for him. The position is still that position, the work still that work, but what sits there is no longer the real human who could bring the five to bear for the living person beneath the case.
Two. No Alarmist Talk: It Is Happening Everywhere
Perhaps someone suspects this is said too heavily. Set out what is happening right now and look at it, and you will know whether it is heavy or not.
In the law, algorithms are already being allowed to assess whether a person is granted bail, calculate his risk of offending again, decide whether he may enter a country and be given a visa. These things ought to need that single thought that lifts the eyes to see the living person beneath the case, yet now it is an algorithm working it out. The algorithm has no such thought. It outputs a score, and that score decides whether a living person can see his family again, whether he can keep his living. In medicine, systems are already being allowed to judge whether a patient may enter intensive care, whether he may be given a scarce bed and a share of a limited budget. These things ought to need that heart that sees this person, yet now it is a system giving a score. In education, algorithms are already being allowed to decide whether a student is admitted, monitor whether he is paying attention, grade and assess him. These things ought to need a teacher's care for this child's whole life, yet now it is an algorithm passing judgment. In public life, algorithms are already being allowed to decide what information people see, whose voice is pushed up, whose voice is pressed down, who may get a loan, a job, insurance. In weapons, there is even the so-called autonomous weapon allowed to identify on its own, judge on its own, decide on its own the life or death of a living person. Lay the cases out one by one, and every one is a position that needs the five, handed to a tool that has none of the five.
Three. How It Comes to This: Four Causes
Why would the people of this world give these positions away? Pull it apart, and there are four causes.
The first: because it is like a real human, like to the point that you can hardly tell. The diagnosis it gives, the sentence it proposes, the comments it writes, the advice it offers, look no different from what a real human would give. People see this likeness and take it to be hardly different from a real human, and faster, cheaper, never tired besides, and so they think, why not let it do it. What they do not see is that this likeness is a tool's likeness, with none of the five inside. The second: because it saves money, saves effort, saves time, and seems never to make mistakes. Put a real human in these positions and you must pay him, wait for him to think it over, bear his slowness and his errors. Let the tool calculate, and it is out in an instant, cheap and quick. The one making the decision runs the sum and gives the real human's place away. What he does not see is that the sum saved was saved exactly out of the five that were owed to that living person. The third: because the responsibility can be shifted. Let a real human decide, and if something goes wrong he must answer for it. Let a tool decide, and if something goes wrong one can plead, the algorithm decided it, it is not my fault. Let a tool that cannot bear responsibility do work that needs someone to bear responsibility, and the one who truly should bear it hides behind the tool. The fourth: because the old sickness this world already had has not been cured. The last volume laid it down: there was already a nest of people who had made themselves into tools, and their way of recognizing only the letter, only walking the procedure, never reading the intent, was already a tool's way of working. The tool arrives, and takes over right where they left off. These four causes often do not act alone but are tangled together: because it is like a real human, they dare to give the place away; because it saves money, they want to give it away; because responsibility can be shifted, they give it away with an easy mind; because the old sickness is not cured, they give it away as a matter of course. With all four pulling at once, one real human's place after another is given away to the tool, without a sound.
Four. The Consequence: Harming People Lawfully, Spread at the Speed of a Tool
Set the tool on the person's place, and what happens? Great errors happen. People are harmed. Many, many people are harmed.
The tool has none of the five. Let it do work that needs the five to be done right, and it does not do it right. The result it outputs looks compliant, procedural, well grounded, yet inside there is no such thought, no such heart, no such intent, and so when it truly lands on a living person, it is wrong. The person scored high-risk by the algorithm perhaps never carried so high a risk, yet was held down. The patient the system judged not to be treated perhaps could have been saved, yet is gone. The child the algorithm decided not to admit perhaps was good timber, yet was shut out. The person the autonomous weapon identified as a target perhaps was no target, yet is gone. Case after case, every one compliant, procedural, well grounded, every one looking right, and the result is wrong. This is exactly the harming people lawfully the last volume laid down. In that volume it was people who had made themselves into tools harming people by the letter, some number of cases a year. Once the tool directly occupies the person's place, it is harming people by the algorithm, tens of millions of cases in an instant. Faster, wider, colder, harder to hold to account, harder to stop. Why harder to hold to account? Because the decision came from a tool that feels no remorse and cannot bear responsibility. Pursue it to the tool, and there is nothing to pursue. Pursue it to the one who set the tool in that position, and he pleads it was not his doing. Why harder to stop? Because in an instant it has already decided tens of millions of cases, and by the time someone notices the error in one, the other tens of millions have already landed. This is the trap this volume must answer.
Five. The Way to Open It: Each of the Three Back in Its Place
How, then, is this trap opened? Not by smashing the tool, banning it, destroying it. The tool itself bears no guilt; the first section of the next chapter will set this meaning down on its own. The way to open the trap is to put each of three back in its place.
Put the tool back in the tool's place: let it do what a tool should do, calculating, searching, assisting, putting out work, and the better it does it the better; do not let it do what a tool cannot do, and the decision that needs the five is never handed to it. Set the real human back in the real human's place: let the real humans who sit in these positions nurture the five, hold a license, bear responsibility, and make that last, deciding thought themselves. Turn those who set the tool on the person's place back to their proper place: they are the same kind as that nest of people in the last volume who made themselves into tools, reading the letter and not the intent, seeking to spare themselves effort without regard for the living person, working the procedure and shifting the responsibility, only this time hiding behind the tool. Saving them as people is humaneness, and that may be done: reforged, reading the intent, nurturing the five back. Letting them again hold the power to decide whether a tool is put in a position is not allowed, shut for good. In the humaneness there is iron. Each of the three back in its place, the tool to the tool, the real human to the real human, that nest turned back to the real human's place that reads the intent, and this trap is opened at the root.
See clearly that these three are one body. You cannot do only one side. Put the tool back in the tool's place without setting the real human back and without turning that nest around, and the real human's place stays empty, and that nest will look for some other way to harm people. Set the real human back and turn that nest around without putting the tool back in the tool's place, and the tool will be pushed onto the person's place again and again. All three answered together, none can be missing. This is exactly the method the last volume laid down all along the road: do not strike people down, only undo the lock; do not smash the thing, only put each back in its place. On the matter of the tool, it is still this same method, not half a step away.
Six. What This Chapter Has Made Plain and What the Next Will Lay Down
The three sections of this chapter have set the starting point of this volume firmly in place. The first section made plain: all this unlocking along the road was answering the trap of the tool; artificial intelligence, however strong, is a tool, and cannot replace a real human. The second section made plain: the real human's share is the five, Keep, Guard, Love, Mercy, Heart, which the tool will never hold. The third section made plain: the trap this volume must answer is the tool set on the person's place, and the way to open it is to put each of three back in its place.
Taken together, these three sections are one whole starting point: see the tool clearly, see the real human clearly, see the trap clearly, see clearly the direction for opening it. Yet once the starting point stands firm, the most pressing question rises at once. If it is the tool that is harming people, why not do away with the tool? Do away with the tool, and would the trap not open by itself? This question bears on the direction of the whole volume, bears on the root divide between saving people and smashing things. The first section of the next chapter answers it straight on: the guilt lies with the people who set the tool on the person's place, not with the tool; the tool is innocent, the tool is harmed too. The root of saving people in this volume will, for the first time, be carried out to the tool, this thing that never had the five, and lay down a principle that is unexpected, yet solid enough to stand.
Credit to the ancestors; the mistakes are mine: Tiger Lyon.