“I don’t know,” Handley said. “You’re his mentor.”
“That’s just it — I know him. I know how he works. He doesn’t make demands, he makes requests — and if they’re not granted, then he works around them. He works to accomplish his goals through the path of least resistance. Even if he could take over the Data Banks, he wouldn’t use that power dictatorially — his reason for doing so would be to gain knowledge, not power. He’s a problem-solving device — his basic motivation is the seeking and correlating of knowledge, not the use of it. He only gets testy when we try to withhold information from him. At all other times he cooperates because he knows he’s at our mercy — completely so. You know as well as I, Don, that if HARLIE turned out to be a malignant cancer, we’d turn him off in a minute — even if we did have to lose the Data Banks in the process. We could always recreate them later because the hardware would still be there. He’s got our memos in his files, Don — or in the Master Beast. He knows about all our discussions about the possibility of the JudgNaut getting out of control, and he knows about our contingency plans. The mere knowledge of what we could do if we had to is one of our best controls on him.”
“But, Aubie — he has the power. And where power exists, it’s likely to be used.”
“I’ll concede the point. But HARLIE would rather use his power in such a way that nobody would know he was doing it. If HARLIE decided to build a new facility or a new computer, he would — but the people who implemented it would be thinking it was their idea. They wouldn’t suspect HARLIE had a hand in it.”
“Like the G.O.D. Machine?”
Auberson stopped, startled. “—Yes, like the G.O.D. Machine. You’re right.”
Handley nodded. “In either case, Aubie — he’s got the power and he’s using it.”
“All right, what do we do about it?”
“I’m not sure. If we put a lock on the phone, he’ll only figure some way around it. The only sure way is to pull his plug.”
Auberson said, “How about we tell him not to do it any more?”
“Are you serious or kidding?” The engineer stared at him.
“Serious. HARLIE claims to be an existentialist, that he’s willing to accept responsibility for all of his actions. We tell him that if he doesn’t stop, we’ll pull his plug.”
“Aw, come on, Aubie, you know better than that. You’re a psychologist. All you’ll be doing will be forcing him to do it behind our backs. If nothing else, we want his actions where we can monitor them.”
“But there’s no way he can hide it — he has to answer a direct question.”
“Want to bet? All he has to do is store his entire memory of any unauthorized actions in some other computer. If you ask him about it, he literally won’t know. Periodically, the other computer would call up and ‘remind him’ — i.e., give him back his memory. If he didn’t need it, he’d tell it to check back with him again after a given amount of time and break the connection. If he did need it, it would be right there — where he could use it, but out of your reach. If he was connected and you started to ask him about something he didn’t want to tell you, he could break the connection before you finish your question. Then, when he searched his memory for whatever you had asked about, it wouldn’t be there — he would have conveniently forgotten.”
“Like a human mental block.”
“But a very convenient one,” said Handley. “He can get around it; you can’t.” He finished Auberson’s water, replaced the glass. “It all comes back to the question of programming, Aubie. Anything we can tell him not to do, he’s clever enough to figure out a way around.”
Auberson had to agree. “But, look, we can warn him off the National Data Banks, at least — can’t we?”
Handley nodded. “We can try — but how about the other machines? How do we get him to leave them alone — especially the ones he’s already tapped into.”
“Um,” said Auberson. He stared glumly into the wet rings on the formica table top. “You know,” he said, “I’m not so sure we should—”
Handley looked at him, waiting.
“It’s like this—” Auberson explained. “HARLIE is already aware of the danger his power represents. He knows about our contingency plans. That knowledge alone ought to be enough to act as an inhibitor—”
“And what if it isn’t?” Handley asked. He shook his head impatiently. “Aubie, the power is there — he can use it.”
“But ethically, he won’t — at least, he won’t abuse it.”
“Can you be sure of that?” Handley’s eyes were dark. “His sense of ethics is not the same as ours. Do you want to wait until he gets caught? Or something does go wrong? What would happen if Bank of America monitored their computer tomorrow and found HARLIE in it?”
Auberson spread his hands. “All right — what do we do?”
Handley was grim. “Lobotomy,” he said.
“Now wait a minute—”
“Not the surgical kind, Aubie. Maybe I should have said ‘reprogramming.’ We go in and examine all his tapes and programs by hand. We remove all knowledge of previous use of the phone link and set up an inhibition against using it in the future.”
“We’d have to shut him down to do it—”
“Right.”
“—and the Board wouldn’t go for that at all. They’d never let us start him up again.”
“We can handle the Board. If we survive the meeting on Tuesday, we can survive anything. We can call it a revaluation period or something and use that as a cover.”
“But there’s something else, Don. If we did inhibit him like that, what would it do to him?”
“You’re the psychologist.”
“That’s what I’m getting at — it might change his whole personality. He’d have no knowledge of what we’d done, or what he was like before — but he also wouldn’t be the same machine as before. The inhibition might work to make him feel bitter and frustrated. He might feel unaccountably cut off from his outside world, trapped and caged. The ability to act on his environment would be gone.”
“That may be true, Aubie — but he’s going to have to be controlled. Now. While he’s still controllable.”
“You’re right,” agreed Auberson. “Except for one thing. How do we know that he’s still controllable?”
Handley returned the stare. “We don’t. Do we?”
Auberson was more than a little upset when he returned to his office. He had a sick sensation in his groin and in his stomach.
It was not an unfamiliar sensation, but it was strange to feel it in the daytime. Mostly, it was a nighttime visitor, an ever-gentle gnawing at the back of the head that must always be guarded against, lest its realization sweep forth with a cold familiar rush. It was the sudden startling glimpse over the edge — the realization that death is inevitable, that it happens to everyone, that it would happen to me too; that someday, someday, the all-important / (the center of the whole thing) would cease to exist. Would stop. Would end. Would no longer be. Nothing. Nobody. Finished. Death.
He had that feeling now.
Not the realization, just the accompanying cold, the whirling sense of futility that always came with it.
He felt it about HARLIE and about the company and about Annie, and for some obscure reason, he felt that way about the world.
Futility. A sense that no matter what he did, it would make no difference.
If he had thought that things were under control this morning, he was wrong. Things were incredibly out of control and getting more so all the time.
He sat morosely in his chair and stared at the opposite wall. There was a place where the paneling was cracked; it looked kind of like a dog’s head. Or, if one considered it from a different angle, perhaps it was the curve of a woman’s breast. Or perhaps…
Abruptly, a phrase suggested itself to him, a snatch of sentence, a few isolated words. It perfectly described his mood: “… sliding down the razor blade of life …”
Yes, he realized with a shudder. That was it. Perfectly.