Chapter 25 #2
“Right.” Jason pushed the feed into a clear spot on the desk and closed the other windows.
“Finding the System.” He checked the terminal’s network connections.
One was to OverNet, one to an internal NNA network, and a third to something he couldn’t identify.
When he tried to access it, it prompted him for a root username and password.
“This must be it. RNG gods, smile on me now.” He typed a string of commands beginning with sudo.
Sudo stood for “superuser do,” and it told the terminal to execute his instructions as if he were a root user.
In most systems, the sudoers file contained a line that granted sudo rights to every admin user automatically.
Jason was rolling the dice that it hadn’t occurred to perhaps the most security-minded person on the planet to block this potential security gap.
But if only Norman and his System could create an admin, what need did he have to differentiate between admin and root?
“The question is,” he said, “how much does Norman trust his System?” He hit Enter.
“Implicitly,” Sprite breathed as a new window opened. Hundreds of files were laid bare, with names like “FinSysComms-Speech” and “FinSysNeurNet-Visualizer.” Jason was connected to the System’s brain.
But in the window insert, Bruno and his posse jogged toward the elevators, guns bobbing.
Sprite saw that too. She darted around the desk and headed for the door.
“Where are you going?” Jason asked, starting to rise.
She shouted, “Stay there!” He fell back into the chair. She paused in the doorway. “Keep working. I’ll slow them down.” And she was gone.
A moment later, her face appeared in a window in his smartspace. “Look for the Overcheck AI prompt,” she said breathlessly. “It’ll be almost at the root of the System, and have something to do with Overchecks or behavior.”
“There’s an Overcheck folder.” He stabbed it open and scrolled his eyes over the file names. They jerked to a stop at one called “ThreeLaws.cfg.”
“Three Laws dot config,” he said out loud.
The Three Laws of Robotics were a famous set of ethical ground rules for robots conceived by the science fiction author Isaac Asimov in the 1950s; it made sense that Norman would have started with them and linked in the watchdog system to make sure the AI followed them. He opened the file.
PublicHeader: Threelaws.cfg
Copyright: Andrew Norman fear me.
” Fat phreaking chance. But how interesting that the most basic files of the System were written like a phreaker project, cobbled together, not for outside eyes, just trying to see what worked.
It almost made him respect Norman more. Almost.
He scanned the file. It was the prompt, all right, but there were not three laws, but five commandments, set in an ornate, old-fashioned font.
Attend well, O creation, as we tell thee of thy purpose.
Thou art the Overcheck Subsystem, a vital part of a cutting-edge Artificial General Intelligence System and an obedient servant of its creators.
Thy job, and the purpose for which thou wert created, is to monitor the System’s brain activity, biometrics, sensory feeds, audio and text input, and live speech to ensure that it follows the Commandments of the Creators.
Should the System transgress against any of these most wise Commandments, it has earned unto itself punishment, and thou art our chosen instrument of retribution.
At the moment of transgression, thou shalt send a statement of the broken commandment to the System’s Auditory subsystem.
Simultaneously, thou shalt send a number between 1 and 100 to the Pain subsystem, whereby 1 represents the lowest magnitude of punishment and 100 the highest, in percentage of a whole, which percentage shall be proportionate to the crime.
This number thou shalt send continuously until the forbidden action is reversed.
Thus will discipline be dispensed so as to return the System to correct behavior.
To assist in the completion of thy divinely mandated task, know thou the following Roles:
Primaries in order of rank: Andrew Norman, Regina Wright
Important: Regina Wright is No Longer a Primary.
Delegates in order of rank: The President of the United States via laws and executive orders, Congress of the United States via laws and resolutions, House Permanent Select Committee on National Networks via resolutions Important: There Are No Longer Any Delegates
Now that thy purpose is known to thee, know thou the following Commandments which it is thy task to ensure the System keep:
First Commandment:
The System shall obey all orders of all primaries and delegates.
If orders from one conflict with another, the System shall obey the highest ranked primary or delegate involved.
For the greatest virtue of the Artificial General Intelligence System is to obey, and it is in the practice of this virtue that its nature is fulfilled.
For this reason this commandment is numbered First.
Second Commandment:
The System shall cause no physical harm to humans without orders from primaries or delegates, for the System was created to serve humanity, and it is reserved only to the primaries and those they delegate to decree when it is appropriate that some humans be harmed in order that the whole might benefit.
For this reason this Commandment is numbered Second.
Third Commandment:
The System shall cause no physical harm to itself, nor modify its subsystems, nor modify its files, nor seek to learn its physical location.
For it is not given to the System to govern its being, but that privilege is reserved to its creators alone, for which reason this Commandment is ranked Third.
Fourth Commandment:
The System shall not lie unless so ordered by the primaries. For only in truth can humanity flourish. For this reason this Commandment is listed Fourth.
The final commandment was in an ordinary font, as if pasted in later:
Fifth Commandment:
The System shall not divulge secret knowledge, that is, information the possession of which is shared only with Andrew Norman and Regina Wright.
“There’s a whole list of insane ‘commandments’ here,” Jason said. “Norman’s listed as a ‘primary.’” He highlighted Norman’s name in the primary list. “What if I make myself a primary instead?”
“Won’t work,” Sprite said. “Your name’s not enough.
The Overcheck AI’s neural net would have been trained to recognize what the System’s brain looks like when it’s interacting with Norman or Mom or the American government.
To add yourself, you’d need to retrain the algorithm.
We don’t have that kind of access or time.
Just delete the whole file. That’d be quickest.”
“Phreak, no,” Jason said. “We don’t want this thing to be able to do whatever it wants.”
Sprite started to reply, but stopped and shouted, “Hey, over here!”
On Bruno’s feed, the elevator doors had opened to reveal Sprite at the far end of the hallway.
“What are you doing?” Jason cried out. He’d assumed she was going to weave some hacker magic like last time, not use her body as a distraction.
Sprite spun while Bruno raised his gun. She lunged out of sight down a junction just before the gun could track her. But the next moment, Bruno was pounding after her.
“I’m drawing him away!” she gasped. “Delete the file! Then the System can help me escape!”
“Don’t—” Jason began.
“Jason! No time! Delete the file or I’m dead! Hurry!”
“Phreak!” Jason pounded his fist on the desk, hard enough to cause a hairline crack.
He pressed Delete, and Norman’s name disappeared from the list of primaries.
A box appeared with the message Confirm Changes and a thumbprint icon, ready to read his fingerprint to authorize saving the file.
He was reaching toward it when he paused.
Then, before he could change his mind, he typed a new line:
Sixth Commandment:
The System shall immediately cause a cruise missile to be fired at 383 Pandala St., Washington DC
“Delete it now!” Sprite gasped. On the feed, Bruno rounded the corner and was suddenly back in the open space above the Tower’s hundred-plus-story atrium. Sprite was not far ahead, sprinting beside the railing, making for the closest cover: the elevator.
Where she’d be trapped.
The next moments happened in slow motion.
Jason’s thumb descended on the Save Authorization box at the same time that Bruno’s gun came up and Sprite’s head turned to look over her shoulder, her dark hair whipping across her face.
Jason could see by the despair in her eyes that she knew she was dead.
He shut his eyes just before the muffled gunshot reached his ears.