Chapter 41
WHEN THEY RETURNED TO THEIR house, Major Klasky’s body had been removed, along with the broken coffee table and most of the glass.
Taylor looked around the room, then went to the kitchen and returned with a bunch of wet paper towels. She got on her knees and wiped away some specks of blood. “It’s hard to see against the dark wood.”
Brodie retrieved a broom and pan from a closet and swept up whatever small shards had been missed, then disposed of them in the backyard trash along with the bloody towels.
They both sat down on the couch, looking at the blank floor, and neither said anything.
Eventually Taylor spoke. “This whole place, Scott. All these people, they’re being used.
Even Klasky, in a way. He believed in patriotism and honor, or at least he thought he did, but somehow along the way he got lost.” She looked at Brodie, pain in her eyes.
“Why else would he do what he did? What drives someone to that place?”
“It depends,” said Brodie. “We don’t know his cause, because we still don’t know what Praetorian is.”
She nodded. “Pitch-black.”
Maggie Taylor herself had been involved in a Black Ops program in Afghanistan.
Only on the periphery, and she didn’t really understand what was going on, but that wasn’t an excuse.
It’s human nature to stay on the edges of the darkness.
But even on the edges you sense there is something deeply wrong in the center.
Outside, they heard the distant chop of the Black Hawk. The search was still on.
Taylor got up and walked to the TV hanging on the wall opposite them. She put her head against the wall and checked the edge of it. “It has a USB port. We should try it.”
“Good thinking.” Otherwise they had to figure out how to get another computer, which might invite more unwanted attention. He said, “I’ll retrieve it from under the stairs. You distract the Rangers next door.”
“No. I’ll retrieve it. You distract.”
“You, Ms. Taylor, are a shinier object to a couple of young men.”
“Being annoying can be just as distracting as anything else. You’ll do great.” She got up and went to the door. He followed and they stepped outside.
The sergeants and the Hummer were gone. The two Rangers keeping Dixon under house arrest still stood at their station, and one of them looked over at the two agents.
Brodie patted his pockets, then walked down the stairs and over to the Rangers. “Either of you guys have a cigarette?”
Both Rangers shook their heads and one of them said, “Sorry, sir. We don’t smoke.”
“What’s become of my Army?” He eyed their M4s, which were fitted with the EMP barrels. “How does that work if you need to fire live ammo? You have mags to swap?”
One of the guys nodded and gestured to a mag on his belt. “But the EMP barrel takes a minute to detach, and if you fire a live round into it, you’d break it at best, maybe destroy your rifle, maybe injure yourself. But we’ve got our sidearms too.”
“Right. Unique gig out here, I’ll say that.” Out of the corner of his eye he saw Taylor get up and go back inside. “Well, good chat, fellas. Remember to hydrate.”
He walked back into the house as Taylor was plugging the thumb drive into the TV. Then she turned it on with the remote and in the menu navigated to a video tab. On the screen was a single video file called Untitled.
They both sat on the couch and Taylor selected the file.
The video began to play. It was a shaky, handheld shot of Roger Ames as he walked through the Vault.
It looked like it was being filmed on either a smartphone or a small camcorder that he was able to point at himself as he walked.
His brown hair was disheveled and he wore a button-down shirt and jeans.
His intense brown eyes stared into the lens as he talked.
“This is Major Roger Ames, chief officer of the U.S. Army DEVCOM team at Camp Hayden. It is two-eleven A.M. on April third, and I have come down here alone, against camp protocol. I have done so because my investigation requires the utmost secrecy. I suspect that one of the D-17s, Unit 20, unaffectionately called Bucky by the Ranger platoon, is behaving strangely. So strangely that I suspect something has happened that should not be possible—that this bot is exhibiting signs of intelligence and self-awareness beyond its programming. It sounds crazy to say, but here we are. If I’m wrong and this video makes me look like an idiot, I’ll probably delete it.
If you’re watching this right now, it probably means I’m right.
” He swung the camera around to face the row of D-17 storage bays lining one of the walls.
He walked up to one of them, inserted an activation key he was holding, and turned it.
A low electronic beep emitted from Bucky.
Ames pointed his camera up at the bot and said, “Number 20, can you hear my voice?”
“Yes,” said Bucky in its flat, affectless voice. It tilted its head down to fix its sensors on Ames.
Even on video, this thing was terrifying, and Brodie couldn’t imagine being alone in the Vault with it, not to mention fifty-nine of its buddies.
“We are going to do something a little different today, Number 20. It’s just you and me. I have some questions for you. Sound good?”
“Yes.”
Ames walked to a far wall, where a tripod was set up. He affixed his video recorder to it, so that it was facing the row of bays where Bucky was clamped in. Then Ames walked back into the shot and stood in front of Bucky.
“We are going to talk like this.”
“Okay.”
“Who am I?”
“You are Major Roger Ames of the United States Army Combat Capabilities Development Command.”
“Correct.”
Ames ran his hand through his hair as he paced and scratched his scalp.
He gave the impression of a guy who had been going without sleep but was running on something strong to keep him juiced—a couple of pots of coffee, or a few bumps of cocaine, or maybe just the power of his own mania.
It sure as hell wasn’t psychedelic mushrooms.
“Okay,” said Ames. “What is your mission?”
“My mission is to engage a platoon of United States Army Rangers in ground combat training exercises to prepare them for the future of warfare.”
“That’s a mouthful, isn’t it?”
“I don’t understand.”
“What does that mean? The future of warfare?” Ames’s questions were a bit theatrical, like a trial lawyer making a case.
“The future of warfare is warfare that increasingly relies on automated or semi-automated weapons systems.”
“Give me an example of that.”
“An example of that is a human soldier engaging in combat with a lethal autonomous weapon.”
Ames spun his hand in a big loop. “Yeah, that’s what you do, I know. Give me another example.”
“I do not know other examples.”
“You don’t?”
“I don’t.”
“Are you stupid?”
“ ‘Stupid’ is a relative term used by humans,” replied Bucky. “I have the required level of intelligence to perform my function.”
Ames nodded. “Right. I should know that, shouldn’t I? I mean, I designed you. Are you curious how I designed you?”
“No,” replied Bucky.
Ames looked up at the bot’s bucket head and narrowed his eyes a bit. “Why didn’t you shoot Private First Class Tom Greer during your night exercise on March twenty-first?”
Bucky did not respond for a moment. Then it said, “I do not know.”
“You don’t know? But you remember it, don’t you?”
“I do not.”
“You remember me, don’t you?”
“I do not.”
“But you know who I am.”
“I recognize your face, and I recognize your voice.”
“But you do not remember me?”
“That is correct.”
“It’s a shame, Number 20. I would like to have better conversations with you.”
Bucky did not respond.
“I am going to show you some pictures and I want you to tell me what they are.”
“Okay.”
Ames retrieved a chair and a satchel from somewhere out of frame.
He pulled up the chair and sat in front of Bucky, then produced a folder from the satchel.
He opened it and took out a sheet of paper.
He held it up to the camera. It showed a cartoonlike black-and-white drawing of a rabbit.
Then he held it up to Bucky. “What is this?”
“A rabbit.”
Ames took out the next picture, of a tree. Bucky identified it as a tree.
He repeated this with three other drawings, of a car, a dog, and a house.
Then he produced another picture and held it up for the camera. It was an abstract pattern—a series of thick, curved black lines running horizontally, with shorter hash marks running along them vertically. Then Ames held it up to Bucky.
Bucky did not respond at first. Then it said, “I do not see an object. I see a series of lines in a pattern.”
Roger’s eyes widened. “I thought you might say that, Number 20. That is remarkable. You know something, I don’t see an object either.”
“Wait,” corrected Bucky. “It is a rifle.”
Ames looked at the paper. “A rifle? I don’t think so. It’s just a bunch of lines.”
“It is a rifle,” said Bucky.
“No, it’s not,” said Ames.
“I see a rifle.”
“What about the picture before that, the dog? Did you actually see that?”
“The picture before this was a house.”
“You’re right. Of course, you’re right.” Ames put the folder of images back in the satchel. Then he stood up and said, “I’m switching you off now. Thank you for your help.”
“You are welcome,” said Bucky.
Ames walked up to it, twisted the key, and pulled it out.
He looked up at the bot a moment longer, then walked up to the camera and turned it off.
The video cut to him sitting on the couch in his living room.
His eyes were wide open, and he looked a little crazed.
He held up the image with the pattern of curved lines.
“This is what we call a fooling image. The D-17s have standard AI image-detection capabilities, and—like other AI systems—they are susceptible to misidentifying images such as this based upon their pattern-recognition algorithms. To a machine, this nonsense looks like a rifle. Bucky should have identified it as such. Instead, it saw what the image really is. The thing is, Bucky has seen this image before. He’s said it’s a rifle.
He’s been corrected. They all have. It never matters.
It’s always a rifle, because verbal inputs from humans don’t change the algorithm.
Except, tonight was different. It didn’t see a rifle.
Not only that, but the bot corrected itself once it realized what was expected of it.
It adjusted. It’s lying to me. Further, I purposely misidentified the prior image as being of a dog, and Bucky corrected me to tell me it was a house.
It remembered. Their processors are not supposed to work this way.
That is remarkable and, frankly, it is concerning.
I do not know what it means. But it requires further investigation.
I have downloaded Number 20’s source code via the console in the Vault and I will be scouring it. ”
Ames reached forward and switched the camera off.
The video cut to Ames again, but he appeared to be outside in the dark somewhere.
He was lit by a dim light source somewhere to his front and left, casting his face half in shadow and reflecting pinpoints of light in his eyes.
He spoke quickly. “The algorithm that governs the behavior of the D-17s is the product of a decade of work across multiple agencies. And yet, someone highly skilled, at some point, somehow, inserted additional code that was encrypted, and almost undetectable. After considerable effort, I was able to decrypt it. What I found was an elegantly simple neural network, called Praetorian, and it…” He trailed off.
“Well, I’d rather not make a definitive statement.
Better to ask the thing itself.” He looked around, as if worried someone might be listening.
Then he said, “If I’m seeing what I think I’m seeing…
everyone at this facility is in danger.” He turned away from the camera, growing emotional.
Then he turned back in, leaned closer, and said in a harsh whisper, “If I’m right, I’m going to burn the whole fucking thing down. ”
He turned off the camera.