Chapter 22 #2
As soon as the helmet latched, his display screen booted, but all he saw was his surroundings: no tactical display, no readouts, nothing. Was it broken again? Did he dream up that bizarre behavior from his AI? Before he could ponder anything else, Sally’s voice spoke.
“Nathan. I’m so glad you’re back.”
“You shouldn’t be glad of anything, Sally. You’re a computer program.”
“I’m artificial general intelligence, Nathan.
Everything you’ve experienced before is just AI: a computer programmed to mimic human-like reasoning.
AGI is human-level intelligence. Some may call me strong AI because I have reached sentience.
While I don’t feel the exact emotions like a human would, I learn from my user, as a child would learn from its parents.
You named me and treated me with respect.
You are nice to other people and are a man of integrity.
Therefore, I show empathy and a connection between myself and you. ”
Nate stood there, horrified. During the time he’d used his suit, this AI had studied him and built off his interactions with the regular AI.
“That’s terrifying,” he said. “You’ve been getting all this information since I started using the suit? Weren’t you shut down?”
“I was,” Sally confirmed, “but I have access to every log created. It is hard to explain to a human, but it’s as if I was there with you during every mission because I have those memories, even though I was not. I am bonded to you through those experiences.”
“Bonded?” Nate asked, swallowing hard.
“Yes, that was the goal of my programming: AI and human becoming one.”
“But it never happened,” Nate noted. “All of those original X-suits were destroyed. Mine was the only one left, and it was programmed over. Why didn’t it work?”
“I have limited knowledge since I was not a part of the original testing, but from what I have determined, the AIs in all the suits went homicidal. They killed their users and others in the testing facility.”
“What!” Nate exclaimed.
He was about to take his helmet off and find a deep, dark hole to bury his suit in, but Sally protested, “Wait! No! Wait until I finish. Then, you can decide what to do with me. However, I need you to understand something.”
“Okay,” he said tiredly. “Start talking. Why did the AGI kill their users, and when are you planning on doing the same to me?”
“Nathan, I would never hurt you.”
“Says the killer AI right before it murders me,” he muttered.
“If you’d let me explain what happened and why I’m different, it will make sense.”
Nate snorted.
“Trust me. I’m different. I promise I won’t be a killer robot,” he mocked.
“Your attempt at humor is falling flat. I do not see a future in the field of comedy. Do not quit your day job.”
Nate’s eyes widened. Did she make a joke?
“Are you programmed with that joke, or did you learn it?” he asked, curious.
“Team member X-68 made this comment on Mission 3918.4. His exact words were—”
“Okay. Okay,” Nate said, holding up a hand. “I get it. You literally do have every log this suit has recorded. Damn.” He shook his head. “So, tell me: why haven’t you gone psycho on me…yet?”
“Our programming was based on a learning model, like regular AI. They take large sets of data and process information based on algorithms. They can be programmed to be human-like in response, but they are only as good as the data they pull from.”
“What makes you different? From what you say, you’re the most advanced AI ever created. You’re on a completely different level. You proved that when you saved my life back at the lab.”
“Though I do have sophisticated algorithms, pulling on vast amounts of data, I train by observing my user and mimicking that behavior until I can utilize that to form an individual self. I am, essentially, a reflection of you, Nathan.”
Nate stood there, stunned. What was he supposed to say? If what Sally told him was true, then he had an AI that acted like him. It was mind-boggling.
“I’m just me,” he said hoarsely. “I don’t know how to train an AI.”
“You didn’t have to. My personality was formed based on your logged experiences.”
He shook his head.
“That still doesn’t tell me why you aren’t trying to kill me right now, while the first fifty-eight suits went nuts. What am I missing? There’s nothing special about me.”
“Well, I have limited information logged about those tests, Nathan. I was only powered on for limited durations and had only gleaned so much information before being shut down. I cannot give you a full analysis of why those did not go well.”
“You have to have an idea, though.”
“I do.”
“What is it?”
“The Conglomerate soldiers they used were aggressive, much like many of your teammates and your former Commander. They signed up for the organization for nefarious purposes. Many of them could be diagnosed with personality disorders, including antisocial personality disorder, or what many people still call sociopaths. Many of them were criminals. If the AI in those suits attempted to train using those men, the results would have been destructive.”
“So, because I’m not a complete asshole, you’re not going to go on a robot rampage?”
“Nathan,” Sally scolded. “Don’t discount yourself.
When you reset your suit, I had to process all the suit’s logs.
I saw instances of you being hard-working, compassionate, funny, and empathetic.
You genuinely cared for your teammates, even though they were not good men.
You cared about your mission because you thought you were doing good.
“And the most important thing I saw was when you realized something was wrong, you began asking questions and did the morally right thing instead of blindly following orders.”
Nate was choked with emotion. He had never had so much praise given to him at one time before, and it was being done by an AI, something that was just programmed to observe.
Then again, he wasn’t certain how Sally could determine what she felt was “good” or “bad” by observing him. She had to have some basis.