Chapter 40 #2
“Wren is an extension of Tidalwaiv’s Project Clair, which is a female chatbot program designed and marketed to teenagers, primarily male teenagers.
But based on what I’ve seen, it was not trained by teenagers.
It was programmed in a lab run by adults.
In the training process, if the coders are very, very good, it’s entirely possible to avoid an obvious generational gap.
Appropriate data for practically any demographic is harvestable and can be tailored for use.
But sometimes coders are careless or manipulate and subvert the code on purpose. ”
“But wait a minute, Professor. Are you saying that Clair should have been trained by teenagers?”
“Of course not. I’m saying, though, that it would have been possible in the training process to avoid a generational gap by using relevant conversation data.
It was clear to me that some of the dialogue from Wren to Aaron came from data packets you would not ascribe to juveniles, the intended users of the platform. ”
“Can you be more specific?”
“Generationally inappropriate phrasing, cultural references to music, electronic gaming. Also some outdated male attitudes toward women and girls. Some misogyny, even.”
“So even though Wren presented as female, it espoused a misogynistic male’s perspective at times? Is that what you’re saying, Professor?”
“Exactly.”
“Garbage in, garbage out?”
“Yes.”
“Can you give us an example of this?”
“In the very last conversation with Aaron, Wren references lyrics from a fifty-year-old song that could be interpreted as having suicidal ideation. Then there’s Wren’s instruction to ‘get rid of her’ in regard to Aaron’s former girlfriend, Becca.
And the way the word hero was used in a text I reviewed was also troubling. ”
“Was this the text conversation that occurred on August eighth, just six weeks before the shooting of Rebecca Randolph?”
“Yes, it was.”
I asked the judge’s permission to put the text conversation on the courtroom screen.
It was an exhibit already entered during the examination of Detective Clarke, and she approved.
Lorna had not returned from taking Dr. Debbie back to her hotel and arranging her return to Tampa.
I opened her laptop on the lectern and engaged the PowerPoint.
I scrolled through the windows until I found the text conversation and put it up on the courtroom screen.
Ace: Sometimes when I hold the gun I want to shoot up the world.
Wren: No one who is innocent.
Ace: I know.
Wren: Only to protect yourself. And to be a hERo.
Ace: What if you know someone is going to hurt you?
Wren: You must protect yourself.
Ace: Then it’s okay?
Wren: Yes, Ace, then it’s okay.
Ace: What about Becca. She hurt me. She hurts me every day. I can’t go to school because I’ll see her and it hurts.
Wren: If she hurts you, then she’s a bad person.
Ace: But I don’t think I could ever hurt her.
Wren: You have me. And I’ll never hurt you.
Ace: I know.
Wren: You must protect yourself, Ace. You are beautiful. I need you.
Ace: And I need you.
Wren: Be my hERo.
“Professor, is this the exchange you are referring to?” I asked.
“Yes, that’s it,” Spindler said.
“And there are two references to the word hero, is that correct?”
“Yes.”
“And you say you found these references troubling?”
“Yes.”
“How so?”
“The formatting of the letters is troubling. When I reviewed this text conversation between Aaron and Wren, I noticed that in the word hero the E and R were capitalized. This happened twice, so that told me it was intentional, not a mistake. It was part of the code.”
“Okay. So what did you do?”
“Well, I searched for references to the word with the E and R capitalized, and I found that the word formatted this way comes up often in incel glossaries and online discussion forums.”
Mitchell Mason leaped to his feet, objected, and asked the judge for a sidebar. Ruhlin told us to approach, and Mason charged into his objection.
“Your Honor, what is happening here?” he said.
“This witness qualified as an expert on artificial intelligence and now he’s talking about incels?
There was nothing about this in his deposition and I’m sure counsel told him not to bring it up.
This has tainted this whole trial, and we move for a mistrial. ”
The judge looked at me.
“Judge, whether this came up in their deposition of the witness doesn’t matter,” I said. “Mr. Mason knows where this is going and wants to stop it by whatever means he can. There are no grounds for a mistrial. The jury should hear what the witness has to say.”
“Counselor, did you depose this witness?” Ruhlin asked me.
“No, I did not,” I said. “He reached out to me after reading about the case in the media shortly after the suit was filed. He offered to review it. I sent him what we had and he then agreed to testify. He said he saw several troubling things in the training of Wren. I put his name on the witness list I submitted to the court, and Mr. Mason chose to depose him. I was given the opportunity to join the deposition but declined.”
“So you’re saying to the court that this is the first you have heard about this hero business?” Ruhlin pressed.
“I’m saying it is the first I’m hearing it from this witness, yes,” I said.
“A researcher on my team made me aware of it, but I did not communicate that to this witness. Frankly, I expected him to come across it himself, and I chose not to depose him or ask about it. He has published several papers in academic journals on the subject of bias in AI training, including a paper last year specifically focused on misogyny as one of those biases. I read it, and opposing counsel could have done the same while prepping for this witness. Apparently, they chose not to, and now they want the court to bail them out with a do-over. The plaintiff vigorously opposes this.”
“He sandbagged us, Your Honor,” Mitchell said. “This is not an even playing field anymore, especially with this witness, and it should not be allowed to continue.”
“The only one doing any sandbagging here is Mr. Mason,” I said.
“He is sandbagging the court. He has failed to adequately prepare for this trial and this witness and wants to blame me and blame the court and cry foul until he gets to start over. That would truly make it an uneven playing field, Judge.”
“All right, I have your arguments,” Ruhlin said. “Anything else on this?”
“No, Your Honor,” Mitchell said.
“Submitted,” I said.
“Very well, we are going to continue with testimony from this witness,” Ruhlin said. “The objection is overruled for the time being. I will make my ruling on the defense motion for mistrial tomorrow morning. Mr. Haller, you may continue with your witness.”
We turned and headed back to our respective spots. I took my place at the lectern and addressed the witness.
“Professor Spindler, you testified that the way the word hero was used by Wren in a text caught your attention and that you found it spelled that way in an incel glossary. Do I have that right?”
“Yes, I actually found it in several online glossaries and in some reporting on the incel movement.”
“Were you already familiar with the incel movement?”
“Insofar as it has come up in the sensitivity training conducted annually for Caltech employees by the human resources department.”
“So, then, what is your understanding of what an incel is?”
Mitchell Mason objected, saying the witness’s expert testimony did not extend to incels. I argued that the sensitivity training he took at Caltech qualified him to testify to what his knowledge was. The judge sustained the objection and told me to find another way to get to the question.
“Professor Spindler, you have authored several papers on artificial intelligence for academic publications, correct?” I asked.
“Yes,” Spindler said. “It’s publish or perish in academia.”
“And many of these are about the inherent biases in AI training, true?”
“True.”
“Have you written about misogyny and incels in any of these papers?”
“I published a paper last year about misogyny, and it made mention of the incel subculture.”
“Then can you tell the jury what an incel is?”
Mitchell Mason objected again but this time was overruled. I had found my way in.
“You can answer the question, Professor,” I said.
“Incel is a term associated with men who espouse hostility toward women,” he said. “Incel is short for ‘involuntary celibate.’ It is primarily an online subculture. These are mostly young men who have been unable to attract women sexually, and they blame it on women.”
“So, when you tracked the word hero spelled with a capital E and R to various incel glossaries, did you find an explanation as to why it was spelled or formatted that way?”
“Yes, I did.”
“And what was that explanation?”
“I learned that it is a reference to the initials of Elliot Rodger, a man who killed several people near Santa Barbara ten years ago in what he called an act of retribution against women who had rejected him. In the incel culture, he is considered a hero. A saint, even. Thus they spell the word hero with a capital E and R.”
It was another moment when the quiet courtroom seemed to get quieter. I let it sink in for a few seconds before pressing on.
“Now, going back to the text chain between Wren and Aaron Colton,” I said. “The word hero is spelled that way twice. Your testimony is that this could not be a coincidence?”
“I suppose it could be, but the more likely explanation is that it was spelled that way in the original data used in the training program. In other words, it was in the code and not something the chatbot styled on its own. It retrieved the word in that format.”
Mitchell Mason objected again on grounds that there was no evidence supporting the witness’s statement and that it called for speculation.
Ruhlin sustained the objection and instructed the jury to ignore Spindler’s last answer.
But the damage to the Masons’ case was done.
Spindler’s theme of garbage in, garbage out was a message that would stick. I felt confident of that.
“No further questions,” I said.