The New Hampshire Attorney General’s Office is investigating a robocall using an AI-generated clone of President Joe Biden telling people not to vote in the primary.
The source of a high-tech new form of election interference, a "deepfaked" robocall spoofing the voice of President Joe Biden that was sent to New Hampshire residents ahead of the state's primary election last month, has been identified, prosecutors said Tuesday. The people involved could face criminal charges.
Two Texas-based companies are among those involved in the calls, which were "a real-life example of an attempt to use AI to interfere with an election," New Hampshire Attorney General John Formella said at a news conference at the Department of Justice headquarters in Concord.
"We had not seen as example as concrete as this," he added.
The two companies have received cease-and-desist letters from New Hampshire prosecutors and the FCC, Formella said, and his office has opened a criminal investigation. The calls were traced back through the telecommunications system through collaboration with federal authorities and other attorneys general.
The source of the call was Texas-based Life Corporation and a man named Walter Monk, prosecutors said. NBC10 Boston and NBC News have reached out to them for comment.
The calls were sent through Lingo Telecom, a Texas-based voice service provider, which suspended Life Corporation after learning the calls were under investigation, according to prosecutors, whose investigation into what happened is continuing.
"We are committed to keeping our elections free, fair and secure. We will not tolerate any action that seeks to undermine our elections or our democratic process," Formella said.
Boston University professor Joan Donovan, a renowned expert in online disinformation and misinformation, said this is not the first time these companies have gotten into trouble.
“Lingo Telecom has gone by numerous names,” Donovan said. “And according to the FCC, Walter Monk and Life Corp. have also been investigated for delivering advertisements over the phone in 2003, which can be illegal if you are on a Do Not Call list.”
The "deepfake," as such AI-generated fake media are called, has prompted new warnings about the effects of disinformation on elections. The calls were sent the Sunday before the election, apparently mostly to registered Democrats, Formella said, falsely informing them that voters couldn't participate in both the primary and general election.
Formella previously said the calls appeared illegal, and said Tuesday that they are believed to have violated various state and federal laws. He didn't want to speak for federal prosecutors on whether they were expected to bring charges.
But he said that his announcement was meant to be a deterrent for others who may be thinking of producing deepfakes meant to sway voters, and that they "think if this happens again, we can track the calls even faster and identify the calls even faster."
In the run-up to the primary, Formella warned residents to ignore any such phone calls.
"These messages appear to be an unlawful attempt to disrupt the New Hampshire Presidential Primary Election and to suppress New Hampshire voters," a release from Formella's office said. "New Hampshire voters should disregard the content of this message entirely."
They added that, despite the message saying, "your vote makes a difference in November, not this Tuesday," voters are able to vote in both the primary and general elections.
Republican Gov. Chris Sununu called the robocall an illegal act of voter suppression.
Anyone who received the message was asked to email the Department of Justice Election Law Unit at electionlaw@doj.nh.gov sharing when they received the call or message, where it came from, what it said and any other information that may be relevant.
At the time, NBC10 Boston asked New Hampshire Secretary of State David Scanlan, who administers elections in the Granite State, about the message, and he said he was aware both about the messages and the investigation into them.
The presence of the apparent deepfake — media created through machine learning that convincingly imitates a real person — "reinforces kind of a national concern that has been out there about the effect of artificial intelligence and how it might impact elections and campaigns in negative ways," Scanlan said.
He also noted that, as deepfake technology is brand new, part of the response to it will have to be voters doing diligence on what they see and hear.
"This is a new area in elections and so we're going to have to study it and figure it out, but there needs to be a quick response to misinformation and disinformation that's out there. And the voters have to take some responsibility, too, in understanding the facts and not believing everything they see, because something like this fake news can come by and just throw people off," Scanlan said.
The fake message said, falsely, that "voting this Tuesday only enables the republicans in their quest to elect Donald Trump again. Your vote makes a difference in November, not this Tuesday." It also gave the phone number of a former New Hampshire Democratic party chairwoman who was running a super PAC for the campaign to write in Biden's name on the New Hampshire primary ballot — his name didn't appear on it because the state didn't move New Hampshire's traditionally first-in-the-nation primary back to February and let South Carolina go first, as the party voted to do.
Donovan noted that there are currently no federal regulations around deepfakes generated by artificial intelligence, though the technology used to create them is abundantly available.
“Unfortunately, our laws often lag very far behind technology,” Donovan said. “Right now, there’s at least a dozen companies where you can create a deep fake and it doesn’t seem to be in the cross hairs of legislators just yet.”
The media manipulation expert is concerned tools like those will be used in the context of political campaigns to engage voters and essentially convince them not to vote.
“It’s just as successful to convince someone to vote for your candidate as it is to convince them that voting isn’t worth your time.” Donovan said. “My fear is that these kind of technologies will be used to further cause turmoil around our voting system.”
Biden campaign manager Julie Chavez Rodriguez previously told NBC News her team was discussing any additional actions that it might take beyond the state investigation, adding in a statement, "Spreading disinformation to suppress voting and deliberately undermine free and fair elections will not stand, and fighting back against any attempt to undermine our democracy will continue to be a top priority for this campaign."
NBC10 Boston's Matt Prichard contributed to this report.
