Musk’s Minnesota Deepfake Fight Could Define AI’s First Amendment

A rapidly developing legal battle between Elon Musk and the state of Minnesota may soon clarify some of the First Amendment’s limits on the government’s ability to regulate artificial intelligence (AI). The dispute stems from a very real online harm—deepfake nude images—but Minnesota addressed the problem in a way that some say the First Amendment prohibits.

Last spring, Minnesota’s legislature, in a nearly unanimous vote, passed HF 1606, titled “Prohibition on Nudification Technology.” The law enables the state attorney general to seek up to $500,000 in fines from a technology company each time that company allows “a user to access, download, or use the website, application, software, program, or other service to nudify an image or video” or to “nudify an image or video on behalf of a user.” The law broadly defines “nudify” as altering or generating a video or image “to depict an intimate part not depicted in an original unaltered image or video of an identifiable individual” provided that the image or video “is so realistic that a reasonable person would believe that the intimate part belongs to the identifiable individual.”

Many states—including Minnesota—already have nonconsensual intimate imagery laws, and some of those laws already cover deepfakes. But these laws impose civil or criminal penalties on the people who created or distributed the images. What makes Minnesota’s new law different is that it places legal responsibility on the company whose technology was used to create the image or video. And that aspect raises novel First Amendment questions, such as whether a company that takes steps to prevent AI misuse should be liable and whether that company should be liable if the subject of the deepfake is the one who created the image.

Judges have some guidelines to assess the law’s constitutionality. As Mike Masnick of Techdirt noted, the Minnesota Supreme Court in 2020 upheld the constitutionality of the state’s nonconsensual intimate imagery law that applies to individual users, in part because it required a showing that the disseminator “intentionally” distributed the image, and that the distribution was done without the consent of the subject. Both of those elements—intent and consent—are absent from Minnesota’s new nudification law.

Under Minnesota’s law, an AI company is strictly liable if a customer uses its technology to “nudify” an individual. An AI company is liable even if the company has anti-deepfake nude policies and technology to enforce them. If a customer manages to circumvent the anti-deepfake safeguards, the company faces up to $500,000 in fines from the state attorney general per image or video, regardless of its attempts to prevent such use of its technology. This is a far cry from imposing liability only on those who act with intent because it could impose liability on a tech company that imposes stringent safeguards on AI misuse if just one user manages to circumvent those safeguards.

Likewise, the Minnesota nudification law, unlike the earlier law that the Minnesota Supreme Court upheld, is not limited to nonconsensual images and videos. An AI company could face a $500,000 fine even if the subject of the image is the one who used the company’s technology to create it.

The law raises other significant constitutional issues. For instance, the Supreme Court has long held that the First Amendment protects satire, but the statute has no explicit exception for satire. And the statute broadly defines “intimate parts” as including “the primary genital area, groin, inner thigh, buttocks, or breast of a human being.” Thus, if someone creates an image of a person in a bathing suit that depicts their inner thigh, the AI company could face up to half a million dollars in liability.

In late July, just days before the law was to go into effect, Musk’s xAI (now renamed SpaceXAI), which develops Grok, sued Minnesota and sought an injunction blocking the law. The company noted that it already prohibits users from generating nonconsensual deepfake nude images. “xAI accordingly does not contest Minnesota’s interest in prohibiting the dissemination of artificially generated nude images of real people without their consent,” the company wrote in its complaint. “But the statute Minnesota enacted extends far beyond that goal, exposing a wide array of protected speech to civil liability and government sanctions.” The law’s steep fines and lack of standard constitutional safeguards, the company wrote, will cause it to impose many limits on Grok. “Protected speech freely available before the law takes effect will thus be chilled.”

In its court filings, the state argued that the statute’s failure to limit its penalties to nonconsensual images and videos is not a problem. “An adult who wants a photo of herself without clothing may easily take a nude photo, just as an adult who seeks a nude photo of an adult intimate partner may simply take or request a nude photo of their partner, with consent,” the state wrote. “The end result is the same—a nude photo of a consenting subject. All that the Nudification Ban proscribes is the use of harmful technology to achieve that result.” Still, the state’s argument fails to explain why it is necessary to prevent people from using AI to create images of themselves.

The state also swiftly dismissed concerns that the penalties apply regardless of the intent of the company. “Clearly, X.AI knows how to identify and quantify illegal uses of Grok Imagine, yet it has failed to stop the flood of digital sexual victimization that its technology makes possible,” the state wrote.

On Sept. 4, the district court weighed in, denying xAI’s request for a preliminary injunction without considering the vital First Amendment issues at stake. Instead, U.S. District Judge Donovan Frank focused primarily on the fact that xAI waited nearly three months after the law was signed to file the lawsuit. “If xAI genuinely feared irreparable harm, the Court does not doubt it would have acted more quickly to file this action and move for injunctive relief,” Frank wrote. Thus, Frank reasoned he would give the First Amendment arguments “full consideration” when he eventually rules on the state’s motion to dismiss the lawsuit.

Rather than wait around for Frank to eventually consider the statute’s constitutionality, xAI directly appealed his denial to the U.S. Court of Appeals for the Eighth Circuit. The company told the Eighth Circuit that it restricted Grok’s image editing tool in Minnesota in response to the new law. “Consequently, Minnesota users cannot create First Amendment-protected speech using SpaceXAI’s industry-leading platform,” the company wrote. “That loss of First Amendment freedoms alone constitutes irreparable harm, and the public has no interest in enforcing an unconstitutional law.” The state of Minnesota urged the Eighth Circuit to reject this First Amendment challenge, arguing that the nudification statute “targets harmful technology, not ideas, beliefs, or expression.”

The case’s First Amendment implications drew attention from beyond Musk’s companies. For example, the Foundation for Individual Rights and Expression, the First Amendment Lawyers Association, and the Woodhull Freedom Foundation filed an amicus brief in support of SpaceXAI. “In pursuing its asserted objective without sufficient precision, HF 1606 regulates far too broadly, to the detriment of protected online speech,” the groups wrote.

The arguments were compelling enough for the Eighth Circuit to grant an injunction in a one-sentence order on Oct. 2, temporarily barring the enforcement of the Minnesota law against SpaceXAI while it fully considers the appeal. Although there is no way to predict with certainty how the Eighth Circuit will rule on the law’s constitutionality, this early ruling temporarily blocking the law is an indication that the appellate court takes the First Amendment concerns seriously.

The Eighth Circuit’s eventual opinion will address AI, an area with little precedential First Amendment law, so its impacts could reach far beyond nudification. In determining whether the Minnesota law violates the First Amendment, the Eighth Circuit will likely need to address a fascinating but unresolved issue: What state of mind must an AI company have to be liable for its users’ conduct?

When laws impose liability on speakers or speech intermediaries, courts examine whether those laws apply only to defendants who act with a particular state of mind, such as intent, recklessness, or knowledge. Generally, the more rigorous the state-of-mind requirement, the more likely the law is to pass constitutional muster. What courts have yet to answer is what state-of-mind requirement applies to the regulation of speech generated using artificial intelligence.

The Eighth Circuit opinion may more broadly address the extent to which the First Amendment applies to artificial intelligence. In its brief to the Eighth Circuit, Minnesota argued that the law goes after technology and not speech. “Targeting the provision of technology that makes digital sexual victimization possible is no different than regulating other harmful conduct, like the possession of chemical precursors to dangerous drugs or the distribution of pamphlets laced with anthrax.”

But artificial intelligence raises issues related to speech that simply are not present with drug manufacturing or biological warfare. While deepfake nudes undoubtedly cause substantial harm, the regulation of artificial intelligence even for this narrow purpose could impact a wide range of protected speech. The Eighth Circuit—and likely other courts—will have to determine how the First Amendment protects against chilling effects on speech created with artificial intelligence. This determination will reach far beyond the issue of nudification. In other words, the Minnesota case could help determine the extent to which governments regulate speech by regulating AI companies.