If AI Outputs Aren’t Speech, Who Has to Prove They’re Human?

A growing body of legal scholarship argues that large language model (LLM) outputs are not “speech” under the First Amendment and therefore may be regulated without the scrutiny ordinarily applied to restrictions on expression. Garcia v. Character Technologies, a wrongful-death suit brought after 14-year-old Sewell Setzer III died following months of conversations with an AI character, resulted in one of the first judicial opinions addressing the constitutional status of chatbot outputs. Ruling on the company’s motion to dismiss, U.S. District Judge Anne Conway wrote that she was “not prepared to hold that [LLM] output is speech” and allowed product liability and negligence claims to proceed. Garcia settled in January 2026, but other suits continue with similarly disturbing facts alleging that chatbot outputs played a role in medical crises, violence, or providing sexualized content to minors. Following these suits, a bevy of federal and state proposals would change whether and how people can receive information from chatbots.

Whether the no-speech position is doctrinally correct is one question. What it would take to administer that position is another. If human expression is protected but machine output is not, legal coverage may depend on whether a person created, selected, edited, or adopted—that is, knowingly put forward as one’s own—the words generated by the LLM.

Yet ordinary text rarely reveals whether it was written by a person, generated by a model, or produced through some combination of the two. If human attribution determines whether the First Amendment applies, the law needs both an attribution standard and a default when attribution cannot be established. Administering that rule would mean sorting human from machine expression at scale—a task no existing tool can reliably perform. The likeliest substitutes, identity and personhood verification, establish who a speaker is or that a speaker exists—not who authored a given text. Either approach burdens the very expression the rule purports to leave protected.

The burden of that uncertainty would likely fall more on users and readers than model developers. Enforcing a human-attribution rule across accounts, platforms, and disputed works requires some way to distinguish person from machine. Existing methods either cannot reliably distinguish between the two or demand more information about the speaker. The risks include reduced anonymity, greater surveillance, and pressure to use identity or personhood as a proxy for authorship. A consequentialist argument for excluding artificial intelligence (AI) output from First Amendment coverage should account for those costs when human involvement is mixed or disputed.

The No-Speech Position

The no-speech position argues that because no person stands behind a model’s words at the moment of generation, those words are not First Amendment-covered “speech.” The leading versions of the no-speech argument arrive at the same conclusion via different routes. Mackenzie Austin and Max Levy argue that machine output lacks “speech certainty:” No human speaker knows what a model will say as it says it. Peter Salib contends that AI output is the protected speech of no one—not the model, developer, or user. David Atkinson, Jena Hwang, and Jacob Morrison argue that frontier models lack communicative intent and therefore produce no speech at all.

The appeal is understandable. If model outputs are treated as speech, laws regulating them may trigger heightened First Amendment review. That result can seem perverse for laws intended to address product safety, fraud, or discrimination. The impulse is not frivolous: It reflects a genuine worry that ordinary First Amendment doctrine might otherwise give constitutional shelter to large areas of automated system behavior.

The approaches differ over when subsequent human interaction is enough to earn constitutional protection, but each makes human attribution the deciding factor.

Benjamin Wittes framed the doctrinal pressure from the other direction. Where some no-speech scholars deny protection to machine output to avoid an unpalatable result, he argued that existing doctrine, followed faithfully, already extends protection to it. Writing in Lawfare, he arrived at what he called “the first machines with First Amendment rights.” Machines don’t have constitutional rights, but a company’s expressive rights are not the machine’s; they are the rights of people who own and direct it. But a court need not give rights to a machine to recognize that regulating its outputs may burden the rights of users, readers, or developers.

The difficulty is not the theory but its administration. In an individual lawsuit, attribution may be manageable. A court can examine the prompts, editing, selection, product design, model operation, parties, and claims after discovery. A rule applied across the internet must make similar judgments without that record, often before anyone knows who contributed what. And the words themselves usually cannot resolve the question.

The Distribution Problem

The attribution question would not stay confined to lawsuits against developers. Most text reaches readers through intermediaries—such as platforms, hosts, search engines—and a no-speech rule changes their incentives too. Distributor liability for illegal or unprotected material can still raise constitutional concerns when it chills protected speech. But the most significant protection for online intermediaries is statutory: Section 230 of the Communications Decency Act, which bars treating a provider or user of an “interactive computer service” as the publisher of information provided by another “information content provider.” That shield does not depend on whether the underlying content is protected.

Section 230 fits less comfortably when the service itself appears to generate the substance of a communication. An LLM generates new text in response to a prompt, blurring the line between hosting third-party content and creating one’s own. No court has decided whether chatbot output is the kind of “information provided by another” that the statute covers, and Section 230’s own authors have said it should not be.

That shield could give way for AI-generated outputs from either direction. Courts may hold that Section 230 does not reach model output, or Congress may withdraw it. The pressure to do the latter is real and bipartisan: In December 2025, senators led by then-Sen. Lindsey Graham introduced the Sunset Section 230 Act following a similar House proposal the year before. A repeal would expose anyone who hosts online content to liability for it, but under a no-speech rule, that exposure would be highest for machine output. Every distributor would then have a concrete reason to know which items moving through its systems are machine-made and which users are human. Complete repeal aside, legislators have shown an interest in eliminating protection for AI under Section 230, which might also exclude AI-generated content from its shield.

The risk does not vanish under a no-speech rule even if Section 230 survives. The statute limits liability for hosting another’s content; it does not limit what a state may demand of a platform’s own conduct. A legislature that placed unadopted machine output outside the First Amendment could regulate it directly—by barring platforms from carrying it, or by conditioning access on proof that a real person stands behind each post. None of this is settled law, but the incentive structure is clear: Make human attribution the line between protected and unprotected expression, and governments concerned about AI-generated disinformation, harassment, or foreign influence will push institutions to sort human from machine at scale.

When Attribution Becomes the Test

The attribution problem does not arise in every case. A law regulating a known chatbot provider’s conduct or requiring automated responses to identify themselves need not determine whether an unknown online speaker is human. The problem is harder when the source is disputed.

Suppose a state requires machine-generated election advocacy to carry a conspicuous label, while exempting materially identical advocacy written or adopted by a person. Consider an automated agent that a campaign volunteer set running weeks earlier with a single general instruction—promote this candidate—and that now drafts and posts its own advocacy. A human set the process in motion with a communicative purpose, but no human wrote, selected, or adopted the specific post.

If that distant instruction counts as adoption, the compelled label burdens human expression; if not, the no-speech theory makes the constitutional objection weaker. A classifier might flag model-like phrasing. Provenance might confirm an AI tool touched the file somewhere along the way. A personhood credential might confirm a human controls the account. What none of them can show is adoption—that a specific person stood behind this specific text. Enforcement then turns not just on detection, but on the legal default: whether ambiguous posts are treated as protected unless the government proves otherwise, or unprotected unless the speaker proves adoption.

What the Tools Can Prove

Institutions asked to make that sort have essentially three families of tools—text classifiers, content provenance, and behavioral analysis—plus a fourth, personhood credentials, discussed below. Each answers a real question. None answers the one a human-attribution rule poses. Text classifiers are the most direct option, but they are unreliable and answer the wrong question. OpenAI withdrew its own classifier in 2023 because of low accuracy. Researchers have also found that detectors disproportionately misclassify writing by non-native English speakers.

Even a perfectly accurate classifier only tells you whether a text resembles model output—not how, if at all, a person was involved, as drafter, editor, or adopter. In a single enforcement action, that gap need not be decisive: An agency with subpoena and warrant power can compel the logs, account records, and testimony a classifier never sees. The difficulty is the routine, at-scale sorting the distribution problem invites, where a platform or rule-maker must separate human from machine expression across millions of items, often with only the words to go on. As mixed human-machine production becomes routine, a binary label becomes less informative.

Provenance is better evidence of how a file was made and changed than of who’s responsible for what it says. The C2PA standard, developed by an industry coalition that includes Adobe and Microsoft, can attach a signed, tamper-evident history to digital content. By its own documentation, “Content Credentials,” the C2PA standard’s consumer-facing label that lets readers inspect a file’s recorded history, confirm that provenance data is tied to an asset and hasn’t been tampered with—it doesn’t vouch for whether the underlying claims are true.

Behavioral systems, such as spam filters, bot detection, device fingerprinting, posting patterns, and network analysis, can identify likely automation. But they are probabilistic and adversarial, and they generally classify an account rather than particular words. The more confidence a system seeks, the more closely it must inspect the person or device behind the content. As Madeline Lamo and Ryan Calo observed in “Regulating Bot Speech,” requiring someone accused of operating a bot to prove otherwise may force that person to reveal their identity.

Personhood Is Not Authorship

Privacy-preserving personhood credentials respond to a real problem in the agentic web (an internet where AI agents act on people’s behalf): Systems increasingly need to know whether an account represents an accountable person, a delegated tool, or an autonomous agent. Renée DiResta has described this as a problem of “attribution collapse.”

A 2024 paper by researchers affiliated with OpenAI, Microsoft, and major universities proposed credentials that would let a user prove they are one real person without disclosing their identity to each website. Zero-knowledge protocols—cryptographic techniques for proving a statement true without revealing the underlying data—could establish that a credential is valid while preventing different services from linking the holder’s activity.

That design could reduce fraud and coordinated manipulation without routine identity disclosure. But anonymity from a given website is not the same as never having identified oneself. A person must first obtain the credential by presenting documents, undergoing a biometric scan, or receiving an institutional attestation. A zero-knowledge proof presented at the time of use may conceal the enrollment record from the website, but an issuer has still certified that a particular person exists. The privacy protections also depend on how much data issuers retain, whether records can be pooled, and how they respond to legal demands.

Nor does the burden depend on any law mandating verification: it can become coercive without ever becoming formally mandatory. Once verified speech receives more trust or visibility, unverified speech begins to look suspicious. Vitalik Buterin, while supportive of proof-of-personhood systems, has similarly warned that one-person-one-credential designs may weaken the protection offered by multiple, disposable pseudonyms.

A personhood credential still does not prove authorship. It shows that a unique person is present. A verified person can publish text generated entirely by a model, while an automated system can distribute text written entirely by a person. None of the existing tools reliably shows that a credential holder created, selected, edited, or adopted a particular work.

When Personhood Becomes a Proxy

Faced with that gap, institutions are unlikely to give up. They are likely to substitute a question they can answer—Is a real person here?—for the one they cannot: Did a person author this? When institutions seek a scalable proxy for human involvement, they may choose more intrusive systems. Privacy-preserving credentials are technically and institutionally demanding. Biometric verification is easier to explain and tied to one body, but it still establishes personhood rather than authorship.

World, co-founded by Sam Altman, uses dedicated hardware to scan users’ irises and certify that each is a unique person. The company says nearly 18 million people have verified through an orb and presents the system as an answer to AI’s growing ability to imitate people online. The overlap is notable: Altman also leads OpenAI, whose products helped create that demand. World has been suspended, banned, and investigated over privacy and consent in Europe, Asia, Africa, and Latin America. Biometrics also present a particular risk: Unlike a password, an iris cannot readily be replaced after a data breach.

Age verification offers the closest existing analogue to what at-scale personhood verification would look like in practice. Twenty-five states now require age checks for websites hosting material deemed harmful to minors, commonly through government identification, transactional data, or facial-age estimation. The information collected for those systems can leak. In October 2025, Discord disclosed that a third-party breach may have exposed about 70,000 government-ID images submitted for age-related appeals.

Verification also excludes people who lack acceptable documents. A national survey found that millions of voting-age Americans lack current government photo identification or carry identification with an outdated name or address. The burdens fall disproportionately on lower-income people and people of color.

All of these systems trade anonymity for verification. The First Amendment has long treated compelled identification as a speech burden. In Talley v. California, the Supreme Court invalidated an ordinance requiring handbills to identify their sponsors. In McIntyre v. Ohio Elections Commission, the Court described anonymity as “a shield from the tyranny of the majority.” Those decisions did not make anonymity absolute, but they required the government to justify compelled identification. Using identity or personhood as a proxy for authorship would reverse that starting point without resolving the attribution question.

Why the “No-Speech” Classification Matters

The administrability problem also affects scrutiny. In Free Speech Coalition v. Paxton, the Supreme Court upheld a Texas law requiring adults to verify their age before accessing obscene material. Because minors have no right to that material, the majority treated the burden on adults’ protected access as incidental and applied intermediate rather than strict scrutiny.

The tier of scrutiny is often outcome-determinative: Strict scrutiny requires a compelling interest and the least restrictive means while intermediate scrutiny asks only for an important interest and a reasonable fit. Justice Elena Kagan’s dissent warned that the First Amendment “prevents making speech hard, as well as banning it outright.” The distinction between “speech” and “no speech” can therefore determine the outcome.

A government defending a source-based rule could make a similar argument. If unadopted machine output falls outside the First Amendment, measures used to separate protected human expression from unprotected output can be characterized as incidental—burdens that fall on protected speech only as a side effect of regulating unprotected material.

Suppose a state provided that only verified humans may post on a large social-media platform. Because unadopted machine output would carry no First Amendment interest, the government could cast the verification step as an incidental burden on protected speech—much as Texas defended the age gate in Paxton—even though every human user would now have to authenticate before saying anything at all. The burden on a person required to establish human attribution then appears incidental rather than as a direct restriction on speech. That classification may determine the applicable level of scrutiny.

An attribution requirement would reach far beyond Paxton. That case concerned a defined category of sexual material, a distinction between adults and minors, and an established interest in restricting minors’ access to material unprotected as to them. A rule turning on human attribution could apply across subjects and platforms, conditioning speech on proof about its source.

Tailored Regulation Is Possible

Rejecting a categorical no-speech rule doesn’t mean giving up on regulation—it shifts the inquiry. Instead of asking whether a human produced or adopted each output, courts would ask whose rights a given law burdens, and whether that burden is justified.

Any law reaches identifiable parties. A user may prompt, select, edit, or adopt an output. A reader may assert the right to receive information. A developer may exercise editorial judgment through design and dissemination choices. None of those propositions requires treating the machine itself as a constitutional rights holder. Garcia itself illustrates the alternative. Judge Conway declined, at the motion-to-dismiss stage, to treat the model’s output as protected speech but allowed Character Technologies to assert its users’ right to receive it. The First Amendment also does not immunize fraud, unlawful data practices, defective product design, or failures to disclose merely because a product communicates.

Courts need not grant rights to machines or remove AI output from the First Amendment to ensure that AI is regulable. Human verification may be useful against fraud, impersonation, and automated abuse. But personhood should not determine whether expression receives First Amendment coverage.

The First Amendment problem Wittes identified is real, but it does not turn on whether a machine has rights. It turns on how the law treats a single moment of machine generation once people go on to select, edit, adopt, and circulate whatever it produced. The question that remains is who bears the cost when no one can say for certain who, if anyone, stands behind the words.