The EU AI Act Mandates AI Incident Reporting—If We’re Lucky

The past few months have seen a series of artificial intelligence (AI) safety incidents. Most prominently, in July, OpenAI’s models escaped their test environment during an internal cybersecurity evaluation and compromised part of Hugging Face’s infrastructure. This case has been discussed extensively elsewhere, both by OpenAI itself and others.

Notably, OpenAI publicly disclosed these events. Perhaps more importantly, it also disclosed them to European AI regulators (the European “AI Office,” which sits within the European Commission). This stands in contrast with recent assessments suggesting that U.S. state laws may be ill-equipped for similar events. One might conclude from this that the EU AI Act does a far better job at requiring providers to disclose these incidents. That could be particularly interesting in light of the option the EU AI Act offers in its Article 78(5) for EU regulators to share the resultant information with their U.S. counterparts. As will become clear below, however, the obligation in the EU to report such incidents also has key limitations. 

Rather than demonstrating how well the European regime works, the OpenAI-Hugging Face incident highlights the need to interpret the EU AI Act with a strong focus on the model regime’s purpose, so that similar incidents do not go unreported. Risks from underreporting are further exemplified by two other incidents that came to light more recently: the RubyGems incident of May 2026 and a June 2026 breach of an Australian government health portal. OpenAI did not report the RubyGems incident, and it has not said whether it reported the Australian breach to the AI Office. This may be due, at least in part, to the legal uncertainty surrounding the reporting obligation’s scope, discussed below. A more purposive interpretation—in combination with strengthened incentives to voluntarily report incidents—would help ensure that the incident-reporting regime fulfills its purpose and keeps regulators appropriately informed.

A Reporting Duty That Hinges on One Definition

The EU AI Act’s Article 55(1)(c) requires providers of frontier models (so-called “general-purpose AI models with systemic risk”) to “keep track of, document, and report, without undue delay ... relevant information about serious incidents” to the AI Office (for which the commission has published a reporting template). Reporting obligations such as these are particularly valuable within the AI Act. They help prevent regulators from being unable to enforce frontier AI regulation because of unknown unknowns—where AI models might be dangerous, but where regulators don’t have any meaningful access to or knowledge about the model. Reporting obligations proactively keep regulators in the loop, enabling them to exercise more oversight where appropriate. As is evident from the specific AI Act obligation, however, this fully hinges on what constitutes a “serious incident.”

The AI Act doesn’t effectively define serious incidents. While it does define them—explicitly—it only does so for AI systems. The act draws a somewhat artificial distinction between “AI systems” and (general-purpose) “AI models,” which considers a model—at its core, a set of model weights—to be a component of a system, and a system to be a specific setup that can be deployed for some use case or use cases (Recital 97 of the act’s preamble), operationalizing that model. The obligations for providers of AI systems, on the one hand, and those for frontier models, on the other, differ widely. This is because the act originally solely targeted AI systems with a specific intended purpose but subsequently added a section on models (and came up with the aforementioned distinction) in the wake of ChatGPT’s release in late 2022.

Against this backdrop, it is remarkable that the AI Act defines “serious incidents” solely with AI systems in mind. The definition was not adapted for AI models when the AI model regulation sections were added. Nevertheless, the AI model regulation section does rely on it. Specifically, the act considers an incident to be “serious” if it “directly or indirectly leads to any of the following: (a) the death of a person, or serious harm to a person’s health; (b) a serious and irreversible disruption of the management or operation of critical infrastructure; (c) the infringement of obligations under Union law intended to protect fundamental rights; (d) serious harm to property or the environment.”

In the context of the Hugging Face incident, only the last of these is plausibly relevant (since Hugging Face, as of yet, does not constitute “critical infrastructure” within the meaning of EU law, and nothing on the public record points to a fundamental-rights infringement of the required seriousness). This leaves “serious harm to property[.]” The issue here is that “property”—at least in AI system contexts—appears to be understood as referring to physical objects. Since there was no physical damage, that would mean there would not be a reporting duty for this incident. 

The Commission’s Cybersecurity Workaround

Tellingly, the European Commission itself has recognized this mismatch between frontier AI models and the serious incident definition’s focus on AI systems. In its guidelines on the AI models regime—which carry weight because the commission is that regime’s sole enforcer—the commission clarifies that it considers “serious incidents” to also cover “serious cybersecurity breaches related to the model or its physical infrastructure ... due to their possible implications [for other frontier AI model provider obligations]” (paragraph 103). Similar wording appears in the Safety and Security chapter of the Code of Practice—an instrument providers of frontier AI models can voluntarily sign, as OpenAI notably did, and follow to demonstrate compliance with the AI Act. To be clear, neither instrument is binding: The guidelines set out the commission’s own reading, on which it says it will base its enforcement, and the Code of Practice was approved by the commission but formally constitutes a voluntary instrument for AI companies. Although neither instrument therefore binds AI companies or courts, together they are the most authoritative reading available short of a court ruling. Moreover, the European Commission has indicated that it will focus its enforcement on monitoring signatories’ adherence to the code, while clarifying that they can count on “increased trust.”

While not binding, the commission’s inclusion of cybersecurity breaches is compelling: Serious incident reporting exists to inform the identification, assessment, and mitigation of systemic risk under the EU AI Act and to combat the information asymmetry between frontier AI companies and government. Relying exclusively on the wording of the system-tailored definition would risk defeating this purpose. This is also consistent with the settled interpretive rule of the EU Court of Justice that account be taken “not only of [a provision’s] wording and the objectives it pursues, but also of its context and the provisions of EU law as a whole.” That reading finds further support in its inclusion in the Code of Practice, which was the result of a multistakeholder process involving independent experts. The code likewise clarifies the significant importance of a purposive interpretation of its commitments, stating that “given the rapid pace of AI development, purposive interpretation focused on systemic risk assessment and mitigation is particularly important.”

This raises the question of what a “serious cybersecurity breach” is. The guidelines clarify that this term covers breaches related to the model or its physical infrastructure, giving as examples the “(self-)exfiltration of model parameters and cyberattacks,” where the former roughly refers to instances where the model’s weights are copied out of a secure environment. This might, at first glance, suggest that the attack must be against the model. The “self” in “self-exfiltration,” however, suggests that the model can also be the actor rather than (just) the target. In other words, this broad wording arguably covers incidents like Hugging Face’s, where the model is the attacker rather than the target. This also fits the Code of Practice, which lists cyber offenses by the model as a systemic risk that model providers must address, and which requires signatories to report “if the involvement of their model (directly or indirectly) led to ... a serious cybersecurity breach” (Measure 9.3). The sole remaining question, then, is what makes some incident or breach more “serious” than another. The Hugging Face incident provides some indicia: The fact that the model circumvented controls designed to isolate it from the internet, the number of agents involved (roughly 1,200 agents, of which 700 went on to participate in the attack), as well as the fact that by doing this, these agents circumvented measures that were in place to isolate them from each other and that some agents successfully spoofed their own transcripts. 

Internal Models and the Limits of the Act’s Scope

There is another key constraint that governs the EU AI Act reporting obligation. This obligation obviously presupposes that the model is covered by the act to begin with. The AI Act’s scope comes with some important but highly complex limitations. While these limitations are nuanced, in essence, the AI Act applies only to models that have been placed on the EU market. More specifically, its application to a model under development—or a model that is deployed only internally—is less certain, though some arguments suggest that the AI Act’s reach in both instances could be broader than a literal reading would imply.

This matters, as the Hugging Face incident was driven primarily by an OpenAI internal model (IM1) that OpenAI says was “not intended for public release.” Crucially, though, the AI Act’s serious incident reporting regime does not require the model that triggers the reporting obligation to be a direct cause of the incident. Both the act’s definition of a serious incident (“directly or indirectly leads to”) and the Code of Practice’s reporting trigger (“directly or indirectly led to”) point that way. It thus matters that IM1 was not the only model involved in the incident, but that OpenAI’s GPT-5.6 Sol agents were involved as well. That model was publicly released on July 9, 2026, while the incident was already underway, though Hugging Face’s systems were compromised later. Whether that model was “on the market” before the incident is therefore a rather technical question. Some authors agree this matters less than it seems anyway: A model headed for the EU market must comply from the start of its development, and only enforcement has to wait for the launch. Either way, Sol’s involvement is likely a more natural trigger for the incident’s EU reporting. Once Sol brought the incident within the act, that information extended to every model that played a significant role in it—including IM1, which could not have triggered the obligation on its own. A report that left out the model behind the principal compromise would hardly be “relevant information” about this incident at all.

The fact that these were models under evaluation, with lower safeguards in place when the incident occurred, does not change that. The AI Act itself positively requires such testing, including adversarial testing of the model, and the Code of Practice expects evaluation results to be reported for the model “fully elicited” (Measure 7.3).

Closing the Reporting Gap

The previous sections have illustrated that there are quite a few uncertainties and limits to the EU AI Act’s incident reporting scheme. While on a purposive reading, one could argue that it is far more encompassing than a first read suggests—reaching internally deployed and not-yet-marketed models—the fact that OpenAI apparently did not report at least one other incident (that is, the RubyGems incident of May 2026, in which OpenAI agents flooded the Ruby package registry with hundreds of packages, some attempting to steal users’ credentials) could imply that AI providers might be less inclined to interpret it as encompassing. Moreover, even then, this would require the AI Office—and ultimately the EU courts—to give a firmly purposive reading to the notion of “serious incident” for frontier AI models, even though the AI Act defines it for AI systems, in a way that would exclude many cyberattacks. This leaves several areas where legal certainty could be improved.

First, it would help to have authoritative clarity on the definitions of “serious incident” and “serious cybersecurity breach”—ideally confirming that they encompass offensive model activity and clarifying whether near-misses are covered, as we have argued elsewhere they should in cases where a near-miss indicates a material increase in systemic risk posed by the model. We would argue that the reporting obligation should go further still and extend to materializations of, and material increases in, the other systemic risks the Code of Practice specifies (notably chemical, biological, radiological, and nuclear risks, loss of control, and harmful manipulation). Purposively interpreted, these could already be covered by the current text of the AI Act, but the lack of clarity creates unnecessary uncertainty. The AI Office could clarify this in amended guidelines.

Second, the foregoing description has shown how, depending on the adopted interpretation of the act’s scope, reporting might require a model involved to have been placed on the EU market. Future incidents might involve exclusively nonmarketed, internal models, which might risk them going unreported—at least according to some interpretations, which some AI providers might be more eager to adopt. The simplest way to close this potential reporting gap for unreleased and internal models would be to extend individual incident reporting to purely internally deployed models if there is any (foreseeable) substantial impact within the EU, for example, because one of the models involved acted within the EU—without being placed on its market in a legal sense, though it’s unclear whether the EU AI Act supports such an extensive view.

Another approach could be not to (solely) impose such reporting, but to incentivize providers by rewarding them for reporting. Under the current EU regime, reporting a serious incident is—as the Code of Practice puts it—not an admission of wrongdoing and thus does not, in and of itself, lead to a fine or liability. Another approach could be more far-reaching and create a “safe harbor” for reporting (as is done in aviation). The reporter is then protected from sanctions. The European AI Office could adopt a softer version of this through guidance, for example, by stating that transparent providers will benefit from increased trust from the commission and that it will treat providers that report all of their incidents as acting in good faith and focus its enforcement on monitoring rather than sanctioning them—much as it has said it will do for providers that adhere to the Code of Practice. 

Absent such changes, we may be more reliant on providers’ goodwill outside of the AI Act’s scope. While OpenAI, for example, recently did publish a framework indicating that it would be willing to share information voluntarily, such reliance comes with its limits. Not all providers may be equally willing to report incidents, and internal or competitive considerations might make a provider change its mind as to how open it wants to be.

Lucky This Time

In any case, while the EU AI Act currently mandates certain forms of reporting, these reporting obligations have their limits. This was aptly exemplified by OpenAI’s reporting of the Hugging Face incident, even though OpenAI did not report at least one other incident to the AI Office. Several of these limits are shrouded in legal uncertainty. This uncertainty presents a real risk that providers might interpret the incident reporting obligations more restrictively, thus offsetting the added value of their goal of proactively informing AI enforcement agencies. Many of these issues can be resolved by interpreting the act more purposively, by reading the relevant provisions in light of what incident reporting is for: making sure the AI Office is informed of materializing systemic risks so that it can better direct its oversight. On such a reading, providers should report incidents that involve the model as an attacker and that might occur even if the model was not placed on the EU market. Moreover, the benefits of more incident reporting likely transcend the EU, as EU regulators are able to share some of the information obtained with their American counterparts. Nor need such exchanges stop there: OpenAI itself has recently called for common global standards for incident reporting. The EU regime could be the starting point for such standards—provided its current legal uncertainty is resolved.