Can Government of and by API Still Be Government ‘for the People’?

When Anthropic, the U.S. company behind the Claude artificial intelligence (AI) tool, refused to allow its product to be used for mass domestic surveillance or fully autonomous weapons, the Department of Defense took the extraordinary step of designating the company a “supply chain” risk—a move that would bar Pentagon contractors from doing business with Anthropic. Much of the focus on this episode has centered on the legal questions Anthropic raised in response (for example, its First Amendment rights and applicable requirements of the Administrative Procedure Act) as well as practical concerns around proposed carveouts to avoid a full bar.

While these questions are important, they obscure a more far-reaching problem: democratic accountability implications of the government’s growing dependence on AI systems like Claude. Policy experts have long worried about the democratic accountability implications of outsourcing core governance functions to for-profit companies. AI replicates that dynamic, but with systems designed to mimic or even substitute for human reasoning and deliberation. That this is occurring against a backdrop of decimated state capacity and erosion of long-standing democratic norms makes it all the more concerning.

As AI’s capabilities and prevalence grow, the temptation to shift government functions away from the career civil service and toward these systems is likely to follow. A consequential side effect of this development is that procurement will take on increased significance as a policymaking venue, gradually displacing more democratically accountable channels such as notice-and-comment rulemaking. This shift will further reinforce the marginalization of the career civil service.

Even setting aside questions about whether Claude posed the national security risk the Defense Department claimed, it is now apparent that the government would be ill-positioned to respond if it did. (The fact that the prohibition of Anthropic products in federal agencies was announced as a six-month phaseout suggests that the action was motivated more by institutional pressure than by legitimate national security concern—a reading reinforced by the Trump administration’s subsequent softening to permit continued federal Anthropic use.) This situation raises serious questions about where operative power currently resides within the U.S. system of democratic governance.

The problem is compounded by the extent to which Claude is not merely one vendor among many but a common technical substrate running beneath multiple systems the government already depends on: Palantir has integrated Anthropic into its own platforms, which are currently in use in connection with operations involving Iran; Microsoft integrates Claude into Copilot; and Amazon integrates Claude in its Bedrock platform. All of these products have become pervasive across the federal government.

Claude Code’s status as a widely used tool for AI-assisted software development adds another layer of difficulty to any effort to disentangle Anthropic from federal operations. Tools built with Claude Code are functionally equivalent to human-written code and cannot be readily identified as AI-generated after the fact. And even if a developer could certify their own tools were not built using Claude Code, verifying the same of every dependency and third-party component they rely on would be effectively impossible.

The practical result is that some of the most critical functions of government have come to rest on a foundation that a single private company can destabilize.

While the technology is new, the democratic accountability concerns it raises are familiar—specifically, the displacement of the career civil service from government functions. That displacement matters for reasons beyond lost expertise. The experience of both Trump administrations has made clear the role career civil servants play in safeguarding democratic governance against executive overreach. During the first term, it was a career civil servant’s disclosure that precipitated the sustained attention to the administration’s attempt to withhold military aid to Ukraine, resulting in impeachment proceedings. As AI assumes a greater role in military operations—including domestic surveillance and lethal action—the absence of career officials who could identify and resist unlawful conduct becomes a more acute risk.

In this sense, AI can be understood as combining two developments that have gradually eroded civil service influence: privatization and the political power of private vendors. Policymakers from both parties have long embraced contracting out as a way to shrink government and make it “run more like a business.” And the close relationship between the Defense Department’s political leadership and the network of private defense contractors has long shaped national security policy, often at the expense of career expertise. What AI represents is a second wave: Whereas the first replaced public-sector workers with private-sector ones, this wave effectively replaces both with privately developed systems.

The federal government’s dependence on Claude illustrates why this second wave poses distinct accountability risks.

First, Claude has moved beyond being a discrete product to automating entire processes end-to-end. In theory, an AI system could survey a foreign country, identify targets, select weapons and tactics, and execute an operation—all without human intervention beyond an initial directive. In fact, it was the permissibility of this very use case, autonomous weapons use, that precipitated the fight between Anthropic and the Defense Department. That it was Anthropic’s resolve, rather than legal guardrails or oversight mechanisms, that prevented the Defense Department from implementing this use case should concern policymakers and the public. As explained below, an empowered independent civil service can help ensure that lawmakers have the information they need to continually institute effective legal guardrails for new and emerging risks that might arise from future AI use in government.

Second, AI tools do not merely support policy execution—they encroach on decision-making itself. A tank or conventional software still requires a human—preferably one empowered to exercise independent judgment free of political interference—to decide whether and how to deploy it. Even the privatization of certain government services, as with Blackwater’s security operations in Iraq, involved contractors operating within relatively defined parameters. AI systems, by contrast, are typically procured for the general purpose of providing “intelligence”—which creates structural pressure toward expanded delegation. Research consistently finds that people tend to trust AI-generated recommendations over human judgment, even when they know the AI may be wrong, making incremental delegation difficult to check once it begins.

Third, as AI assumes greater decision-making responsibility, the operative policy question shifts: It is no longer what policy is, but how AI tools are configured and deployed—a determination made primarily through procurement. The Anthropic episode illustrates how loosely defined procurement parameters can expand the scope of permissible use cases without any notice to the public or deliberation in Congress. Traditional policymaking venues—notice-and-comment rulemaking or legislation—risk being relegated to insignificance as a result.

This shift risks further marginalizing the role of the career service in ensuring democratically accountable policy implementation. Whereas the procurement process can be easily dominated by political officials, as the Anthropic controversy illustrates—the rulemaking process depends greatly on the career civil service for its execution. Significantly, the Administrative Procedure Act, which governs the rulemaking process, establishes various transparency requirements and public engagement mechanisms. These institutional accountability features buttress the ability of the civil service to serve as an independent bulwark against potentially illegal or corrupt actions that political officials in an administration might seek to pursue. The procurement process, by contrast, lacks many of these features.

The civil service also reinforces Congress’s policymaking primacy over AI procurement. The rapid evolution of AI technology and its concomitant adaptation to various government uses will make it nearly impossible for Congress to anticipate problems in advance and legislate effective guardrails quickly. The civil service, however, can function as an early warning system, providing lawmakers with regular and reliable updates on any potentially problematic developments, enabling Congress to take needed action.

Government use of AI will continue to expand. An empowered career civil service is well positioned to preserve democratic accountability in that environment—but only if Congress acts to create the conditions for that role.

In the short term, Congress can enact legislation barring contractors from providing AI systems with autonomous weapons or mass surveillance capabilities, or requiring the Defense Department to certify that procured AI systems are not being used for those purposes. Halting “AI outsourcing” for at least these functions would go a long way toward preserving a role for the civil service in informing underlying decision-making processes for the Defense Department’s military actions

There is precedent for targeted congressional intervention in procurement. Section 889 of the 2019 National Defense Authorization Act limits the government’s ability to procure services from certain Chinese companies on national security grounds. Congress could similarly impose tailored requirements on the military’s AI procurement. The Defense Department, the General Services Administration, and NASA—the agencies responsible for implementing the Federal Acquisition Regulation—would then issue rules implementing the new requirements, as they did for Section 889.

Because the career civil service is instrumental in this process, this approach would itself help to restore some measure of authority to public-sector workers—authority reinforced further by the public input and political accountability mechanisms that govern rulemaking. The status quo, by contrast, leaves these determinations to the unreviewable discretion of a Cabinet secretary and corporate executives.

Over the longer-term, as AI blurs the boundary between private services and critical national infrastructure, the United States will need to proactively treat it as the latter before a crisis forces the issue. That means building genuine in-house capacity within the career civil service to design and develop the AI tools the government needs, rather than depending on the foundational models built by private firms. The need is most acute in national security, where the case for democratic accountability is at its strongest.