Lawfare Daily: Nuclear Weapons in the Age of AI, with Joshua Keating

[Intro]

Joshua Keating: You know, at a certain point, what does it actually mean to have a human in the loop if the human is getting all their information from AI and, and their decisions are being influenced by it? Are we really that comforted by the fact that it's ultimately a human being whose finger is on the button?

Scott R. Anderson: It's the Lawfare Podcast. I'm senior editor Scott R. Anderson, with Joshua Keating, senior correspondent at Vox.

Joshua Keating: When you talk about AI and nuclear weapons, it's, it's, it's the nuclear weapons you should be worried about more than the AI, and that the same tools, political, technological, that, you know, have kept us from blowing ourselves up for the last 80 years, I think we can continue to apply those even in this new world of artificial intelligence.

Scott R. Anderson: Today, we're talking about nuclear weapons in the age of artificial intelligence.

[Main Podcast]

So Josh, you spent the last several months writing a series about what I think is in some ways the most cinematic, the most widely discussed fear of AI, and this is the idea of AI intersecting with nuclear technology. AI taking control of nuclear weapons, the thing that we all think of as the most destructive force that humanity has developed and still uses, still has available to it, or, or at least to many states, including here in the United States.

And it's interesting because to some extent, in a universe where regulation and constraints on AI, at both the domestic level and particularly at the international level, are completely absent, this is the one topic where we have something that looks like regulation, or at least begins to step in that direction. We have an agreement between then President Biden, although current President Trump hasn't rescinded it as far as we know, and Xi Jinping of China basically saying, "We agree we're not gonna hand over control of nuclear weapons to AI systems," that you need to keep a human in the loop, in the, in the vernacular of automated weapons.

But in your, these pieces that you've put together for us, you make the point that that doesn't get at the real concern about AI, or at least doesn't alleviate it entirely. Talk to us about why that is and where this intersection of AI and nuclear weapons, in your view based on your research, can still be particularly dangerous.

Joshua Keating: Yeah, it's interesting that you use the word cinematic because when I, when I told people about this project I'm working on, you know, it, it's movies that really kind of inform a lot of people's thinking on this. Often you'll see articles about this topic illustrated with pictures of the Terminator with glowing red eyes. Several people I spoke to at Los Alamos told me, you know, assured me that they're not building Skynet. The former head of STRATCOM has assured us they're not building WOPR, the computer from the 1980s Matthew Broderick movie Spy Games.

But- That's kind of the idea we have, this idea that, you know, computers, that an artificial intelligence system would actually take over launch command of the nuclear weapons, that it would launch the weapons without, you know, humans in the loop, to use that phrase.

But, a- and there is actually quite a bit of agreement that we don't want that. You referred to the Xi-Biden agreement. If, you know, I talked to several former commanders of Strategic Command, and they were all very adamant that, you know, they would not, they wanna maintain a human in the loop. They don't wanna give the launch codes to ChatGPT.

But that isn't really the issue. M- At the same time, there is an increasing encroachment of AI systems into the larger nuclear architecture. We see, you know, different aspects of AI used for threat identification, for so-called decision support when it comes to, you know, both to nuclear command and control itself and increasing use of AI in conventional military targeting systems. And, you know, there's concerns that that could lead to greater risk of nuclear escalation.

So, you know, at a certain point, what does it actually mean to have a human in the loop if the human is getting all their information from AI and, and their decisions are being influenced by it? Are we really that comforted by the fact that it's ultimately a human being whose finger is on the button?

Scott R. Anderson: So, you make the point, and I think it's a, it's a savvy one, about why this temptation of AI is so real in the nuclear context. We have lived under the security umbrella, by some accounts also, though it comes with a fair amount of risk, as many of others would point out, of this nuclear command and control system that is designed specifically to act quickly and with a degree of autonomy, potentially if, to account for different nuclear contingencies, like first strike by an enemy.

So talk to us about why AI is such a tempting tool given the strategic context in which we've kind of developed and continue to operate this command and control system. What about its speed, its efficiency, its autonomy makes it so potent, so potentially something that, obviously, in spite of this agreement that, yeah, you'll keep a human in the loop, it brings so many tempting advantages or tempting capabilities to people who think about nuclear strategy, who are implementing U.S. command and control, and presumably Chinese, Russian, other countries' command and control as well?

Joshua Keating: Right. I mean, when it comes to command and control, it's really a question of speed. In an event of a nuclear attack, the president might have as little as 30 minutes or even less to make a decision about whether to retaliate. And that's not really a reasonable time for any human being who's not prepared for it to make any decis- any important decision, much less one with, you know, literally civilizational consequences.

And so, you know, what advocates of greater use of AI will tell you is that, you know, these systems can, you know, pull signal from the noise faster than humans can. They can identify threats. They can provide a list of possible courses of action for the president or, you know, senior military decision-makers that can just help them, support them through what are re- what's really an impossible decision and help them make it more quickly.

The problem with this is that there's any number of near misses in the history of nuclear war going through the Cold War, where we've gone to the brink because of failures of automated systems and been pulled back from the brink because of humans taking a step back. You know, the, the famous case of Stanislav Petrov in the Soviet Union the, the officer who detected, you know, what looked like a U.S. missile launch that turned out to be some light reflecting off clouds and chose not to notify his superiors in Moscow and wait for a few minutes to see if they could correct it.

There's the famous story of Zbigniew Brzezinski, Jimmy Carter's national security advisor, being woke up, woken up in, in the middle of the night, being told that Russian submarines off the West Coast had launched missiles, and he had to decide whether to wake Jimmy Carter up. He ultimately decided not to. He waited a few minutes. You know, these, these are ultimately cases of human beings who decided that something just didn't feel right, something didn't make sense, and they wanted to wait and, you know, not trust the data they were being given by what at the time were the most advanced surveillance and communication systems available.

Now, the, with artificial intelligence, the sort of threat detection and the signal processing is a lot more advanced than it was in the 1970s and '80s. But the flip side of that is, you know, perhaps we're just more trusting of these systems. There's a problem some researchers I spoke to have talked to called automation bias, where people sort of default to trusting these systems. So maybe artificial intelligence is better than the computers we were using in the '70s and '80s, but maybe it's also a problem that we're less likely to question the data that it's providing for us.

Scott R. Anderson: So let's talk about what the unique dangers are of AI compared to other technological systems or of human evaluations. Humans make mistakes. Human makes evaluative errors. AIs, there's a variety of circumstances where there's reason to think AI may be better calibrated to avoid errors than human beings.

On the flip side, you have, we have this record where, because of exactly the severity of nu- the use of nuclear weapons, the human costs of it that maybe humans are uniquely sensitive to, you've seen instances where we've seen individuals in the loop pause and wait, wait for more information, second-guess something. I suspect if it were a less weighty decision, you may not have seen the NSA hesitate to take action or respond to that data. It's precisely 'cause of the severity of the nuclear strike, it strikes me  that, that you have this second guessing on the part of the human actors.

So what is it about AI specifically as opposed to existing data systems which have some degree of automation and have for a long time because of that speed desire? What makes it particularly dangerous about the inputs of data, about the kind of black box nature of them? What does that all amount to? What are the different specific vectors of concern we have to think about, about AI that are particularly relevant in the nuclear context?

Joshua Keating: One issue is the black box problem. The fact that, you know, even the designers of some of these large language models don't always fully understand why they are saying the things they are saying, why they're providing the answers they are. And in a situation of this severity, you wanna understand not only the information you're getting, but why you're getting that information. You know, what, you know, reasoning process the program is using to arrive at the recommendations that it's giving you.

Another particular problem in the context of nuclear weapons is that we just don't have a lot of training data. Nuclear weapons have only been used in warfare twice. We've never fought a nuclear war with, you know, thankfully, where two sides are both employing them.

So when you look at what information what models are these systems relying on to, you know, recommend a course of action in a nuclear war, a lot of it's tabletop exercises, a lot of it's theoretical, a lot of it's books. So, you know, are you getting a recommendation out of, you know, some system that's read Schelling or read Herman Kahn, and maybe that's a good thing. But it's, it's not like other endeavors where there's, you know, sort of real world examples that these systems could draw on.

And there's, there's actually a couple of recent studies that show that in many cases where they have, you know, large language models like Claude, like ChatGPT, play war games, play nuclear war scenarios they tend to be on the hawkish side. They tend to be faster to recommend nuclear signaling or even nuclear use than human players of these games. And, you know, I, I, I, it, it, you know, you and you wonder, you know, is, is this because they've read their Herman Kant's on nuclear war and just sort of see this as a rational course of action, as a, you know, form of coercive bargaining, whereas, you know, a human being who might be thinking about their family being incinerated in that moment might calculate things a little differently.

Scott R. Anderson: So, to some extent, we've already seen elements of this come into play, particularly in the conflict in Iran, to some extent in the conflict of Ukraine. We've seen AI used in a variety of ways in conventional warfare. What have we learned about from the applications of AI in those contexts that might either alleviate or accentuate some of these concerns in the nuclear context? And have they pointed towards any sort of strategies that might help address them?

Joshua Keating: It's been interesting. O- over the course of the months that I've been working on this project, I think it's, it's one of these things that's gone from a more theoretical futuristic problem to one that feels very present.

You know, if you can look at the war in Gaza, for instance, where there's been reporting of the Israeli Defense Forces using a targeting system called Lavender to identify potential Hamas targets. You know, there, there's, according to some reports, that system has an error rate of up to ten percent, which is, you know, has, has human consequences when you're talking about the war in Gaza. Could have, you know, worldwide catastrophic consequences if you're talking about a ten percent error rate in a nuclear context.

And then, of course, there was the debate around the use of anthropic software by the Pentagon and the sort of clash between Secretary Hegseth and the leaders of Anthropic. And at least part of that dispute it's been reported, came down to the question of whether Anthropic would allow its systems to be used in a kind of missile defense scenario, in one where there was an incoming missile attack and military commanders had to plot about retaliation.

And that might seem like a slam dunk case. Like, you know, of course, we'd wanna use any advantage we could to shoot down incoming missiles. But you know, I think any, anyone who's sort of looked at the history of missile defense and, and nuclear war knows that, you know, that, that when you get into those systems, there are escalatory consequences to the type of planning that's going on, so.

So I mean, ultimately, I, I think what it comes down to is that I'm sort of less worried than I was when I started this reporting about AI getting control of nuclear weapons and wiping out humanity, and more concerned about humans using nuclear weapons to wipe out humanity and artificial intelligence sort of helping them a-along in that process.

You know, i-in the case of, for instance, the Minaab school bombing in Iran, there's been a lot of reporting and speculation about whether artificial intelligence was used in that targeting and, you know, were they using Claude, were they using Project Maven, this Palantir design system? And y- you know, I, I'm very interested in the answers to those questions, but ultimately, I think it's ultimately less important than the fact that it seems as if outdated targeting information that, you know, showed that this school was a military target was what was being fed into the system, and that this was all happening in a context of a Pentagon that's, you know, has a certain amount of contempt for rules of engagement and, and the, the role of military lawyers and you know, wants to take the fetters off when it comes to targeting and, and loosen the you know, regulations when it comes to preventing civilian casualties.

So when you add AI to those contexts, then, then yes, I'm concerned that it's the kind of thing that's sort of speeding up decision and making, maybe taking some of the human deliberation out of the equation. But I think ultimately the bigger concern is how humans are using these products than the, the intrinsic dangers of these products themselves.

Scott R. Anderson: And we've already seen this debate about the use of these products come up in a pretty high-profile way. That is, as one of the kind of two prongs of the argument between Anthropic that develops the Claude, the various models of Claude, the LLM, including the Fable and Mythos model that have been particularly controversial more recently, and the Department of Defense, or sometimes the-called the Department of War, where we know limits Anthropic maintains, kind of as a matter of principle and policy, on its use in an autonomous weapons systems was a sticking point, and a point which ultimately led the Department of Defense to take what I think many people characterize as retributive action against Anthropic and the Claude model including a supply chain designation that's currently the subject of litigation. Although we've seen the department back off some of those positions, the broader U.S. government back off them a bit in the ensuing weeks and months.

You've done a little coverage of this in relation to this series. Talk to us about that sort of conversation and how that interfaces with this. Is this an issue set where we're seeing self-regulation by the AI industry? Is there a good reason the Department of Defense wants to break down these barriers imposed by Anthropic as a corporate entity? How does that all enter into this nuclear context, which is Obviously, again, a very high profile and high concern context for autonomous weapons, but not the exclusive one by any stretch of the imagination.

Joshua Keating: It's been interesting to watch the Pentagon become sort of more reliant on working with, you know, companies like Anthropic and Google and, and also this sort of new generation of you know, what they call neo-prime, Silicon Valley-based defense companies, the Palantirs and Andurils and SpaceXs of the world.

You know, I, I, I talked about this with Jack Shanahan, a former Air Force general who was one of the sort of f- the, he was the first leader of Project Maven, which was sort of the partnership with Palantir that developed this sort of AI-enabled targeting and battle management system. And the way he put it is, you know, when the Pentagon goes to a company like Northrop Grumman or Lockheed, if they come to them and say they want to build the Death Star, the question they'll get back is, "How big do you want it?" You know? "How bi- how big a laser gun do you, do you want to put on the Death Star?" Whereas,

Scott R. Anderson: Avoid that tranche.

Joshua Keating: Right.

Scott R. Anderson: You have to get that out of there, which would be-

Joshua Keating: Yeah.

Scott R. Anderson: Would be a recurring problem.

Joshua Keating: Whereas, you know, these companies are a little different sort of culturally. I mean, they, they, they have their own ideas and, and often you know, we've seen y- you know, sometimes it's, there was, when Project Maven first emerged, there was a lot of controversy with Google. A number of employees, you know, didn't wanna be part of building these systems for the military.

We've seen in the, you know, Ukraine context, Elon Musk decide to, you know, cut Starlink access to the Ukrainian military because he was worried about nuclear escalation. During the Iran war, you know, SpaceX, you know, jacked up the price of you know, broadband systems for these suicide drones that the Pentagon was using.

So yeah, I mean, it, it's a different sort of culture, and I think that an interesting political shift we're seeing is that these companies kind of have their own ideas about how these products are gonna be used in a way that you know, the sort of traditional military industrial complex did not. And I, I think that does change some power dynamics, and I think, I think that's partly what's going on in the Claude Department of War Pentagon dispute. I, I think we're, we're sort of seeing sort of new relationships and rules of the road and norms emerging out of that, and I'm not sure exactly how it's gonna shake out yet.

Scott R. Anderson: So when we talk about nuclear weapons, nuclear strategy, nuclear policy in the United States and frankly most countries for the last, most of the last century since we've entered the nuclear age 80 or so years ago, the thing, the word that is at the center of everything is escalation. The concern that, you know, countries that even don't want to get in a nuclear conflict, that generally have avoided getting a nuclear conflict, we've only seen nuclear weapons used twice by the United States at the end of World War II, nonetheless will find themselves in a cycle that will move them up the escalation ladder to ultimately use nuclear weapons at a strategic level. That could be devastating for humanity and for, you know, the, the globe as a whole.

You make the argument in one of your pieces that in fact the integration of AI into nuclear technology could change the conventional understanding or conception of escalation, particularly that escalation ladder model that is so inherent in a lot of how we think about escalation in a nuclear context. Talk to us about that and some of the kind of new models, the new ways to conceive of escalation that come about when you bring AI into the picture.

Joshua Keating: Yeah, I mean, there, there's a great model that comes from the book “Army of None” by Paul Scharre. He's a former Pentagon official. Where he, he compares it to the Flash Crash. People may remember back in 2010 there was this crash of the Dow Jones, the stock market, which was basically caused by these very fast advance trading algorithms just responding to each other and, you know, the, y- the Dow Jones lost 9% of its value within minutes before, you know, human traders even realized what was going on and then, and then just as quickly it shot back up.

And so I think, you know, when, when people talk about the risk of AI and escalation I, I think that's what they are concerned about. That, you know, in a, in a U.S.-China brinksmanship scenario If both sides have these advanced AI systems, if both sides are using AI to interpret the other's moves, to detect threats, to recommend courses of action that they're just going to be responding to each other faster than human officials or military commanders actually want them to. And before we know it, a conventional military conflict could turn into a nuclear one.

This is the idea of you know, what would happen if we had had AI during the Cuban Missile Crisis, for instance. If you know, when the U.S. imposed a blockade of Cuba, you could imagine, you know, alternate universe President Kennedy just telling the Russians that all the ships in the U.S. Navy had been programmed to automatically fire on any ship breaching the blockade, and that you know, he, he has no control over it.

And so, you know, th- this is kind of, in some ways, escalation management 101 is the famous, like, Cold War analogy of playing a game of chicken where one of the drivers tosses the steering wheel out the window of the car. That if, if you want to show your adversary that you're really serious and that they should back down, you sort of take the control out of your hands. You're like, "I can't do anything. It, it, you know, the, the system's all automated, you know. Mess with me right now, I, I, I can't pull back."

But, you know, I think, do the other side, do they trust that assurance? If your advantages in AI are so formidable, does that give incentive to your adversary to go nuclear earlier to sort of head off the advantage you have? And let's not forget, these systems are, could be vulnerable to cyberattacks or to misinformation, to, to misinformation, y- AI-generated misinformation is another vector of this.

And so I think, you know, I, I think there are a lot of ways that the traditional escalation ladder and the traditional ways we understand the ways that a political crisis can turn into a war and a conventional war can turn into a nuclear war gets scrambled in an age where decisions are being made much faster, where we are perhaps less trusting of the information we're seeing, where it's easier to generate disinformation where we don't always understand the sources of the information we're getting or why the systems that we're using are recommending the courses of action that we are. I think these are the ways that AI can sort of compound traditional escalation risks.

Scott R. Anderson: So, in your most recent piece in this series, you talk about a war game you participated in that brought to the fore some of these challenges that you actually had a first-hand role in, in, in simulating the role of a policymaker, a Chinese policymaker, I think, if I recall correctly. So, talk to me about that experience and, and what you learned about how the promises and perils of AI manifest in the nuclear context, particularly when, when actually making decisions as a policymaker.

Joshua Keating: Yeah, this was a war game, an ongoing one run by the Hoover Institution at Stanford. And they've been running these games for the past couple years. The one I participated in in D.C. last fall, my group and I played the role of a senior Chinese general basically advising Xi Jinping during a hypothetical Taiwan Strait conflict. You know, from our point of view, the U.S. had been you know, meddling in our backyard for too long and was on the verge of conveying a military advantage to Taiwan that would mean that the island would be lost to China forever. So we had to decide how to deter the Americans, get them to back down from their support of Taiwan without it escalating into full-scale war.

And part of it was this decision of whether to turn on a new automated AI-enabled targeting system, one. And then, once we turned it on, we had to decide if we wanted to make it fully autonomous or to put it in human in the loop mode, one where it would recommend whether to fire on the American ship or not. But a, you know, a, a human commander had to be the ultimate one to make that decision.

It showed one sort of disconcerting thing and one reassuring one. The disconcerting thing was in a sort of complex crisis where you have a lot of military assets in the region, where the risks are high, where the rules of engagement might be unclear, that there is gonna be strong incentive to try to sort of outsource some of that cognitive load on an AI system. In my group, we decided we want to turn, turn the system on.

On the other hand, I think it shows the limits of trust people put in these systems as well. I think everyone who played the game essentially decided they wanted to keep a human in the loop, that no- nobody thought that, you know, fully, fully autonomous was the way to go.

And I think one thing that struck me as interesting was that I think humans still see automation as itself a form of escalation. That the idea of taking away some of your own control, giving the machine more authority, is seen as in itself an escalatory move as, as a sort of provocative step people would make.

You know, I, I talked to one of the former STRATCOM commanders I talked to for my piece. You know, I said, "You know, yes, you don't... You, you want a human in the loop. How would you respond if, you know, you learn that North Korea, for instance, had fully automated their, you know, nuclear launch protocols that, that, that they, they, they, they had handed over the launch codes to ChatGPT or, or, you know, some equivalent?" And he said, you know, he doesn't think they would do that, but if they did, he would recommend a much more aggressive nuclear posture. That he was like, "This, this would change my thinking dramatically, and that we would have to get more, much more serious about our own nuclear posture and targeting of North Korea, and the military options were there."

So I think, you know, humans still, for all the concern about automation bias in a crisis scenario, which I think is real, I think humans ultimately still don't fully trust these systems. And, and there's actually been some recent research that shows that you know, trained West Point cadets when, you know, they take a course on artificial intelligence are actually less prone to automation bias than the public at large. So if there's good news here, you know, humans do still have some healthy skepticism about turning over control to these systems in a, you know, the most dangerous crisis we can imagine.

Scott R. Anderson: So, one interesting historical coincidence, although it may not be a coincidence, is that this rise of AI in the nuclear context, among many other contexts, is coinciding with the breakdown of what has long been the kind of international regime for non-proliferation and for controlling nuclear weapons: the end of the New START agreement, before that the end of the INF treaty, and a variety of other treaties that the first and second Trump administrations have withdrawn the United States from, chosen not to renew, otherwise allowed to break down, along with some other points of consistent tension that contributed to those decisions that have existed across multiple administrations, particularly with Russia. You know, repeated credible allegations that Russia was violating the treaties even before the United States ultimately chose to exit them.

Talk to us a little bit about what that means for these AI questions for the development of nuclear weapons, for the development of AI-related nuclear systems. I know you interviewed former Assistant Secretary Rose Gottemoeller, who was the Assistant Secretary of State for Arms Control under President Obama, about these sorts of issues. What did you learn from her and how has that entered into these sorts of AI questions?

Joshua Keating: I think that it's one thing that for all that the sort of nuclear diplomacy framework that we built up is breaking down, there is some general consensus on this question, at least on human in the loop, on this idea that, you know, the automated system should not be given full control. And, you know, if I had to put an optimistic view on it, it was like this might be the low-hanging fruit that could allow some of these conversations to resume.

I mean, one thing Rose Gottemoeller pointed out was that what worries her right now is that we're sort of getting out of the habit of nuclear diplomacy, that, you know, even through all the ups and downs of the U.S.-Soviet relationship and then the U.S.-Russian relationship in the sort of late Cold War and early post-Cold War era, that there was this sort of ongoing conversation about nuclear weapons, that the two sides, the negotiators kind of got to know each other. There was a kind of framework, a kind of nuclear diplomacy institution that built up, and now that's kind of dissipating as, as the folks who sort of participated in those talks, as, as they retire and move on. And, and, you know, with the expiration of New START, there's kind of not really much left in terms of nuclear agreements between the U.S. and Russia.

Now you have the added complication of, thanks to China's recent nuclear buildup, there's a third not quite pure nuclear superpower, but one that's sort of getting toward the same level where you have to start thinking about, you know, the so-called three-body problem, the, the sort of trilateral nuclear diplomacy and deterrence, which is a much more complicated problem than just two countries.

And that maybe, given that there's some agreement on, on at least this question that could be a way to sort of start these conversations. You know, we've, we've seen Secretary General Guterres at the UN, he's sort of warned about this. You know, the, the, the Pope's encyclical recently on artificial intelligence, you know, had a, had a lot about autonomous weapons that, you know, maybe in the absence of sort of wider nuclear diplomacy, like this could be at least a way to sort of restart some of those conversations.

Scott R. Anderson: So another interesting application of AI in the nuclear context has occurred outside of the command and control chain that's been the focus of a lot of our conversation, the focus of a lot of nuclear strategy considerations. And it's been at the laboratory level about developing nuclear technology, not just limited to weapons, but also more recently with weapons.

You had a really interesting series of conversations with people at some of the leading U.S. nuclear laboratories about how AI is affecting their work in good ways and potentially in, in some ways that might have concerns. Talk to us a little bit about that and the other ways AI is feeding into this nuclear system, even if not directly related to the use of nuclear weapons.

Joshua Keating: Yeah. In January, I had the opportunity to travel to Los Alamos and tour the Los Alamos National Laboratory. A couple of things struck me on that trip. One was that there's this sort of history of a symbiotic relationship between advanced computing and nuclear weapons. I mean, in the original Manhattan Project, the first computers were human beings making calculations. They were often the wives of the physicists working at Los Alamos were the human computers, and eventually they were replaced by IBM punch card machines. And ENIAC, the first digital computer, was used for some of the earliest hydrogen bomb research.

And that travels up through today, where some of the world's most advanced supercomputers are at these laboratories that their primary purpose is to maintain and research the U.S. nuclear weapons stockpile. And Los Alamos recently signed a partnership with OpenAI, the company that makes ChatGPT, to install ChatGPT on Venado, which is one of the supercomputers that models nuclear weapons tests.

So because the U.S. hasn't actually detonated a nuclear weapon since the early '90s, all the kind of research is done in simulations on these very advanced supercomputers that can simulate the effects of a nuclear blast, the impact of various environmental and atmospheric conditions of the materials that you would use. They're basically using ChatGPT to interface with these systems and to conduct these experiments.

And what's interesting, when you visit Los Alamos, in spite of the fact that, if you look at its budget, it's primarily a nuclear weapons lab, and they're actually now once again producing the plutonium pits, the sort of the actual plutonium cores of our nuclear weapons. But they're very anxious to point out to you all the pure science research that goes on there. And it's true, they do lots of re- the recent breakthroughs on fusion energy happened there. There's a lot of medical research that happens. But recently, Venado was moved from the non-classified to the classified network in Los Alamos, which means that it's being primarily used for nuclear weapons research.

And this is just one aspect of this sort of wider project by the Trump administration known as Project Genesis which is to sort of accelerate the adoption and use of AI throughout the National Laboratory System in America through- throughout these labs run by the Department of Energy, by the National Nuclear Security Administration.

You know, going back to the, you know, earliest days of nuclear weapons, you know, o- one thing it reminded me of is, is a famous quote, Niels Bohr, the, the physicist who did some of the earliest nuclear research, was skeptical that America would ever be able to build an atomic bomb. He said you basically have to turn the whole country into a giant factory to produce the fissile material needed to make this bomb and, and to actually design it. And when he came to the U.S. and saw what they were doing at Los Alamos, he was, you know, he was shocked, and he told Oppenheimer, like: "Wow, you really did it. You turned the whole entire United States into a giant factory to build nuclear weapons."

And, you know, that reminded me, you know, when we look at the amount of you know, money and energy, I mean, like literally watts of energy going into running these data centers to sort of fuel AI, to run these supercomputers, to run data centers, like it, it seems we're once again turning the entire country into a factory this time to sort of, not to produce nuclear weapons, but to sort of support the adoption of, yhese advanced artificial intelligence models.

And I, you know, I, I think the AI nuclear analogy can be a little overdone sometimes. They're very different technologies and being produced in very different contexts, and certainly the people who work on nuclear weapons at Los Alamos are, are sort of resistant to that analogy. But, but I think there are some parallels there in terms of the amount of resources and energy we're devoting to a technology where we don't really fully understand the consequences of it.

Scott R. Anderson: So, this project has given us a window, to some extent, into a future that we're entering. And frankly, at this point, it's probably not avoidable. AI is becoming a reality in a huge range of areas, and I don't know how realistic it is to pretend it's gonna be fenced off from nuclear weapons or the use of nuclear weapons, or if that's even a good thing, even if there are maybe risks that come with whatever rewards it may bring forward.

Based off your research, based off the numerous scholars you talk to, practitioners you talk to, what do you look for when looking at how this administration, future administrations, future Congresses approach these questions, and I guess foreign governments as well, to think about how we're doing it responsibly? Is the Biden-Xi agreement at least a good first step in the right direction? And how much further does it need to go, whether domestically or through international agreement, to feel like we're entering the nuclear AI age, or the AI nuclear age, in a responsible fashion?

Joshua Keating: In some ways, I think the Biden-Xi agreement is sort of the low-hanging fruit, that there's agreement that you know, we want humans make- humans, as flawed as they are we want them the one, to be the ones to make these decisions that could, you know, hold literally millions of lives at stake. We don't want AI systems, no matter how advanced or how fast, making those decisions.

But, you know, when you look at what, what does it actually mean to have AI as part of the nuclear enterprise, I mean, it, you know, there, there are plenty of applications very few people would object to. There's things like, you know, predictive maintenance, where you can use AI to predict when you're gonna have to repair, you know, some technical system, when, when some valve is gonna break down. AI can anticipate when that's gonna happen, so you don't have to wait till it happens.

Most people wouldn't object to that, but there's sort of a wide gulf between that and Skynet and you know, we're probably gonna land somewhere in the middle. And I think that we have to negotiate where we're comfortable drawing that line and, and hopefully, I, that's a international conversation, not just a national one. It's one that we're talking both to our allies and also to our adversaries about.

There was one researcher I talked to who's actually a very staunch advocate of more automation, who's somebody who, who wants, you know, effectively a dead-hand system like, like the Russians had during the Cold War, where if the president got in- incapacitated, that systems could be launched automatically. And what he told me was, "If you don't trust humans to design an AI good enough to make these decisions, then you shouldn't trust them to have nuclear weapons either." And I said, "You know what? Even as somebody who's more skeptical on nuclear weapons, I think I agree with you. If we can't be trusted with AI, like, we shouldn't be trusted with nuclear weapons either."

And so I guess it's, ironically, you know, in, in exploring this, this sort of brave new world that I wrote about, I think the questions and the challenges aren't all that much different than they're always been. I mean, you know, we're, we're in a situation where we're a species that, you know, since 1945 has had the ability to wipe itself out. That's something new in human history and I think that when you talk about AI and nuclear weapons, it's, it's, it's the nuclear weapons you should be worried about more than the AI, and that the same tools, political, technological, that, you know, have kept us from blowing ourselves up for the last 80 years, I think we can continue to apply those even in this new world of artificial intelligence.

Scott R. Anderson: Well, that is a point of potential hope. That's a good one for us to end on. Joshua Keating, thanks for joining us here today on The Lawfare Podcast.

Joshua Keating: Thanks for having me.

[Outro]

Scott R. Anderson: The Lawfare Podcast is produced by the Lawfare Institute. If you want to support the show and listen ad-free, you can become a Lawfare material supporter at lawfaremedia.org/support. Supporters also get access to special events and other bonus content we don't share anywhere else. If you enjoy the podcast, please rate and review us wherever you listen. It really does help.

And be sure to check out our other shows, including Rational Security, Allies, The Aftermath, and Escalation, our latest Lawfare Presents podcast series about the war in Ukraine. You can also find all of our written work at lawfaremedia.org. The podcast edited by Jen Patja with audio engineering by Cara Shillenn of Goat Rodeo. Our theme song is from Alibi Music. And as always, thank you for listening