Artificial Intelligence – A Counterintelligence Perspective: Part I

Artificial intelligence will change the world. Because so many people and companies believe this, AI and the entire technological ecosystem in which it functions are highly valuable to private-sector organizations and nation-states. That means that nations will try to identify, steal, and corrupt or otherwise counteract the AI and related assets of others, and will use AI against each other in pursuit of their own national interests. And that presents the United States and its allies with a classic counterintelligence problem in a novel and high-stakes context: How do we protect a valuable national asset against a range of threats from hostile foreign actors, and how do we protect ourselves against the threat from AI in the hands of adversaries?

In the broad and diverse discussion of artificial intelligence in the global technological and economic infrastructure of the future, this question has received remarkably little attention.

In this post and others to follow, I will endeavor to explore some of the counterintelligence risks and problems presented by AI and the AI ecosystem. I’ll first talk about the general problem of AI and counterintelligence and then, in later posts, dive into some of the specific areas that cause me the greatest concern in this sphere.

Technological advancements often change society. With AI, however, the nature, scope, and speed of the change are different from what we are used to. AI will impact society broadly and deeply, and it is all going to happen very quickly. As a result, it will be increasingly hard to think clearly about the risks and opportunities that AI presents. Calculating risks by thinking about threats, vulnerabilities, probabilities, and the impact or costs of bad outcomes is more challenging with AI because it is so different. These sorts of calculations normally take time. They also require at least an intuitive understanding of the risks of bad outcomes—and the probabilities associated with those risks. And they require decision-makers who are savvy about these risks and have good instincts about technology that is changing fast. None of these factors are sufficiently present right now.

A Few Key Concepts

AI is already revolutionizing the world and will continue to do so at an increasing rate. As the Brookings Institution’s John Allen and Darrell West have written recently, AI will significantly impact the world’s economy and workforce, the finance and health-care systems, national security, criminal justice, transportation, and how cities operate. All of this change is likely to redistribute and concentrate wealth, challenge political systems, and generate new cyber threats and defenses. As Paul Scharre points out in his excellent recent book, “Army of None,” AI also will fundamentally alter the way wars are fought. It will change how nations conduct espionage against each other and what secrets they try to steal. As others have pointed out, the range of national security implications of AI is substantial, everything from significantly enhanced intelligence analysis of large data sets to AI-facilitated forgeries of audio and video media to swarms of armed flying drones autonomously attacking targets on the battlefield.

But what, exactly, is AI? Can the concept be defined clearly enough that people who are not computer scientists understand what is being talked about? There are many ways to define and explain it—for example, see here, here, here, and here. Stuart Russell and Peter Norvig have a useful framework for thinking about what AI is here. For present purposes, however, let’s take the following as a useful working definition of AI in the counterintelligence context in which I will be addressing it. As a lawyer who is not a technologist, I approach this task cautiously because AI is complex and difficult to grasp fully.

One of the most basic definitions of AI is, “AI is a machine that can think.” That high-level definition, however, raises certain questions: What makes up a machine? What is thinking? How do thinking machines act? Answering those requires more effort, and a comprehensive analysis of all of that is beyond the scope of this article. But here are a few things to consider about those questions for present purposes.

First, the machine. For my purposes here, an AI machine or system consists of: (1) hardware (such as very fast computer processors); (2) the software that runs on the hardware (including an operating system and the instructions and algorithms that enable the machine to engage in cognitive functioning); (3) input and output devices for the machine to take in information and instructions and to communicate or act on what it thinks (such as sensors to enable it to ingest and process visual or audio information, communication interfaces like text or speech, and robotics to enable it to take certain actions like walking, driving or flying); and (4) data that the machine can store, process, and analyze so that it can learn and improve (and the more of the right kind of data, the better).

Machines also require something that that counterintelligence officials need to think about: lots of money and uniquely talented people to make it all happen.

There is never enough money to go around, and talented AI experts are exceedingly rare and therefore highly valuable. So when you think about the impact, value and counterintelligence risks associated with AI machines or systems, you need to think holistically about this entire ecosystem. AI exists, in short, in a context that integrates various sub-disciplines and related fields including Big Data (and Big Data analytics), high-speed computing, sensor technology and robotics (including autonomous vehicles and systems), and that also requires people and significant capital.

Next, what is thinking—and what about a corollary to thinking: learning? A comprehensive and completely accurate definition of “thinking” may not be possible and, anyway, is beyond the scope of this post. The same might be true for “learning.” And I’m not Alan Turing, in any event, and wouldn’t endeavor to try a rigorous test for either. So for our purposes, let’s just say that an AI machine or system can access or perceive information, process it in some meaningful way, learn from the information or stimuli to which it has access, and improve its performance over time. As Russell and Norvig highlight, an AI system might think or act in ways that are analogous to what humans do, or it might do so in some way that is rational but different from what humans would do.

Now, what about how AI systems act? Currently, AI can perform certain specific or narrow functions (“Narrow AI”) that humans can also perform, such as processing and analyzing structured and unstructured data. Structured data includes formatted text documents. Unstructured data includes audio and video input or files. So AI systems can evaluate large volumes of metadata, emails or Word documents, for example, and also understand speech, recognize faces, perceive physical objects, and move. In many instances, AI can do these things faster, better and more cheaply than people can; in others, AI systems are not as good as humans are. Many of these narrow AI advancements come together today in autonomous vehicles, including cars, drones and robots. AI is also highly relevant to cybersecurity, both offensively and defensively (more on that in later posts). And you probably use AI systems daily if, for example, you text friends using an iPhone or Google voice-to-text system or if you use certain navigation apps.

A few other aspects of AI are relevant for this overall discussion. As far as is known publicly, AI does not yet exist at the level of human capability across the broad range of things that we can do both cognitively and physically. AI that is at roughly the same level as humans is often referred to as Artificial General Intelligence (AGI). Artificial Super Intelligence—AI that exceeds human capabilities—would have profound implications for society and the human race (see here and here, for example).

Many scholars doubt whether advancements such as AGI and Artificial Super Intelligence are possible. I’ll leave that discussion to others, but note one important feature of it for counterintelligence: Any nation or organization that achieved either of these breakthroughs would have substantial advantages over all of its competitors, perhaps permanently. Imagine a country that could build a computer with AGI or super intelligence and then scale that production and build an analytic army of those computers. Whether or not this is possible, the dream of it, both offensively and defensively—the desire to do it and the desire to preclude other countries from doing it and the desire to steal whatever progress anyone else may be making toward it—is a significant motivator for any number of international actors. In other words, the AI-related fears reflected in “I, Robot,” the “Terminator” movies or “Black Mirror” may or may not have any merit or plausibility, but they are a background factor even as we think about much more limited narrow AI applications.

What’s more, those narrower AI applications that exist already and that can reasonably be expected to emerge in the foreseeable future already present numerous opportunities and risks that will be challenging enough for the moment.

One additional background point: It is important to discuss the difference between what is called “explainable AI” or “ethical AI” and what is sometimes referred to as “black box AI.” The basic idea is simple: An explainable AI is one in which a human can understand and explain what the AI did and why it did it. An ethical AI is a system that is not biased in some unacceptable way. For example, with explainable and ethical AI we can answer basic questions such as, why did the autonomous vehicle decide to turn left instead of right? Why does the AI think that the face in the surveillance camera video is the same as the one in the mug shot? Why did it translate this sentence from Arabic into English in this fashion? Why does the AI think that this person is more likely than another person to commit a crime? Why does the AI think that person is not a good credit or insurance risk? Is the system biased for or against certain races or genders? AI systems that we can’t understand are black-box AI systems. Black-box systems are the opposite of explainable systems and of ethical systems, though a system can be explainable and still be biased.

AI systems can have biases that need to be understood. Those biases may come from the programmers who write the initial code to create the AI algorithms, or from the data sets that the programmers exposed the AI to for it to learn, or from some other source that people may not understand initially. The worry here is that gender, racial, ethnic or some other bias may have unintentionally influenced the initial programming or selection of the data set for learning. Such biases can have profound effects on the outcomes that the AI produces, something people need to be mindful and cautious about. Making sure that AI is explainable and ethical requires dedication and effort. As I understand it, the mathematics associated with some AI design today are so complicated that even the most advanced human mathematicians struggle to keep up. Developing and maintaining explainable and ethical AI systems will require effort and vigilance. In later posts, I’ll have more to say about some counterintelligence concerns I have about black-box AI that expands on comments I made at a recent Brookings event.

With that as a high-level background on AI, let’s consider a similarly high-level background on the field of counterintelligence. In subsequent posts, I will explore why AI poses particular challenges for counterintelligence professionals.

Counterintelligence—An Overview

Counterintelligence is a broad field and one shrouded in secrecy for obvious reasons: At its core, it is about spies and the people who try to catch them. It is not possible for me to provide a comprehensive treatment of the field here, but I’ll try to highlight a few key points that I think are at the root of what counterintelligence is about. I have tried to make my points below regarding counterintelligence broadly applicable, but I acknowledge that, given my background, they are more U.S- and FBI-centric.

As with AI, it is important to define what we are talking about when we speak of counterintelligence. One definition I find useful is that used in Executive Order No. 12333, as amended, which is the principal executive order governing the activities of the U.S. intelligence community. Section 3.5(a) of the executive order states: “Counterintelligence means information gathered and activities conducted to identify, deceive, exploit, disrupt, or protect against espionage, other intelligence activities, sabotage, or assassinations conducted for or on behalf of foreign powers, organizations, or persons, or their agents, or international terrorist organizations or activities.” I will talk about various aspects of that definition below but mostly in relation to the activities of the foreign intelligence services of hostile foreign nations and their agents, which can include actual officers of such services (intelligence officers, or “IOs”) but also individuals those services recruit, co-opt or dupe into assisting them. In practice, dealing with that threat is the main focus of counterintelligence agencies and the most dangerous threat faced by private organizations in the AI business.

Under Executive Order 12333, several elements of the U.S. intelligence community have counterintelligence responsibilities, but the FBI has assumed the lead counterintelligence role within the United States. The FBI, which has been in the counterintelligence business since 1917, says that its main counterintelligence goals are to:

• Protect the secrets of the U.S. Intelligence Community, using intelligence to focus investigative efforts, and collaborating with our government partners to reduce the risk of espionage and insider threats.

• Protect the nation’s critical assets, like our advanced technologies and sensitive information in the defense, intelligence, economic, financial, public health, and science and technology sectors.

• Counter the activities of foreign spies. Through proactive investigations, the Bureau identifies who they are and stops what they’re doing.

• Keep weapons of mass destruction from falling into the wrong hands, and use intelligence to drive the FBI’s investigative efforts to keep threats from becoming reality.

Notice as a preliminary matter that protecting the AI assets of the United States falls within the scope of those self-defined goals. And those goals are useful reference points for understanding the counterintelligence world. But let’s dive in a bit deeper in order to set up a framework for comments I will make later about the AI-related counterintelligence threat. So here are 10 general points that I think are relevant to understanding the counterintelligence world:

1. Counterintelligence is reactive. Counterintelligence is about stopping bad things from happening. It is about identifying and countering the intelligence activities of hostile foreign intelligence services and other hostile foreign actors (such as terrorist organizations). This includes countering both clandestine intelligence gathering activities and other clandestine intelligence activities. The former category is what many might think of as traditional spying; the latter is more like conducting sabotage and secret influence and disinformation campaigns to manipulate a foreign population. Those “other” clandestine intelligence activities would fall within the category of what we in the United States would think of as “covert action”—activities seeking to influence political, economic or military conditions abroad, where it is intended that the role of the particular government is kept secret.

2. Counterintelligence is proactive. To be effective, counterintelligence must not be just reactive; it also must be aggressively proactive. So if you are a counterintelligence official, you are proactively trying to collect information about the identities of your adversaries; what they are thinking, planning and doing; and whom they have recruited, co-opted or fooled to help them. You have to be willing to take thoughtful and well-considered risks in order to recruit sources inside hostile foreign intelligence services and on foreign soil. You have to deploy sensitive technical collection techniques, tactics and procedures to spy on adversaries. All activities in this regard need to be intelligence-driven, strategic and executed using the best possible tradecraft.

3. Counterintelligence is protective. As a counterintelligence official, you are trying to protect and defend the nation, its people, its assets, and those of its allies from your adversaries. Indeed, the motto of the British Security Service (or MI-5 as it is more widely known), which is Britain’s lead counterintelligence agency, is “Regnum Defende” ("Defence of the Realm"). MI-5 says that its “mission is to keep the country safe. For more than a century we have worked to protect our people from danger whether it be from terrorism or damaging espionage by hostile states.” You have to understand comprehensively and deeply what are the most important national assets to protect.

   For example, you might need a comprehensive “heat map” of all the valuable assets of the entire nation; or of a particular geographic region or of important sectors of the economy, telecommunications system or the defense industrial base that you need to protect. You might want a heat map of the AI ecosystem that catalogues clearly what assets you need to protect, how vulnerable they are, how effectively the risks to those assets have been mitigated and what gaps remain. You also need to understand what the bad guys think they need to do regarding what you are trying to protect—in other words, what do they think they need to steal or corrupt? Unfortunately, you will never be certain about whether you or they have a better picture of what assets they should focus on.

4. Counterintelligence challenges are overwhelming. There are too many adversaries and too much to protect. In a free society, you can’t defend against everyone everywhere all the time. You have to prioritize the threats and what you are trying to protect (your heat map will help). Too often, the adversaries will win.

5. Counterintelligence is done at home. Counterintelligence activities occur mainly domestically, even though they are focused on hostile foreign intelligence services. Of course, adversaries operate globally, and they will try to recruit and exploit Americans abroad and will engage in malicious cyber activities anywhere, but mostly they focus on the U.S. homeland. This means that counterintelligence authorities must find foreign intelligence operatives who are hiding domestically and mixing in and interacting with the very people you are trying to protect. As a result, it is often hard to find the bad guys as well as the Americans who are helping them, either wittingly or unwittingly. This is the same type of problem counterterrorism officials face.

   Because all of this is happening at home, counterintelligence activities intended to identify and disrupt intelligence threats domestically necessarily pose risks to the civil liberties of Americans. To be effective, counterintelligence officials must engage in a range of intrusive clandestine activities in the United States intended to thwart their adversaries, including electronic surveillance, surreptitious searches, recruitment of sources, physical surveillance and undercover operations. Such activities undertaken here at home invariably implicate the rights of Americans no matter how hard counterintelligence officials try to avoid doing so.

6. Counterintelligence is confusing. Your adversary is actively trying to deceive you, and it is hard to figure out what is going on. It is easy to make mistakes—you can easily miss real threats, plots and actors, and you can waste time, resources and effort following the wrong people or countering the wrong (or nonexistent) plot. Your adversary is trying to get you to be complacent or chase your tail, and, if you are not careful, it is all too easy to let that happen.

7. Counterintelligence is sophisticated. Effective foreign intelligence services are very important for the security, political stability and economic well-being of nations. Foreign intelligence collection provides national leaders and government agencies with the information they need to make informed military, diplomatic, economic and other important decisions; anticipate and address the actions of foreign nations; and protect the nation from attack, sabotage and other threatening activities by adversaries. As a result, countries are going to invest a lot in making sure that their intelligence services can defeat counterintelligence efforts to stop them.

   Accordingly, foreign intelligence adversaries are well resourced, experienced, dedicated, motivated and aggressive. They use complex, novel and subtle means to defeat counterintelligence services. They recruit and place sources; they co-opt or threaten insiders; they conduct physical and electronic surveillance and break-ins; and they engage in sophisticated intelligence tradecraft to obscure their activities and deceive adversaries and innocent third parties. Moreover, the spy business has changed a great deal over the years with the advent of the Internet and the explosion of open-source information. And intelligence activities are now inextricably intertwined with the digital ecosystem, malicious cyber activities, and advanced perception management and manipulation campaigns.

   The essence of the clandestine intelligence-gathering business (i.e., espionage) is to collect secret information by secret means. In other words: (a) someone wants to protect information, data, technology, weapons systems or other important assets from being stolen, damaged or destroyed; (b) spies want to steal or do other bad things to those assets; and (c) the spies want to do so, if at all possible, in a way that keeps the victim from knowing or understanding what happened. From the spy’s perspective, it is best if the victim never knows that something was stolen or corrupted or, if the victim does find out that something bad happened to the asset, that the victim does not know the identity or role of the spy in the activity.

   U.S. counterintelligence officials must figure out how to deal with all of this. And while they have to adhere strictly to the Constitution and laws of the United States, their adversaries follow different rules or no rules at all. The same is true for our closest foreign partners, who must follow the rules of their own countries.

   Some people have likened the counterterrorism challenge to that faced by a soccer team that cannot allow the opposing side to score even one goal. In both the counterterrorism and counterintelligence contexts the situation is actually much worse. Imagine that the soccer team not only needs to prevent the opposing team from scoring any goals but also has to deal with an opposing side that may be either invisible or wearing uniforms identical to their own; that plays by different rules or no rules at all; that can score by leaving the field and lifting up the net from behind and putting the ball in for a goal; and that all of this is happening on a field that is undulating and otherwise changing constantly because of the dynamic nature of technology, the economy and the intelligence needs of adversaries. Counterintelligence is hard and requires sophistication, in part, because the environment can be highly disorienting.

   For a variety of reasons, the field of counterintelligence has at times been considered a backwater by some and has played second fiddle to efforts to address ordinary crime (before 9/11) and counterterrorism (after 9/11). Counterintelligence investigations, however, present some of the most complex and vexing problems that a national security agency can face. For example, the counterintelligence investigations of Robert Hanssen, the Russian illegals network (eventually known as “Ghost Stories”), the Russian efforts to influence the 2016 presidential election, and a plethora of cyber-enabled foreign intelligence activities by China and others demonstrate the sophisticated nature of counterintelligence work. In order to disrupt appropriately the activities of foreign intelligence services while preserving the long-term efficacy of their sensitive sources and methods, counterintelligence officials are regularly called upon to make hard choices about whether, when and how to carry out affect such disruptions.

8. Counterintelligence is forever. Hostile foreign intelligence services represent persistent threats. Nation-state adversaries generally don’t go away or give up, even if regimes change. For example, the FBI has been dealing with intelligence threats from the Soviet Union and the Russian Federation since 1917. The FBI will be dealing with the intelligence threat from China for as long as it exists. To be sure, the clandestine intelligence activities of foreign powers change over time depending upon the needs of the country and developments in fields such as science and technology, politics, and the economy. So counterintelligence officials must be adaptive and creative over an extended period.

9. Counterintelligence is powerful. The FBI has all of its national security and law enforcement authorities available to use against hostile foreign intelligence services and their agents. It can leverage the resources of its partners in the U.S. intelligence community; other federal agencies; its foreign intelligence, counterintelligence, and law enforcement partners around the globe; and approximately 18,000 domestic law enforcement agencies. It has the authority and capability to conduct highly intrusive electronic surveillance and searches of Americans and others in the United States (generally without ever being required to tell them that it did so). It can arrest Americans and others in the United States, and seek the arrest and extradition of anyone anywhere.

   But as many casual observers do not understand, arrest and prosecution of foreign intelligence operatives are not the only ways to thwart the intelligence activities of hostile foreign adversaries. Without going into much detail here, counterintelligence officials have a range of other options available to identify, understand and disrupt the activities of foreign powers and their agents. These include recruiting and doubling-back sources the adversaries themselves have recruited (i.e., “double agents”); deporting foreign nationals (with the help of the Department of Homeland Security) and kicking foreign diplomats out of the country (with the help of the State Department); providing information to support diplomatic actions or the imposition of economic sanctions; public exposure; and providing defensive briefings to individuals or organizations that are being targeted by a hostile service. Counterintelligence officials can accomplish their mission even if no one is ever prosecuted or jailed, and they will continue their mission even if there is no prospect of arrests and prosecutions.

   Combined with the forever nature of counterintelligence, this point has important implications. There is a constant spy-vs.-spy quality to counterintelligence: us watching them, and them watching us. It has a plethora of different modalities. And it never stops.

10. Counterintelligence is highly regulated. Because of the nature and scope of the counterintelligence activities described above, and the misuse of counterintelligence authorities and resources in the past against Americans (such as with the FBI’s investigation of Dr. Martin Luther King Jr. in the 1960s), counterintelligence activities are highly regulated in the United States. For example, the FBI must conduct its counterintelligence activities in conformance with Attorney General Guidelines, the Foreign Intelligence Surveillance Act (FISA) and a range of other federal statutes that regulate its actions. The FBI is accountable with respect to its counterintelligence mission to the attorney general, the director of national intelligence, the inspectors general of the Department of Justice and the U.S. intelligence community, and the permanent intelligence committees of Congress. In a free society, it is essential to have robust oversight and accountability mechanisms in place to monitor the activities of our counterintelligence agencies.

In future posts, I want to integrate these two complex subjects in ways that illuminate present and future problems, such as in protecting the AI assets of the U.S. and its allies, and defending against our adversaries’ use of AI.