Bugs in Our Pockets: The Risks of Client-Side Scanning

For more than two decades, U.S. law enforcement has fought against the use of strong cryptography by the public in telecommunications. In 1992, the FBI argued that due to encryption, 60 percent of criminal wiretaps would be useless within three years—and, in the worst case, none might be intelligible. Ever since the U.S. government loosened cryptographic export controls in 2000, the FBI talked of doom and gloom regarding criminal investigations due to the public’s use of encryption.

Since the 1990s, the bureau has tried to thwart the use of end-to-end encryption, a system in which only the sender and the receiver can read the message. First, there was the Clipper, a National Security Agency design in which digitized voice communications would be encrypted with keys that would be split and escrowed by two agencies of the U.S. government. That didn’t fly; neither industry nor other nations were willing to use such a system. Next, there was the effort by FBI Director James Comey to press for exceptional access—strong encryption that provides access to unencrypted content to legally authorized searches. Technologists, including Lawfare contributor Bruce Schneier and me, argued that such solutions weren’t feasible. Mandating such a solution would decrease society’s security, not increase it. The Obama administration agreed, seeing the cost of widely available encryption tools as outweighed by the costs to public safety, national security, cybersecurity and economic competitiveness of imposing access requirements.

Law enforcement, and some national security agencies, haven’t given up. And despite the increasing number of former senior national security and law enforcement officials who have publicly supported the widespread use of encryption, U.S. law enforcement and allied countries around the globe are back with a new proposal to get around encryption. This one, in fact, does exactly that.

The new proposal is client-side scanning, scanning content on a user’s device prior to its encryption or after decryption. Supporters of the technology argue that such scanning can uncover child sexual abuse material (CSAM) without putting people’s privacy at risk. The supporters reason that people whose phones don’t have CSAM will have nothing to fear; the scanning will be local and, if there is no targeted material on the device, no information will ever leak from it.

Paul Rosenzweig wrote a long and thoughtful piece on the law and policy of client-side scanning in these pages a year ago. He followed up with two recent posts, one on Apple’s CSAM effort and another on Apple’s postponement of its deployment. Now my colleagues and I have written a technical analysis of the threats posed by client-side scanning systems, “Bugs in Our Pockets: The Risks of Client-Side Scanning.”

The proponents of these systems argue that they enable privacy while ensuring society’s safety by preventing forbidden content from being sent to the world. But as we describe in our paper, there are multiple ways in which these systems can fail, including by failing to detect targeted content, mistaking innocuous content for targeted material, and the like. It is far from clear that client-side scanning systems can provide the kind of successful evidence gathering that its proponents claim. At the same time, client-side scanning brings great danger. Such systems are nothing less than bulk surveillance systems launched on the public’s personal devices. Currently designed to scan for CSAM, there is little that prevents such systems from being repurposed to scan for other types of targeted content, whether it’s embarrassing personal photos or sensitive political or business discussions.

In 1928, in his dissent in Olmstead v. United States, Justice Louis Brandeis wrote:

When the Fourth and Fifth Amendments were adopted, “the form that evil had theretofore taken” had been necessarily simple. Force and violence were then the only means known to man by which a government could directly effect self-incrimination. It could compel the individual to testify—a compulsion effected, if need be, by torture. It could secure possession of his papers and other articles incident to his private life—a seizure effected, if need be, by breaking and entry. Protection against such invasion of “the sanctities of a man’s home and the privacies of life” was provided in the Fourth and Fifth Amendments by specific language …. But “time works changes, brings into existence new conditions and purposes.” Subtler and more far-reaching means of invading privacy have become available to the government. Discovery and invention have made it possible for the government, by means far more effective than stretching upon the rack, to obtain disclosure in court of what is whispered in the closet.

Moreover, “in the application of a Constitution, our contemplation cannot be only of what has been, but of what may be.” The progress of science in furnishing the government with means of espionage is not likely to stop with wire tapping. Ways may some day be developed by which the government, without removing papers from secret drawers, can reproduce them in court, and by which it will be enabled to expose to a jury the most intimate occurrences of the home.

Client-side scanning, by exposing the personal photos, thoughts, and notes from a user’s phone, does exactly what Brandeis feared might come to pass. Read our paper to understand the technical flaws of client-side scanning solutions and why they provide neither safety nor security for society.