Algorithmic Disgorgement is Bad for Science and Society

The Federal Trade Commission (FTC) is using a penalty called algorithm disgorgement to punish companies who use improperly sourced data in algorithm development and training. Their goal is to dissuade others from using illegal data and to punish offending firms.  

However, algorithm disgorgement punishes society at large by throwing away valuable data and, potentially, advances in computer and data science. While it is a strong threat against illegal data collection perpetrators, it risks damaging beneficial technical advancement. As an alternative, publicly releasing the wrongfully obtained data punishes the offender—perhaps more than by deleting the data—and ensures that society gets the benefits from the harm that it has already suffered.

What is Disgorgement?

Disgorgement is a penalty that requires “a party who profits from illegal or wrongful acts to give up [these] profits” While it has been used recently by the FTC in regards to algorithm data collection offenses, the disgorgement penalty has a history that goes beyond these recent developments.  

The Texas Gulf Sulphur case, in the late 1960s, was a seminal demonstration of the federal government’s ability to force violators—in this case of insider trading laws—to give up (or disgorge) their profits from the offending activity. Every year, the Securities and Exchange Commission (SEC) pursues numerous insider trading actions that result in disgorgement. In fact, the level of disgorgement is often higher than the civil penalties that the agency wins in these cases. Between 2005 and 2015, over half a billion dollars of profits was disgorged—just due to insider trading.  

The SEC has also used disgorgement in cases where brokers made choices that advanced their organization’s interests over that of their duty to clients, such as by moving higher-performing traders to an internal fund or not disclosing the likelihood of mortgages in a security failing.

When the SEC collects these funds, it doesn’t take pallets of U.S. currency out back of the agency’s offices and burn them. This, however, would be the equivalent of the remedy used by the FTC with algorithm disgorgement

FTC Algorithm Disgorgement

In a recent case against WW International (formerly Weight Watchers) and its subsidiary Kurbo, Inc., the defendants were ordered to “delete or destroy any Affected Work Product,” which was defined as “models or algorithms developed in whole or in part using Personal Information Collected from Children through the Kurbo Program.” The defendants allegedly “marketed a weight loss app for use by children as young as eight and then collected their personal information without parental permission.” While the FTC is to be lauded for protecting children and punishing firms for “illegally collecting kids’ sensitive health data,” the penalty has broader implications than the particular case and punishes society at large—not just the perpetrators—for their bad acts.

The Kurbo case is not the only example of the FTC’s use of disgorgement. It was also used in the Everalbum and Cambridge Analytica cases. The Everalbum case centered around the company’s use of user-supplied photos to build a facial recognition service that was sold to commercial customers to recognize individuals for security purposes without proper user knowledge and consent. The Cambridge Analytica case related to harvesting of user information from Facebook to purportedly build “black box” algorithms.

Algorithm disgorgement was also mentioned by FTC chair Lina Khan as part of the agency’s toolbox.  Commissioner Rebecca Slaughter went further, stating that the “enforcement approach should send a clear message to companies engaging in illicit data collection in order to train AI models.” She also suggests its potential use in a variety of other types of cases that might come before the FTC, such as those involving decision making bias impacting members of protected classes and those involving price-setting and collusion.

Disgorgement’s continued use seems likely. Some have suggested, for example, that the internet data harvesting collection-based training process used by ChatGPT could potentially be targeted for disgorgement. And it has even been proposed as a penalty for “misbehaving self-driving cars”.

The FTC’s Argument for Disgorgement

The key argument behind algorithm disgorgement is that a fine may not be a sufficient deterrent to prevent illegal actions (such as unauthorized or otherwise inappropriate data collection) being used to create an algorithm. Given the high value of some algorithms for gaining and holding consumer interest, recommending products, and other uses (TikTok’s algorithm, for example, is at the heart of ByteDance’s over $200 billion valuation), even the largest FTC fines may be a cost a company is willing to pay to obtain a high-performing algorithm in a key area. Disgorgement of algorithmically derived profits, while another potential remedy, is inherently problematic, as determining exactly what profit can be attributed to a given algorithm may be difficult—if not impossible.

Algorithm disgorgement—in addition to other remedies such as fines—makes it so that the company can’t engage in illegal practices, seeing any potential fine as a cost of doing business in making the algorithm. Disgorgement inherently changes this perception because it makes it so the company can’t keep its illicitly obtained algorithm benefit, no matter how much it pays in fines.

Implementation Problems

There is some question as to what authority the FTC has to order algorithm disgorgement; The agency hasn’t had to make a legal case for this authority yet, due to its use being confined to unchallenged consent orders. Yet, even supporters of the technique suggest that it may be successfully challenged.

Additionally problematic is that significant confusion exists as to how disgorgement can actually be implemented. Full disgorgement would require an ability to track all direct and indirect beneficiaries of data. This is a herculean task and may be impossible if records of data use don’t exist. There is also a question of how the FTC could be sure that the disgorgement had been completed in full.

Moreover, would the FTC expect—or could they force—a third party, uninvolved in the consumer protection violation, to destroy a technological development that benefited from the offending data product (or perhaps even its derivative data products)? It seems unlikely that they could secure such a remedy against algorithms outside the U.S.’s borders, potentially harming U.S. competitiveness. 

In addition to lacking enforcement mechanisms outside of the United States, a number of other international considerations exist. First, algorithm developers may avoid domiciling their company in the U.S. if they fear the potential of algorithm disgorgement. Other countries could become algorithm havens, potentially creating the type of national security concerns that have been raised regarding TikTok, due to U.S. users’ data being stored and processed beyond the reaches of U.S. government oversight.

Second, if companies don’t domicile themselves in an algorithm haven altogether, they might have business partners or others who do (or even create an algorithm holding company there). In many cases, this will result in these companies (and, perhaps, their governments) having access to the disgorged algorithm that the U.S. company that developed it doesn’t have access to anymore. These companies could potentially use it to compete with both the offending firm and other unrelated U.S. firms. Alternatively, they might try to license it to U.S. firms to use.

While the FTC could take additional actions to try to prevent use of the algorithm in the U.S., this is burdensome on the agency and requires some mechanism to identify subsequent uses. 

Beyond all of these implementation considerations—and even if there weren’t problems with confusion and enforcement—throwing away the potential societal gains doesn’t make sense. Consumers have been harmed—they shouldn’t be harmed again by being denied whatever benefit can be gained from their first harm.

Public Release as a Solution

Instead of throwing away the algorithm or data, it could be made available for public use. The FTC could force firms to provide the algorithm and other supporting materials to nonprofits and government agencies. Any personal data could be sanitized (and, in some cases, data wouldn’t be needed at all); however, the fundamental advance and knowledge would then be available to benefit society. If the personal data could not be sanitized, the underlying dataset wouldn’t be made readily available (though the disgorged algorithm, generally, could be). In some cases, non-sanitizable data—such as data where training had used personally identifying information or information that could facilitate deductive re-identification of subjects—could be made available to researchers under disclosure and use restrictions consistent with ethical use and public benefit considerations.

While the firm—like everyone else—would be able to continue to use this knowledge, so would their competitors. They would lose the competitive advantage that this knowledge provides. The threat of competitors accessing algorithms could, potentially, be an even greater deterrent than losing them—as the company can’t just build another algorithm (possibly based on knowledge and principles gained from the offending activity and remembered by employees) to retain its lead.

Providing the algorithm to society at large mirrors the approach used for financial disgorgement, where profits are forfeited to the government (or, potentially, victims). It advances science, potentially removing a need for taxpayers to fund similar research in the future. It also makes a cache of knowledge available, in the public domain, for use by academic researchers, students, and others who lack the resources of large companies. This could potentially lead to other advances and societal benefits, without additional public burden.

Public release disgorgement is also responsive to the national competitiveness issues mentioned above. It punishes the offender, levels the playing field, and prevents firms outside of the reach of the FTC from gaining a commercial advantage over those subject to its regulation.

Developing a National Policy on Algorithm Disgorgement

Questions of data ownership, algorithm rights, and responsibility for algorithm decision making are growing in importance and will continue to do so. Developing a national policy related to the area of algorithm disgorgement, if it continues to be used as a penalty as seems likely given Commissioner Slaughter’s comments, is essential to ensuring that offenders are suitably punished while also protecting society’s interest in technological advancements.

Algorithm destruction disgorgement as well as, to a lesser extent, techniques that attempt to remove the influence of some data without requiring complete destruction, impair the public from receiving the benefits from data that may have been obtained improperly, causing damage that has already occurred and cannot be undone.  

Using prudence as to when disgorgement is merited is one solution. This would need clear policy guidelines to make sure that society and other interested parties’ interests are protected, instead of just the agency and offender parties to a particular case. Partial disgorgement-deletion may also be a possible remedy in some cases where only part of the algorithm is deemed to be offending regulations, if this is technically feasible (which won’t always be the case). Another possibility is use restrictions on the offending AI, which would require ongoing compliance oversight by regulators and presents a heightened potential for reoffending through improper use decisions.

There may be some algorithms that are seen to provide no public benefit where deletion disgorgement is appropriate (perhaps with an archival copy stored by the government, just in case it is needed later—or to facilitate certain types of research). However, in the vast majority of cases where deletion-based disgorgement would be seen as an appropriate remedy to punish the offender for illegal data collection, disgorgement through release to the public would provide a similar (and perhaps greater) penalty while providing public benefit. Thus, public release disgorgement, as opposed to destruction, is—for many cases—prudent public policy.