U.S. Vows to Fight Distillation Attacks
-security.jpg?sfvrsn=982907ba_5)
U.S. Vows to Fight Distillation Attacks
The U.S. government has committed to countering Chinese "distillation attacks," which are being used to steal the proprietary capabilities of American frontier artificial intelligence (AI) models. We love a little governmental fist-shaking, but we don't think its plan will have China's AI labs shaking in their boots.
Distillation attacks, also known as model extraction attacks, upskill less capable models on the cheap by training them on the outputs of more advanced models.
Back in February, OpenAI, Google, and Anthropic each said they had been victims of distillation attacks. Anthropic said that Chinese labs had collectively generated "16 million exchanges" with Claude, across 24,000 fraudulent accounts. Google cited an attack that involved 100,000 queries to Gemini.
Last week, a memo released by the White House acknowledged the problem:
[F]oreign entities, principally based in China, are engaged in deliberate, industrial-scale campaigns to distill US frontier AI systems. Leveraging tens of thousands of proxy accounts to evade detection and using jailbreaking techniques to expose proprietary information, these coordinated campaigns systematically extract capabilities from American AI models, exploiting American expertise and innovation.
The memo also promised action. It's great to get a rapid response from the government, but when it comes to the specific actions the administration has committed to we are, sadly, underwhelmed.
According to the memo, the administration will share information about distillation attacks; enable private-sector coordination against attacks; facilitate development of best practices to "identify, mitigate, and remediate" distillation attacks; and "explore a range of measures" to hold actors accountable.
These feel an awful lot like the ineffective measures the U.S. government used throughout China's 20 years of intellectual property (IP) theft.
But wait, it's not just information sharing, coordination and best practices … We have a strongly worded letter to add to the mix as well!
Reuters reported the State Department has cabled diplomatic posts and directed them to raise "concerns over adversaries' extraction and distillation of US AI models" in their host countries. The cable also said that a "demarche request and message has been sent to Beijing." That demarche is a formal raising of concerns with the Chinese government.
In response, the Chinese Embassy in Washington said the White House's accusations of AI intellectual property theft were "pure slander." We're shocked, too.
To be fair, the Trump administration's promised actions are reasonable and will make some difference. But they won't stop Chinese distillation attacks. The technology is such a game changer that China's AI labs will be fully committed to overcoming the U.S. government's countermeasures.
The disappointing thing about the memo is that it made no mention of strengthening the most effective tool the U.S. government has: semiconductor export restrictions.
There is already evidence that chip export restrictions are hampering the development of Chinese models. The release of Chinese AI company DeepSeek's latest V4 model was significantly delayed because it unsuccessfully tried to train the model on Huawei's Ascend processor. DeepSeek eventually reverted to using chips from American company Nvidia to train the model. This was possible despite restrictions because Chinese companies do have access to older chips, plus … export restrictions have been leaky. DeepSeek is using Huawei chips to actually run the model to answer queries, a process known as inference.
Additionally, a Chinese tech blogger reported that "constraints on computing power and cash" is why DeepSeek V4 is a text-only model rather than being multimodal. In its V4 technical report, DeepSeek itself says the model "trails state-of-the-art frontier models by approximately three to six months."
Huawei's Ascend can theoretically be produced by Chinese firms, but historically the bulk of Ascend production has occurred in Taiwan in violation of sanctions. Ascend chips are not as performant as Nvidia's leading chips, and, at least so far, Chinese firms cannot produce as many chips as Nvidia.
Suffice it to say that export controls are complementary to measures that counter distillation attacks. They must be part of the solution to maintaining America's AI advantage. Leading AI firms have also argued for stronger export controls. It was counterproductive to loosen controls when access to chips is the only structural advantage that the U.S. has in the AI technology race.
There may be good political reasons export restrictions were not mentioned in the White House's memo. In just over two weeks, President Trump is scheduled to meet Chinese President Xi Jinping. A blow-up over chips could upset that meeting.
Still, China has spent the past 20 years pillaging intellectual property from advanced economies, using a comprehensive range of techniques that covered the gamut from economic inducements to cyber espionage.
It would be an absolute tragedy if key technologies for the next 20 years were stolen as well.
Good News, Everyone! Chinese Hackers Adopt Botnets
Chinese threat actors are moving en masse to using botnets of compromised smart devices to facilitate their operations. This makes a network defender's job more difficult but presents opportunities for government disruption.
On April 23, the U.K.'s National Cyber Security Centre (NCSC) and a host of international cybersecurity authorities jointly released an advisory detailing a "major shift" in the way Chinese cyber actors are operating. These actors have shifted from rolling their own individual infrastructure to using large-scale networks that are sometimes managed by third parties. The networks are made up primarily of compromised small office, home office (SOHO) routers and Internet of Things devices. The NCSC refers to them as "covert networks." They are commonly known as botnets.
The report says these networks are used for all aspects of a cyber operation, including reconnaissance, malware delivery, command and control, data exfiltration, and deniable internet browsing such as researching exploitation techniques.
The covert networks are a low-cost, low-risk way to disguise the origin of malicious activity. Multiple covert networks have been created, they are constantly being updated, and any one network could be used by multiple actors. The NCSC believes "the majority of China-nexus threat actors" are using them.
Leveraging botnet-based networks isn't a new idea in the world of state-sponsored espionage. Russian agencies have a very long history of doing exactly that. The FSB ran the Snake malware network for 20 years from 2003, and the GRU created the botnets known as VPNFilter and Cyclops Blink.
It does, however, appear that China's equivalents are commercial endeavors rather than being created and run by a state intelligence organization.
Take the Raptor Train botnet, for example. In 2024, the U.S. announced a court-authorized takedown of the Chinese botnet that Lumen Technologies had analyzed and named. That botnet was formed in 2020, contained more than 60,000 devices at its peak, and had, over time, compromised more than 200,000 devices including SOHO routers, DVRs, and IP cameras. The botnet was run by the hacking group Flax Typhoon, which the U.S. government linked to a Beijing-based cybersecurity firm, Integrity Technology Group.
Rather than being a top-down mandate from the Ministry of State Security (MSS) or Ministry of Public Security (MPS), the shift toward using botnets was likely "led by market forces," Dakota Cary, a China-focused consultant at SentinelOne, told Seriously Risky Business.
Eugenio Benincasa, who authored various reports into the Chinese cyber espionage ecosystem, told SRB the system is both "competitive and collaborative." He believes the companies that build these covert networks could sell them to multiple customers.
Additionally, the provincial arms of the MSS and MPS often work "semi-independently," and Benincasa pointed to the i-Soon data leak as showing a single firm "working with dozens of [MPS or MSS] bureaus across more than 30 provinces." This fragmentation makes it less likely that the adoption of covert networks is the result of centralized direction.
A 2024 Mandiant report described the beginning of this shift. It detailed what it called a "growing trend among China-nexus cyber espionage" toward covert networks. At the time, Mandiant thought the trend was motivated by "rais[ing] the cost of defending an enterprise's network and shift[ing] the advantage toward espionage operators by evading detection and complicating attribution."
From a government perspective, botnets actually present a disruption opportunity, one that the U.S. has done a half-decent job taking advantage of. In 2024, in addition to Raptor Train, it also disrupted the Chinese KV botnet and the criminal "911 S5" service.
In March this year, the Justice Department announced that it had disrupted four distributed denial-of-service botnets.
So, although these networks have proven worth to adversaries, they are also a point of vulnerability. Adversaries are on a treadmill that involves constantly maintaining, developing, and renewing these networks.
The U.S. disruption track record is, as we said, half decent. But we'd like to see a constant drumbeat of takedowns. Authorities need to increase their pace.
Three Reasons to Be Cheerful This Week:
- U.S. launches scam crackdown: The U.S. government announced a multipronged action that targets the entire life cycle of scam operations. These actions include levying criminal charges against two Chinese nationals, sanctions targeting scammers in Cambodia, disrupting recruitment, and "restraining" cryptocurrency to prevent it from being moved. Chainalysis has further coverage.
- Alleged Silk Typhoon hacker extradited to the U.S.: Xu Zewei, a Chinese national has been extradited to the U.S. from Italy, where Xu and his wife were arrested while vacationing in Milan. The U.S. alleges Xu is a member of the Chinese hacking group known as Silk Typhoon (previously Hafnium) and was involved in the 2021 mass exploitation of Microsoft Exchange servers, among other things. The Record has further coverage.
- Another Scattered Spider arrest: A 19-year-old dual U.S. and Estonian citizen has been arrested and is facing charges related to being a member of the Scattered Spider cybercrime group. The Chicago Tribune has further details based on court records that were temporarily unsealed.
Risky Biz Talks
In our latest "Between Two Nerds" discussion, Tom Uren and The Grugq discuss what the North Korean hack of Drift can tell us about the future of hacking.
From Risky Bulletin:
U.K. NCSC blasts SOC metrics: The U.K.'s cybersecurity agency has advised public and private organizations against relying too much on bad metrics to evaluate the efficiency of their security operations centers (SOCs).
Officials say bad metrics incentivize SOC teams to be careless about their jobs and rush through tickets and detections rather than be dedicated to protecting their networks.
While metrics can be used for other information technology departments to evaluate their effectiveness, the true value of a SOC team comes from insight and not speed or quantity; hence, SOC teams should not be treated as any other department that needs to be optimized.
New fingerprinting technique can track Tor users: Firefox and Tor Browser users are advised to install the latest security patches to address a bug that can allow threat actors to track them across the internet.
The bug works in normal browsing mode, in private browsing windows, and, in the case of Tor, across different Tor sessions.
The issue, found by the team at Fingerprint, resides in IndexedDB, a Firefox API that allows websites to store data inside a user's browser for future visits.
The bug allows a threat actor to create an IndexedDB database on the user's browser—when they visit a malicious website or see a malicious ad. The bug itself is that IndexedDB will return the contents of that database in the same order every time, which creates a universal fingerprint ideal for tracking.
There are now SIM-farm-as-a-service providers: An ugly-looking web panel has been linked to 94 SIM farms located across 17 countries around the globe.
ProxySmart, as the panel is called, is among the first SIM-farm-as-a-service providers observed in cybercrime underground circles.
According to security firm Infrawatch, the panel was developed by a group operating out of Belarus. Infrawatch describes the group as "individuals with long-running involvement in SIM farm and mobile proxy operations."
