Chipping Away at Chinese AI Risks
Chipping Away at Chinese AI Risks
The Trump administration has been trying to address two separate artificial intelligence (AI)-related risks in recent months: the specific risk to U.S. national security from China developing powerful AI and the global risk that powerful hacking machines will be available to all and sundry. A proposed bill suggests a sensible way for the U.S. to chip away at both of these risks, at least a little.
The Collaboration on Adversarial Threats and Security Risks Act proposes safe harbor provisions for AI companies so that they can share information related to AI-specific security risks. The idea here is to encourage frontier labs to work together to counter threats from Chinese AI labs, particularly what they describe as intellectual property (IP) theft via distillation. Without this bill, sharing this kind of information could fall afoul of antitrust legislation that prohibits collusion.
We know the frontier labs can already detect distillation because they're always complaining about it after the fact. The idea behind this bill is that more permissive information sharing would let them respond more quickly and disrupt at least some distillation campaigns.
A bill like this isn't necessarily the most effective way for the U.S. government to prevent or deter distillation attacks. But the more direct approaches, like adding Chinese companies to the Entity List, come with problems.
American companies, for example, are increasingly using open-weight AI models, the best of which are produced by the same Chinese labs that are carrying out large-scale distillation campaigns.
Indeed, after Treasury Secretary Scott Bessent issued a statement saying that "Entity List designations will be on the table," a tech industry coalition of 76 companies published an open letter supporting the role of open-weight AI models in the technology mix.
So, the consensus seems to be that punitive action targeting Chinese labs could be a bonus for the American frontier AI companies, but likely a negative for the U.S. economy writ large.
Really, though, the battle for AI dominance is being fought around hardware access, not the models themselves. The U.S. controls chip exports, and there is some evidence that these controls are having an impact.
When AI-focused journalists Lily Ottinger and Kai Williams toured Chinese AI labs in May this year they reported:
Basically every AI researcher we talked to in China brought up the same complaint: their companies lack sufficient access to the advanced compute resources necessary to train and run AI models.
American chip export controls are imperfect, but they're biting, and reinforcing them is the most effective way for the U.S. to stay ahead.
In the areas around AI that aren't about staying ahead, we think there's room for the United States and China to cooperate.
When open-weight models get really good, we suspect neither government will be happy with everyone on the planet having access to powerful AI hacking machines. That time could be only a matter of months away, as the recently released open-weight Chinese Kimi K3 model is roughly equivalent to U.S. frontier models from about six months ago, according to a joint U.S./U.K. evaluation.
China already has strict AI regulation, but it's focused on making sure that AI does not upset the Chinese Communist Party's information control and censorship apparatus.
However, we think Chinese regulators will eventually rein in the release of very capable models. Very capable open-weight models will be used for all sorts of hacking, some of which will make the Chinese Communist Party look bad. The party will not like that kind of political risk, so cyber safety regulation is coming.
But right now there is an opportunity for the U.S. government to engage the Chinese government to encourage and shape this regulation to its advantage.
So as an overall strategy, what we've laid out above makes sense: Laws that free up frontier labs to combat distillation collectively; tightening chip controls to make sure the U.S. stays ahead of China; and engaging with Chinese regulators to prevent the spread of top-tier hacking capabilities to all and sundry.
Will this be what happens? Long-term strategic thinking is not a Trump admin forte, so we'll hope for the best, but prepare for the status quo.
Iran's Perfectly Calibrated Cyberattack
A cyberattack on Minnesota water utilities coupled with a U.S. federal government warning raises the possibility that Iranian hackers have begun targeting U.S. critical infrastructure in earnest.
This week more than 30 Minnesota community water systems were targeted by a "coordinated cyberattack," according to the state’s information technology (IT) services. Disruptions were reported in a number of cities, but the water remained safe to drink. The worst impact reported was in the city of Braham, where residents were asked to limit their consumption of water for a few hours as the city was relying on supply in a single water tower.
The attack came less than a week after U.S. federal government agencies updated a warning about the possibility of Iranian state-affiliated cyber actors targeting U.S. critical infrastructure. The warning cites incidents since March in which multiple victims across several critical infrastructure sectors had been compromised and some victims "experienced operational disruption and financial loss."
There is no formal attribution of the Minnesota attacks, but security firm Tenable says the "operational pattern" is consistent with the CyberAv3ngers, a threat actor associated with Iran's Islamic Revolutionary Guard Corp (IRGC).
In March we wrote that even if the war in Iran successfully achieved the White House's stated objectives, it could still "result in an enduring increase in Iran's capacity and appetite for cyber mayhem." Cyber operations targeting the U.S. and Israel could provide propaganda victories with very little chance of a kinetic response.
The Minnesota attacks caused minimal impact, so it would be tempting to argue that they were a flop. Given that the war is ongoing and also very unpopular, however, we think these attacks might be perfectly calibrated. They're significant enough to generate headlines and public unease and, maybe, make Americans wonder about the point of the entire war. But they're not lethal or damaging enough to ensure America bands together against a single enemy.
Our crystal ball says we can expect more of this.
Three Reasons to Be Cheerful This Week:
- U.S. visa restrictions for cybercriminals: Last week, Secretary of State Marco Rubio announced visa restrictions for cybercriminals including sextortionists and those running cyber-enabled scams. The policy applies to the " individuals responsible" for the crimes and, in some cases, their family members.
- AI security tools are a thing: This week Microsoft announced a security-focused AI model that, when coupled with its MDASH harness, is cheaper and better at vulnerability identification and remediation than current frontier models. It's not this specific announcement that makes us cheerful but, rather, what it indicates. Microsoft announcing a "world-class security" machine as a product means there will likely be a number of competitors offering a variety of similar products.
- AI works on cryptographic algorithms too: Anthropic has posted about Claude Mythos Preview being used to find flaws in two different cryptographic systems. One of the results came in a proposed post-quantum algorithm that was being evaluated by the National Institute of Standards and Technology. This is a good example of AI being used preventively to improve standards.
Shorts
North Korean Hackers Hack North Korea
The Daily NK reported last week that a group of former North Korean military intelligence operatives had hacked two of the country's banks to steal funds for their own personal enrichment.
A source told the Daily NK that the group had stolen funds from two North Korean banks, the Chosun Central Bank and the Foreign Trade Bank. The stolen funds were then converted into cryptocurrency and laundered before being converted back into cash in China. The ringleaders were cyber operations veterans formerly from North Korea's Reconnaissance General Bureau.
It looks like the outcome will be grim for the hackers. Per the Daily NK:
Officials in Pyongyang expect harsh punishment for those involved. "They used the skills the state trained them with to defend the country, and instead robbed the country’s coffers," one official said, according to the source. "This goes beyond ordinary guilt-by-association penalties. It will be hard for the entire family line to survive."
Crikey.
Risky Biz Talks
In our latest "Between Two Nerds" discussion, Tom Uren and The Grugq discuss how important people are to cyber power and whether the rise of AI is changing that.
From Risky Bulletin:
New Chinese cyber contractor identified: The cyber sleuths at Intrusion Truth have uncovered a new secretive Chinese IT company that appears to work as a cyber contractor and tool developer for Chinese state-sponsored hacking operations.
Online clues appear to suggest that Guangdong Chanming appears to have developed RedRelay (aka ORBWEAVER), an ORB network (aka proxy botnet) that was used by almost a dozen Chinese APT groups to hide the origin of their attacks.
According to Intrusion Truth, the company's customers allegedly include the Chinese People's Liberation Army and the Ministry of Public Security, which manages China's police forces.
A JSON RCE bug is about to rock the Java world: Threat actors are exploiting a vulnerability in Alibaba's Fastjson, one of the Java ecosystem's most popular libraries for working with JSON-formatted data.
Active exploitation began last week, a day after details about the security flaw were revealed by cybersecurity firm FearsOff.
The attacks, first spotted and documented by Imperva and ThreatBook, target CVE-2026-16723, a vulnerability that can enable unauthenticated remote code execution attacks against Java projects that use the Fastjson library as a component.
Western cyber agencies warn of Russian hacks of Zimbra servers: Cybersecurity and intelligence agencies from multiple Western countries have issued joint security advisories on Thursday warning of a major Russian hacking campaign that's targeting Zimbra email servers.
The attacks have been going on since last year. The zero-day, tracked as CVE-2025-66376, was patched in November but attacks have been traced back to at least July.
The zero-day itself is a stored cross-site scripting (XSS) bug that allows the attackers to load malicious code inside a Zimbra webmail client via the CSS @import feature. The malicious code would load a web tool called Ulej (Russian for Beehive) that could be used to harvest credentials, session tokens, backup multi-factor authentication codes, browser-saved passwords, and the contents of the victim’s mailbox going back 90 days.
