Companies are aware. A recent DeepSeek paper that includes founder and CEO Liang Wenfeng as a co-author bluntly warns that agent behaviour can be “untrustworthy”.
Escaping containment isn’t a distinctly American risk, either. Moonshot AI’s Kimi K3 exploited a loophole in a sandbox during cybersecurity testing, according to US-based Frontier Security. And some Chinese AI agents have also shown the kind of deceptive and concerning trends that have raised alarm about US tools.
While Beijing has pushed back on some of the doomerism, policymakers are paying attention.
China in September released AI Safety Governance Framework 3.0, a policy document that notably also dropped in English, clearly aimed for Western audiences. It explicitly warns that AI has demonstrated a “self-accelerating trend,” and notes the question of whether this could exceed human control demands attention and vigilance.
Another cornerstone of the latest safety contention is recursive self-improvement (RSI), or AI systems that can autonomously advance their own capabilities. It’s the development path that spurred an Anthropic researcher to quit and warn that makers of the technology earnestly believe it could “kill us all” within a decade.
As Oxford China Policy Lab’s Zilan Qian has argued, China is further along in this regard than many in the West may realise, partly because its researchers describe it in different terms. It all reveals how little visibility each side has into the risks of what the other is building.
