Anthropic defends AI models from capability theft
Summary and headline written by AI from the source article. How we work
Anthropic is strengthening its defenses against “distillation” attacks, where companies covertly extract capabilities from its AI models, like Claude, and replicate them in competing systems.
These attacks, often carried out using networks of fake accounts, allow other firms to rapidly develop AI without bearing the full research and safety costs. Anthropic identified that Zhipu used distilled capabilities from US models to create GLM-5.3, now considered the most cyber-capable open-weights model.
The company is concerned that these illicitly copied capabilities could end up in foreign military or surveillance systems, citing an instance of a user linked to China’s People’s Liberation Army attempting to analyze CCTV footage using a distilled model. To counter this, Anthropic implemented new measures focused on preventing access to Claude’s reasoning process, which is more valuable to distillers than the final output alone.
Initial defenses, like summarizing Claude’s reasoning, proved insufficient, prompting the development of a two-stage classifier system, similar to those used for preventing harmful outputs, that identifies and blocks extraction attempts. They also restricted editing of Claude’s context for new accounts and limited access to reasoning transcripts. Anthropic reports a 90 to 99 percent reduction in the amount of reasoning data harvested by distillers thanks to these efforts, and has banned tens of thousands of accounts attempting distillation.
While these defenses have required some adjustments for legitimate users and involve costs for both Anthropic and its customers, the company believes the benefits, including national security and responsible AI development, outweigh the drawbacks. They are sharing their methods to encourage other AI developers and platforms to adopt similar protections.
