Adversarial Distillation: OpenAI Cuts Attack Success Rate from 54% to 0.2%
New OpenAI research reveals that distillation can replicate powerful AI capabilities, but safety guardrails are not automatically inherited. They published a three-layer defense system and concrete test data, and also sparked discussion about model governance.
OpenAI today published research on adversarial distillation. The core message is straightforward: distillation can replicate powerful AI capabilities, but it does not inherit safety guardrails. An attacker can use the outputs of one model to train another model, producing a version with similar capabilities but not bound by the original model's safety policies. For example, if what is distilled is cyberattack capability, that new model may help carry out attacks that the original model refused to assist with.
Distillation is a common technique—a model learns from data generated by another model. The problem is that when the data comes from a model's internal reasoning process (chain-of-thought), the student model may learn more than the surface answer. OpenAI's internal experiments showed that student models trained on exact reasoning traces outperformed models trained only on reasoning summaries on cybersecurity benchmarks. In other words, the complete reasoning process is a key channel for capability transfer.

OpenAI therefore focused its defenses on protecting "protected reasoning." They used three layers of request-level defenses: model behavior training (teaching the model to recognize extraction intent and refuse), input/output classifiers (detecting reasoning leakage, including tool-call parameters), and API-level protections (limiting reasoning reuse across sessions and across models). There is also campaign-level detection: using account signals to find coordinated extraction activity, then banning the account networks. These defenses reinforce one another—attacks found in investigations are used to train more robust models.
How effective is it? In an adaptive black-box evaluation, attackers tried to extract reasoning traces generated by previous requests from GPT-6.2 Astra. The average attack success rate fell from 54.0% under GPT-6 Astra's original protections to 0.2%; for extracting reasoning traces from current requests, the success rate fell from 22.9% to 0.9%. The numbers are impressive. But OpenAI also acknowledges that static and adaptive evaluations need to be combined, and that defenses are not one-time; they require continuous adversarial testing.

There is another easily overlooked point: distillation is a national security issue. OpenAI mentions that earlier this year there were activities indicating that DeepSeek was trying to distill U.S. frontier models; in September they reported disrupting a campaign to extract reasoning traces, with a core participant tied to Moonshot AI (the developer of Kimi). They support legislation to give the U.S. government more tools to identify and sanction such behavior.
Interestingly, reactions online are divided. Some joked: "No one is distilling your model, don't worry." Others felt that OpenAI itself trained its models on all of humanity's knowledge: "You distilled all of humanity, and now you're saying others can't distill you?" Still others worried about the practical impact: "In the end, everyone may face tighter rate limits, not just those doing distillation." These voices have a point.
At the end, OpenAI offers several recommendations to other model developers: protect private reasoning across interactions, check assistant messages and tool-call parameters for leakage, keep attack logs as regression tests, combine request-level and campaign-level defenses, and coordinate security requirements with deployment partners. These recommendations are concrete and worth considering.
The cat-and-mouse game of adversarial distillation will continue. OpenAI says it shared these findings to raise the industry's overall protection baseline, but ironically, each round of defense upgrades may move ordinary users a little farther from the model's internals.
Reference: OpenAI official blog "Combatting adversarial distillation" (https://openai.com/index/combatting-adversarial-distillation/)
发布时间: 2026-11-21 05:35