PaperActive

Negative Self-Distillation: Learning to Reason by Avoiding Flaws

Updated 3 h ago · first seen 11 Sept 2026

paper_01M294FR8NZQXY4Y4VP1WPXVNW

Published: 11 Sept 2026; T1 · 3 h ago
arXiv: 2609.11699; T1 · 3 h ago
Category: cs.CL; T1 · 3 h ago

Abstract

On-Policy Self-Distillation (OPSD) has emerged as a popular paradigm for large language model (LLM) self-improvement, allowing models to act as their own teachers by leveraging privileged information such as ground-truth solutions. However, recent findings indicate that OPSD can severely degrade the performance of LLMs on complex reasoning tasks: By forcing the student to imitate an artificially confident reasoning trace conditioned on privileged information, OPSD inadvertently suppresses expressions of uncertainty and penalizes the exploratory, self-corrective behaviors required to solve challenging problems. To address this, we introduce Negative Self-Distillation (NSD), a new framework that optimizes LLMs by diverging from flawed reasoning rather than imitating privileged solutions. Instead of relying on ground-truth answers or external supervision, NSD uses the model itself to generate a question-specific negative condition (eg, acting as a ``careless reasoner'') and pushes the student's distribution away from this self-generated negative teacher. Naively applying unlearning objectives to achieve this divergence is problematic, as flawed reasoning tokens are confounded with basic linguistic tokens; indiscriminately penalizing both risks catastrophically degrading the model's foundational language capabilities. We resolve this by designing a dynamic gating mechanism that automatically identifies and isolates reasoning-critical tokens, ensuring gradient updates target only behavioral flaws while preserving the model's linguistic priors. Empirically, NSD consistently outperforms OPSD and other label-free, self-bootstrapping reinforcement learning (RL) baselines.

Authors 6

Rongcan Pei, Zhepei Wei, Shuyao Xu, Xinyu Zhu, Wei-Lin Chen, Yu Meng

Specification

Official page: arxiv.org/abs/2609.11699
Source:arXiv (Atom API + RSS)T1observed 3 h agohigh
Arxiv announce type: new
Source:arXiv (Atom API + RSS)T1observed 3 h agohigh
arXiv id: 2609.11699
Source:arXiv (Atom API + RSS)T1observed 3 h agohigh
Categories: cs.CL, cs.LG
Source:arXiv (Atom API + RSS)T1observed 3 h agohigh
Github repo: Prongcan/NSD
Source:Hugging Face Hub (public pages, model cards, papers)T2observed 3 h agomedium
Hf paper url: https://huggingface.co/papers/2609.11699
Source:Hugging Face Hub (public pages, model cards, papers)T2observed 3 h agomedium
Github stars: 4
Source:Hugging Face Hub (public pages, model cards, papers)T2observed 3 h agomedium
Hf comments: 1
Source:Hugging Face Hub (public pages, model cards, papers)T2observed 3 h agomedium
Upvotes: 4
Source:Hugging Face Hub (public pages, model cards, papers)T2observed 3 h agomedium
PDF: arxiv.org/pdf/2609.11699
Source:arXiv (Atom API + RSS)T1observed 3 h agohigh
Primary category: cs.CL
Source:arXiv (Atom API + RSS)T1observed 3 h agohigh
Published: 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 3 h agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

Source tiers

T1T29 / 5

Freshest observation

3 h ago

Conflicts

2 flagged

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history

17 claims · 14 properties2 conflicting

Official pageofficial_url1

Claim history for Official page
Value	Valid from → to	Status	Source	Confidence	Extractor
https://arxiv.org/abs/2609.11699	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Abstractabstract1

Claim history for Abstract
Value	Valid from → to	Status	Source	Confidence	Extractor
On-Policy Self-Distillation (OPSD) has emerged as a popular paradigm for large language model (LLM) self-improvement, allowing models to act as their own teachers by leveraging privileged information such as ground-truth solutions. However, recent findings indicate that OPSD can severely degrade the performance of LLMs on complex reasoning tasks: By forcing the student to imitate an artificially confident reasoning trace conditioned on privileged information, OPSD inadvertently suppresses expressions of uncertainty and penalizes the exploratory, self-corrective behaviors required to solve challenging problems. To address this, we introduce Negative Self-Distillation (NSD), a new framework that optimizes LLMs by diverging from flawed reasoning rather than imitating privileged solutions. Instead of relying on ground-truth answers or external supervision, NSD uses the model itself to generate a question-specific negative condition (eg, acting as a ``careless reasoner'') and pushes the student's distribution away from this self-generated negative teacher. Naively applying unlearning objectives to achieve this divergence is problematic, as flawed reasoning tokens are confounded with basic linguistic tokens; indiscriminately penalizing both risks catastrophically degrading the model's foundational language capabilities. We resolve this by designing a dynamic gating mechanism that automatically identifies and isolates reasoning-critical tokens, ensuring gradient updates target only behavioral flaws while preserving the model's linguistic priors. Empirically, NSD consistently outperforms OPSD and other label-free, self-bootstrapping reinforcement learning (RL) baselines.	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Arxiv announce typearxiv_announce_type2

Claim history for Arxiv announce type
Value	Valid from → to	Status	Source	Confidence	Extractor
new	11 Sept 2026 21:03 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic
cross	11 Sept 2026 21:02 UTC → 11 Sept 2026 21:03 UTC	superseded	arXiv (Atom API + RSS)T1	high	deterministic

arXiv idarxiv_id1

Claim history for arXiv id
Value	Valid from → to	Status	Source	Confidence	Extractor
2609.11699	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Authorsauthors1

Claim history for Authors
Value	Valid from → to	Status	Source	Confidence	Extractor
Rongcan Pei, Zhepei Wei, Shuyao Xu, Xinyu Zhu, Wei-Lin Chen, Yu Meng	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Categoriescategories1

Claim history for Categories
Value	Valid from → to	Status	Source	Confidence	Extractor
cs.CL, cs.LG	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Github repogithub_repo1

Claim history for Github repo
Value	Valid from → to	Status	Source	Confidence	Extractor
Prongcan/NSD	11 Sept 2026 21:10 UTC → current	current	Hugging Face Hub (public pages, model cards, papers)T2	medium	deterministic

Hf paper urlhf_paper_url1

Claim history for Hf paper url
Value	Valid from → to	Status	Source	Confidence	Extractor
https://huggingface.co/papers/2609.11699	11 Sept 2026 21:10 UTC → current	current	Hugging Face Hub (public pages, model cards, papers)T2	medium	deterministic

Github starsmetric.github_stars1

Claim history for Github stars
Value	Valid from → to	Status	Source	Confidence	Extractor
4	11 Sept 2026 21:10 UTC → current	current	Hugging Face Hub (public pages, model cards, papers)T2	medium	deterministic

Hf commentsmetric.hf_comments1

Claim history for Hf comments
Value	Valid from → to	Status	Source	Confidence	Extractor
1	11 Sept 2026 21:10 UTC → current	current	Hugging Face Hub (public pages, model cards, papers)T2	medium	deterministic

Upvotesmetric.upvotes1

Claim history for Upvotes
Value	Valid from → to	Status	Source	Confidence	Extractor
4	11 Sept 2026 21:10 UTC → current	current	Hugging Face Hub (public pages, model cards, papers)T2	medium	deterministic

PDFpdf_url1

Claim history for PDF
Value	Valid from → to	Status	Source	Confidence	Extractor
https://arxiv.org/pdf/2609.11699	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Primary categoryprimary_category1

Claim history for Primary category
Value	Valid from → to	Status	Source	Confidence	Extractor
cs.CL	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Publishedpublished_at3conflicting claims

Claim history for Published
Value	Valid from → to	Status	Source	Confidence	Extractor
10 Sept 2026	11 Sept 2026 21:12 UTC → current	conflicting	Hugging Face Hub (public pages, model cards, papers)T2	medium	deterministic
10 Sept 2026	11 Sept 2026 21:10 UTC → current	conflicting	Hugging Face Hub (public pages, model cards, papers)T2	medium	deterministic
11 Sept 2026	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	conflicted	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.CL	feed	T1· Official	3 h ago	1
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.LG	feed	T1· Official	3 h ago	1
Hugging Face Hub (public pages, model cards, papers)	huggingface.co/papers	listing	T2· Quality secondary	3 h ago	2

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.