DP-Muon: Differentially Private Optimization via Matrix-Orthogonalized Momentum — Paper

Abstract

We study differentially private optimization with matrix-orthogonalized momentum. DP-Muon uses conventional global per-example clipping and one Gaussian gradient release per step; matrix updates and auxiliary updates are post-processing. Our main contribution concerns the additional mean distortion created when fresh Gaussian noise passes through a nonlinear matrix map. Conditioning on the actual adaptive history immediately before the current noise yields an exact Gaussian heat identity. For a smooth Newton-Schulz map, first-order DP-MuonBC reduces this conditional output bias from second to fourth order in the fresh noise scale, and an arbitrary-order extension has bias of order $2K+2$. We prove matrix-block stationarity bounds under global clipping, retain finite-step orthogonalization error explicitly, and give an exact criterion for improvement of the resulting upper bound. A separate inequality exposes the effect of auxiliary Adam updates. GPT-2 experiments on E2E at four privacy targets favor the reported Muon configurations over Adam baselines in test NLL.

Authors 2

Jihwan Kim, Chenglin Fan

Specification

Official page: arxiv.org/abs/2605.12994
Source:arXiv (Atom API + RSS)T1observed 5 h agohigh
Arxiv announce type: replace
Source:arXiv (Atom API + RSS)T1observed 5 h agohigh
arXiv id: 2605.12994
Source:arXiv (Atom API + RSS)T1observed 5 h agohigh
Categories: cs.LG
Source:arXiv (Atom API + RSS)T1observed 5 h agohigh
PDF: arxiv.org/pdf/2605.12994
Source:arXiv (Atom API + RSS)T1observed 5 h agohigh
Primary category: cs.LG
Source:arXiv (Atom API + RSS)T1observed 5 h agohigh
Published: 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 5 h agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

9

Source tiers

T19

Freshest observation

5 h ago

Conflicts

None

No models linked to this paper yet.

Authors: Jihwan Kim, Chenglin Fan

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history · Primary category

1 claims · 1 propertiesShow all properties

Primary categoryprimary_category1

Claim history for Primary category
Value	Valid from → to	Status	Source	Confidence	Extractor
cs.LG	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
New paperPaperDP-Muon: Differentially Private Optimization via Matrix-Orthogonalized Momentum
New paper: DP-Muon: Differentially Private Optimization via Matrix-Orthogonalized Momentum
arxiv
rss.arxiv.org

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.LG	feed	T1· Official	4 h ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.