Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs

Abstract

Large Language Models (LLMs) can memorize and reproduce sensitive, copyrighted, or otherwise undesirable training content, creating privacy, safety, and regulatory concerns. Machine unlearning offers a practical alternative to full retraining, but many existing methods apply broad or fixed parameter updates that can degrade utility and remain brittle under deployment changes such as post-training quantization, where forgotten knowledge may partially re-emerge. We propose Forgetting Only What Matters via Unlearning Layers (FOM-UL), a layer-level unlearning framework that selects transformer layers using a forget-to-retain significance score. This score identifies layers with high influence on the forget set and low sensitivity to the retain set, allowing FOM-UL to concentrate updates where they are most effective while leaving most of the model unchanged. This targeted update strategy improves the forgetting-utility trade-off and provides an empirical path toward quantization-resilient unlearning by reducing the chance that small, diffuse updates are erased by low-bit rounding. Across TOFU, KnowUnDo, and MUSE-style evaluations, FOM-UL reduces residual memorization compared with strong GA, NPO, KLD, SURE, ReLearn, and LUNAR-based baselines while preserving retain-set utility close to the vanilla model. Under 8-bit and 4-bit post-training quantization, FOM-UL maintains stronger memorization suppression and utility preservation than competing methods, and adversarial prompt evaluations show lower recovery of forgotten content. Overall, FOM-UL provides an efficient unlearning strategy that improves targeted forgetting, utility preservation, and deployment robustness without claiming formal guarantees of erasure.

Authors 3

Ravi Ranjan, Olivera Kotevska, Agoritsa Polyzou

Specification

Official page

arxiv.org/abs/2609.10439

Source:arXiv (Atom API + RSS)·T1·observed 2 h ago·high

Arxiv announce type

cross

Source:arXiv (Atom API + RSS)·T1·observed 2 h ago·high

arXiv id

2609.10439

Source:arXiv (Atom API + RSS)·T1·observed 2 h ago·high

Categories

cs.LG, cs.AI

Source:arXiv (Atom API + RSS)·T1·observed 2 h ago·high

PDF

arxiv.org/pdf/2609.10439

Source:arXiv (Atom API + RSS)·T1·observed 2 h ago·high

Primary category

cs.LG

Source:arXiv (Atom API + RSS)·T1·observed 2 h ago·high

Published

11 Sept 2026

Source:arXiv (Atom API + RSS)·T1·observed 2 h ago·high

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance