Revenge of Monosemanticity: Neuron Specialization as a New Form of Feature Learning in MLPs — Paper

Abstract

Understanding how neural networks learn and organize features is central to understanding their behavior. Much existing theory of feature learning has focused on the emergence of a global low-dimensional representation. We show that this picture is incomplete. In regression problems with clustered data, we demonstrate that multilayer perceptrons (MLPs) naturally develop monosemantic specialized neurons: individual neurons become strongly aligned with a specific predictive feature relevant to a particular region of the input space. Rather than learning a single global low-dimensional representation, MLPs learn a collection of local low-dimensional representations. We show that this ability to specialize gives MLPs a provable data-efficiency advantage over feature-learning methods based on a global low-dimensional representation.

Authors 4

Amirhesam Abedsoltan, Enric Boix-Adsera, Fivos Kalogiannis, Mikhail Belkin

Specification

Official page: arxiv.org/abs/2608.24007
Source:arXiv (Atom API + RSS)T1observed 4 h agohigh
Arxiv announce type: replace
Source:arXiv (Atom API + RSS)T1observed 4 h agohigh
arXiv id: 2608.24007
Source:arXiv (Atom API + RSS)T1observed 4 h agohigh
Categories: cs.LG, stat.ML
Source:arXiv (Atom API + RSS)T1observed 4 h agohigh
PDF: arxiv.org/pdf/2608.24007
Source:arXiv (Atom API + RSS)T1observed 4 h agohigh
Primary category: cs.LG
Source:arXiv (Atom API + RSS)T1observed 4 h agohigh
Published: 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 4 h agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

9

Source tiers

T19

Freshest observation

4 h ago

Conflicts

None

No models linked to this paper yet.

Authors: Amirhesam Abedsoltan, Enric Boix-Adsera, Fivos Kalogiannis

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history · Abstract

1 claims · 1 propertiesShow all properties

Abstractabstract1

Claim history for Abstract
Value	Valid from → to	Status	Source	Confidence	Extractor
Understanding how neural networks learn and organize features is central to understanding their behavior. Much existing theory of feature learning has focused on the emergence of a global low-dimensional representation. We show that this picture is incomplete. In regression problems with clustered data, we demonstrate that multilayer perceptrons (MLPs) naturally develop monosemantic specialized neurons: individual neurons become strongly aligned with a specific predictive feature relevant to a particular region of the input space. Rather than learning a single global low-dimensional representation, MLPs learn a collection of local low-dimensional representations. We show that this ability to specialize gives MLPs a provable data-efficiency advantage over feature-learning methods based on a global low-dimensional representation.	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
New paperPaperRevenge of Monosemanticity: Neuron Specialization as a New Form of Feature Learning in MLPs
New paper: Revenge of Monosemanticity: Neuron Specialization as a New Form of Feature Learning in MLPs
arxiv
rss.arxiv.org

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.LG	feed	T1· Official	2 h ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.