The information geometry of large language models is shared, learned, and controllable — Paper

Abstract

Large language models learn similar behaviours, yet it remains unclear what structure they share or how to change one behaviour without disturbing others. The Fisher-Rao geometry of next-token probabilities connects these questions: behaviour determines this geometry up to output-preserving symmetries, whereas activation geometry depends on coordinates. Across transformer, state-space and recurrent models, output geometries agree more strongly than activation geometries, and shared geometry supports semantic-category transfer. Agreement with human word choices increases with predictive accuracy, scale and training, and improves further after model-only calibration. Token probabilities and read-out geometry jointly predict the spectrum and its effective dimension. Controlled language assignments show that geometry follows the language law across architectures. Pretraining corpus statistics predict held-out fact acquisition without recalibration, while randomised experiments show that deeper evidence substantially delays acquisition across every tested architecture and evidence construction. Finally, the geometry prescribes minimum-disturbance local interventions, predicts their relative cost, and supports reusable control: updates learned on donor prompts transfer to unseen prompts while better preserving behaviour on reference prompts than Euclidean control. The same geometric correction improves steering, editing, attribution, dictionary learning and fine-tuning.

Authors 1

Dario Picozzi

Specification

Official page: arxiv.org/abs/2609.11063
Source:arXiv (Atom API + RSS)T1observed 15 min agohigh
Arxiv announce type: cross
Source:arXiv (Atom API + RSS)T1observed 15 min agohigh
arXiv id: 2609.11063
Source:arXiv (Atom API + RSS)T1observed 15 min agohigh
Categories: cs.LG, cs.CL
Source:arXiv (Atom API + RSS)T1observed 15 min agohigh
PDF: arxiv.org/pdf/2609.11063
Source:arXiv (Atom API + RSS)T1observed 15 min agohigh
Primary category: cs.LG
Source:arXiv (Atom API + RSS)T1observed 15 min agohigh
Published: 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 15 min agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

9

Source tiers

T19

Freshest observation

15 min ago

Conflicts

None

No models linked to this paper yet.

Authors: Dario Picozzi

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history

10 claims · 9 properties

Official pageofficial_url1

Claim history for Official page
Value	Valid from → to	Status	Source	Confidence	Extractor
https://arxiv.org/abs/2609.11063	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Abstractabstract1

Claim history for Abstract
Value	Valid from → to	Status	Source	Confidence	Extractor
Large language models learn similar behaviours, yet it remains unclear what structure they share or how to change one behaviour without disturbing others. The Fisher-Rao geometry of next-token probabilities connects these questions: behaviour determines this geometry up to output-preserving symmetries, whereas activation geometry depends on coordinates. Across transformer, state-space and recurrent models, output geometries agree more strongly than activation geometries, and shared geometry supports semantic-category transfer. Agreement with human word choices increases with predictive accuracy, scale and training, and improves further after model-only calibration. Token probabilities and read-out geometry jointly predict the spectrum and its effective dimension. Controlled language assignments show that geometry follows the language law across architectures. Pretraining corpus statistics predict held-out fact acquisition without recalibration, while randomised experiments show that deeper evidence substantially delays acquisition across every tested architecture and evidence construction. Finally, the geometry prescribes minimum-disturbance local interventions, predicts their relative cost, and supports reusable control: updates learned on donor prompts transfer to unseen prompts while better preserving behaviour on reference prompts than Euclidean control. The same geometric correction improves steering, editing, attribution, dictionary learning and fine-tuning.	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Arxiv announce typearxiv_announce_type2

Claim history for Arxiv announce type
Value	Valid from → to	Status	Source	Confidence	Extractor
cross	11 Sept 2026 21:03 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic
new	11 Sept 2026 21:02 UTC → 11 Sept 2026 21:03 UTC	superseded	arXiv (Atom API + RSS)T1	high	deterministic

arXiv idarxiv_id1

Claim history for arXiv id
Value	Valid from → to	Status	Source	Confidence	Extractor
2609.11063	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Authorsauthors1

Claim history for Authors
Value	Valid from → to	Status	Source	Confidence	Extractor
Dario Picozzi	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Categoriescategories1

Claim history for Categories
Value	Valid from → to	Status	Source	Confidence	Extractor
cs.LG, cs.CL	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

PDFpdf_url1

Claim history for PDF
Value	Valid from → to	Status	Source	Confidence	Extractor
https://arxiv.org/pdf/2609.11063	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Primary categoryprimary_category1

Claim history for Primary category
Value	Valid from → to	Status	Source	Confidence	Extractor
cs.LG	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Publishedpublished_at1

Claim history for Published
Value	Valid from → to	Status	Source	Confidence	Extractor
11 Sept 2026	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
Property changedPaperThe information geometry of large language models is shared, learned, and controllable
The information geometry of large language models is shared, learned, and controllable: arxiv announce type changed from new to cross
Arxiv announce typenew→crossarxiv
rss.arxiv.org
11 Sept 2026
New paperPaperThe information geometry of large language models is shared, learned, and controllable
New paper: The information geometry of large language models is shared, learned, and controllable
arxiv
rss.arxiv.org

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.CL	feed	T1· Official	15 min ago	1
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.LG	feed	T1· Official	15 min ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.