Reference-Based Bias Detection in LLMs via Relative Representations of Hidden States — Paper

Abstract

Existing bias auditing methods typically rely on model outputs, requiring costly benchmarks or judge models and potentially missing internal shifts that never appear in generated text. We propose a reference-based method that audits bias in hidden-state representations across related model variants, for example before and after fine-tuning. Because fine-tuning reshapes representation geometry, absolute hidden states are not directly comparable, so we encode each sentence by its similarities to a fixed set of anchor sentences, yielding relative representations in a shared comparison space. There we measure how target groups shift in their association with positive and negative attributes, a quantity we call the Representational Bias Shift $\Delta B$. Across three model families and the WildGuardMix, DecodingTrust and ToxiGen benchmarks, $\Delta B$ correlates with output-level bias change in 15 of the 18 settings we test, reaching $|r| = 0.84$ ($p < 0.001$) under full fine-tuning and becoming more model-dependent under parameter-efficient adaptation. Thresholding $\Delta B$ detects checkpoints whose bias increased with ROC AUC between $0.65$ and $0.99$, and on WildGuardMix and DecodingTrust it separates them better than a SEAT-based baseline for all three families. $\Delta B$ is also stable under changes to the anchor set, attribute sets and target templates. Our method requires no task-specific evaluation data and audits a model in about three minutes, using $3$-$50\times$ less compute than the output-level benchmarks considered here. We view it as complementary to output-based auditing rather than a replacement for it.

Authors 4

Marek Jeli\'nski, Jan Dubi\'nski, Maciej Chrabaszcz, Sebastian Cygert

Specification

Official page: arxiv.org/abs/2609.10060
Source:arXiv (Atom API + RSS)T1observed 6 h agohigh
Arxiv announce type: new
Source:arXiv (Atom API + RSS)T1observed 6 h agohigh
arXiv id: 2609.10060
Source:arXiv (Atom API + RSS)T1observed 6 h agohigh
Categories: cs.AI
Source:arXiv (Atom API + RSS)T1observed 6 h agohigh
PDF: arxiv.org/pdf/2609.10060
Source:arXiv (Atom API + RSS)T1observed 6 h agohigh
Primary category: cs.AI
Source:arXiv (Atom API + RSS)T1observed 6 h agohigh
Published: 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 6 h agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

9

Source tiers

T19

Freshest observation

6 h ago

Conflicts

None

No models linked to this paper yet.

Authors: Marek Jeli\'nski, Jan Dubi\'nski, Maciej Chrabaszcz

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history · arXiv id

1 claims · 1 propertiesShow all properties

arXiv idarxiv_id1

Claim history for arXiv id
Value	Valid from → to	Status	Source	Confidence	Extractor
2609.10060	11 Sept 2026 21:03 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
New paperPaperReference-Based Bias Detection in LLMs via Relative Representations of Hidden States
New paper: Reference-Based Bias Detection in LLMs via Relative Representations of Hidden States
arxiv
rss.arxiv.org

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.AI	feed	T1· Official	5 h ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.