Counterfactual Marginalisation: Framework for Evaluating Robustness to Nuisance Variables — Paper

Abstract

Machine learning models can achieve strong test performance while relying on demographic or acquisition-related shortcuts. We propose counterfactual (CF) marginalisation as a test-time evaluation procedure for assessing robustness of classification models to such variables. Given a CF image generator, we intervene on nuisance parent variables such as age or sex, generate CF versions of each test image, and average predictions over a target intervention distribution. This produces intervention-aware predictions that marginalise demographic effects while preserving patient-specific latent information. We use these predictions to define metrics for CF risk, calibration, stability and worst-case sensitivity. We demonstrate this framework's utility for quantitative robustness evaluation.

Authors 4

Yasin Ibrahim, Hermione Warr, Robin J. Evans, Konstantinos Kamnitsas

Specification

Official page: arxiv.org/abs/2609.10778
Source:arXiv (Atom API + RSS)T1observed 34 min agohigh
Arxiv announce type: new
Source:arXiv (Atom API + RSS)T1observed 34 min agohigh
arXiv id: 2609.10778
Source:arXiv (Atom API + RSS)T1observed 34 min agohigh
Categories: cs.LG, cs.AI
Source:arXiv (Atom API + RSS)T1observed 34 min agohigh
PDF: arxiv.org/pdf/2609.10778
Source:arXiv (Atom API + RSS)T1observed 34 min agohigh
Primary category: cs.LG
Source:arXiv (Atom API + RSS)T1observed 34 min agohigh
Published: 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 34 min agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

9

Source tiers

T19

Freshest observation

34 min ago

Conflicts

None

No models linked to this paper yet.

Authors: Yasin Ibrahim, Hermione Warr, Robin J. Evans

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history · PDF

1 claims · 1 propertiesShow all properties

PDFpdf_url1

Claim history for PDF
Value	Valid from → to	Status	Source	Confidence	Extractor
https://arxiv.org/pdf/2609.10778	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
New paperPaperCounterfactual Marginalisation: Framework for Evaluating Robustness to Nuisance Variables
New paper: Counterfactual Marginalisation: Framework for Evaluating Robustness to Nuisance Variables
arxiv
rss.arxiv.org

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.LG	feed	T1· Official	34 min ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.