ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs
Published 16 Sept 2026arXiv:2609.15635
Updated 7 h ago · first seen 14 Sept 2026
paper_01M2JJGQ7RM7Q4TTY3J1RFA9PP
Abstract
A radiology report can already answer a clinical question, so it is hard to tell whether a vision-language model also uses the image. ModaLens, a paired image-swap audit, measures how report availability changes image sensitivity: MedGemma-27B on 3,199 paired MIMIC-CXR cases from 293 patients, all 14 questions per case (13 finding-specific and one composite), each image replaced by one from another study, usually of the same patient, with question and report fixed. Under an explicit answer instruction, the model's generated answer changes on 4.26 percent of trials with the report and 20.94 percent without it, a paired increase of 16.7 points (patient-clustered 95 percent CI 15.6 to 17.7), so report availability reduces image-swap sensitivity under this protocol; the original prompt with a lowercase first-token readout gives 4.70 percent against 17.07 percent, and substitutions also move continuous answer scores where the binary prediction does not change. The labels are derived from reports, which limits conclusions about visual correctness; the direction replicates in two further model lineages. Code, the exact prompts and a run record for every number are at https://github.com/criticaldata/MODALENS.
Organizations
Organizations 0
No organization stated. arXiv metadata does not carry affiliations; an organization is linked only when a model card or lab page cites the paper.
Models
Models introduced or described 0
Inbound described_by relations from model cards and documentation.
No model links this paper yet
Datasets
Datasets used 0
No dataset relation recorded.
Benchmarks
Benchmarks used 0
No benchmark relation recorded.
Code
Repositories & frameworks 0
No repository linked.
Timeline
Timeline 6
ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs: arxiv announce type changed from new to cross
Arxiv announce typenew→crossarxivModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs: published at changed from 2026-09-15T04:00:00+00:00 to 2026-09-16T04:00:00+00:00
Published15 Sept 2026→16 Sept 2026arxivModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs: arxiv announce type changed from cross to new
Arxiv announce typecross→newarxivModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs: published at changed from 2026-09-14T00:00:00+00:00 to 2026-09-15T04:00:00+00:00
Published14 Sept 2026→15 Sept 2026arxivModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs: authors changed from ["Anqi Peter Li", "Ashley Zhang", "Felipe Ocampo Osorio",… to ["Anqi Peter Li", "Ashley Zhang", "Felipe Ocampo Osorio",…
AuthorsAnqi Peter Li, Ashley Zhang, Felipe Ocampo Osorio, Kushul Reddy Palakala, Leo Anthony Celi, Maximin Lange, Quang Bui, Rafi Al Attrach, Sahil Kapadia, Sebastián Andrés Cajas Ordóñez, Xinyue Zhang, Zakaria Laouabdia Sellami→Anqi Peter Li, Ashley Zhang, Felipe Ocampo Osorio, Kushul Reddy Palakala, Leo Anthony Celi, Maximin Lange, Quang Bui, Rafi Al Attrach, Sahil Kapadia, Sebasti\'an Andr\'es Cajas Ord\'o\~nez, Xinyue Zhang, Zakaria Laouabdia SellamiarxivNew paper: ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs
huggingface
Sources
Sources 3
Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.