When Does Text Inform? Benchmarking Information-Theoretic Metrics for Multimodal Time-Series Forecasting — Paper

Abstract

Multimodal forecasting models that combine time series with text annotations promise richer prediction through textual context, but how do we know whether a text annotation meaningfully contributes to the forecasters prediction? This is an information-theoretic question, but to evaluate whether information-theoretic metrics can reliably measure the predictive value an annotation provides, a ground truth benchmark is needed, and none currently exist. We create a synthetic time series signal with annotations in three categories: semantically correct, incorrect, and irrelevant. Because the data generation process is fully controlled, ground-truth information content is known exactly, enabling principled evaluation of six complementary mutual information estimators (KSG, MINE, InfoNCE, CCA, PID and V-information). We show that all six estimators identify correct annotations as most informative, and are able to audit the quality of mixed text corpora, choosing the annotations that result in the best downstream forecasting results without the need for model training. Our benchmark identifies limitations of each estimator, and these are validated on seven real-world datasets, which show how estimator performance differs on weak signals. Finally, we establish practical rules for implementing these metrics for annotation auditing and fusion selection.

Authors 2

Emma Andrews, Gianmarco Mengaldo

Specification

Official page: arxiv.org/abs/2609.11282
Source:arXiv (Atom API + RSS)T1observed 1 h agohigh
Arxiv announce type: new
Source:arXiv (Atom API + RSS)T1observed 1 h agohigh
arXiv id: 2609.11282
Source:arXiv (Atom API + RSS)T1observed 1 h agohigh
Categories: cs.AI, cs.IT, math.IT
Source:arXiv (Atom API + RSS)T1observed 1 h agohigh
PDF: arxiv.org/pdf/2609.11282
Source:arXiv (Atom API + RSS)T1observed 1 h agohigh
Primary category: cs.AI
Source:arXiv (Atom API + RSS)T1observed 1 h agohigh
Published: 12 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 1 h agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

9

Source tiers

T19

Freshest observation

1 h ago

Conflicts

None

No models linked to this paper yet.

Authors: Emma Andrews, Gianmarco Mengaldo

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history · Official page

1 claims · 1 propertiesShow all properties

Official pageofficial_url1

Claim history for Official page
Value	Valid from → to	Status	Source	Confidence	Extractor
https://arxiv.org/abs/2609.11282	12 Sept 2026 04:12 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

12 Sept 2026
New paperPaperWhen Does Text Inform? Benchmarking Information-Theoretic Metrics for Multimodal Time-Series Forecasting
New paper: When Does Text Inform? Benchmarking Information-Theoretic Metrics for Multimodal Time-Series Forecasting
arxiv
rss.arxiv.org

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.AI	feed	T1· Official	1 h ago	2

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.