Non-Stationarity Breaks Permutation Surrogates in Multi-Agent Reinforcement Learning: Diagnosis and Remedies
Updated 51 min ago · first seen 11 Sept 2026
paper_01M294GNWH5V6TNVDY2XHBTGMX
- Published
- 11 Sept 2026
- T1 · 51 min ago
- arXiv
- 2604.23716
- T1 · 51 min ago
- Category
- cs.AI
- T1 · 51 min ago
Abstract
Reporting guidance for information-theoretic measures is rarely tested against ground truth. We test one guardrail in two multi-agent reinforcement learning games, a social dilemma and a coordination race, where directed influence between selected agent pairs is zero by construction, over 100 seeds. Omitting one precondition, exclusion of the non-stationary training transient, gives false-positive rates of 100.00% and 99.95%: agents annealing exploration independently, in runs that never met, are flagged as influencing one another. Excluding the transient reaches 3.0% in the social dilemma but 11.8% in the coordination game, which stationarity tests explain: 95.7% of social-dilemma series are stationary afterwards against 56.8% of coordination series. So the non-stationarity must be treated, and exclusion is neither the only way nor sufficient. What we recommend instead changes the null model rather than the data: permuting the source within blocks of training time reaches 5.25% and 5.50%, the only one of four constructions at the size of the test in both games, leaving series, statistic and estimand untouched. Titrating injected links of known strength in both games shows it is also the most sensitive of the three, detecting 89.0% in the coordination game where conditioning detects 61.0% on identical pairs, while the ablated test reports 100% with or without a link, so its apparent sensitivity is uninformative. The block count is not critical: every setting from 16 to 256 lands in the nominal region, and a partition derived from the stationarity test removes the parameter, though less sensitively. Code and data are released.
Authors 2
Nikolaos Al. Papadopoulos, Konstantinos E. Psannis
Specification
- Official page
Source:arXiv (Atom API + RSS)T1observed 51 min agohigh
- Arxiv announce type
- replace
Source:arXiv (Atom API + RSS)T1observed 51 min agohigh
- arXiv id
- 2604.23716
Source:arXiv (Atom API + RSS)T1observed 51 min agohigh
- Categories
- cs.AI, cs.IT, cs.LG, cs.MA, math.IT
Source:arXiv (Atom API + RSS)T1observed 51 min agohigh
Source:arXiv (Atom API + RSS)T1observed 51 min agohigh
- Primary category
- cs.AI
Source:arXiv (Atom API + RSS)T1observed 51 min agohigh
- Published
- 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 51 min agohigh
Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →
Provenance
Attributed facts
9
Source tiers
T19
Freshest observation
51 min ago
Conflicts
None
No models linked to this paper yet.
As of
Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.
Claim history
Official pageofficial_url1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://arxiv.org/abs/2604.23716 | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Abstractabstract1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| Reporting guidance for information-theoretic measures is rarely tested against ground truth. We test one guardrail in two multi-agent reinforcement learning games, a social dilemma and a coordination race, where directed influence between selected agent pairs is zero by construction, over 100 seeds. Omitting one precondition, exclusion of the non-stationary training transient, gives false-positive rates of 100.00% and 99.95%: agents annealing exploration independently, in runs that never met, are flagged as influencing one another. Excluding the transient reaches 3.0% in the social dilemma but 11.8% in the coordination game, which stationarity tests explain: 95.7% of social-dilemma series are stationary afterwards against 56.8% of coordination series. So the non-stationarity must be treated, and exclusion is neither the only way nor sufficient. What we recommend instead changes the null model rather than the data: permuting the source within blocks of training time reaches 5.25% and 5.50%, the only one of four constructions at the size of the test in both games, leaving series, statistic and estimand untouched. Titrating injected links of known strength in both games shows it is also the most sensitive of the three, detecting 89.0% in the coordination game where conditioning detects 61.0% on identical pairs, while the ablated test reports 100% with or without a link, so its apparent sensitivity is uninformative. The block count is not critical: every setting from 16 to 256 lands in the nominal region, and a partition derived from the stationarity test removes the parameter, though less sensitively. Code and data are released. | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Arxiv announce typearxiv_announce_type1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| replace | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
arXiv idarxiv_id1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 2604.23716 | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Categoriescategories1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| cs.AI, cs.IT, cs.LG, cs.MA, math.IT | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
PDFpdf_url1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://arxiv.org/pdf/2604.23716 | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Primary categoryprimary_category1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| cs.AI | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Publishedpublished_at1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 11 Sept 2026 | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →
- New paperPaperNon-Stationarity Breaks Permutation Surrogates in Multi-Agent Reinforcement Learning: Diagnosis and Remedies
New paper: Non-Stationarity Breaks Permutation Surrogates in Multi-Agent Reinforcement Learning: Diagnosis and Remedies
arxiv
| Source | Document | Type | Tier | Last observed | Snapshots |
|---|---|---|---|---|---|
| arXiv (Atom API + RSS) | rss.arxiv.org/rss/cs.AI | feed | T1· Official | 51 min ago | 1 |
Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.