Statistically Valid Post-Training Hyperparameter Selection: From Tuning to Guarantees
Updated 7 h ago · first seen 11 Sept 2026
paper_01M294FTA3F2RDNYCWKF0PVS1V
- Published
- 11 Sept 2026
- T1 · 7 h ago
- arXiv
- 2606.25601
- T1 · 7 h ago
- Category
- stat.ML
- T1 · 7 h ago
Abstract
-cross Abstract: Post-training hyperparameter selection is a critical step in the deployment of modern artificial intelligence systems, given the need to tune degrees of freedom of pre-trained models such as inference-time parameters, implementation-level settings, and thresholds driving decision rules. Despite its practical importance, hyperparameter selection is typically performed using best-effort empirical methods such as grid search or Bayesian optimization, which provide no formal statistical guarantees on reliability or safety. This monograph, intended for an audience of signal processing and machine learning researchers, presents a unified statistical framework for reliable post-training hyperparameter selection, centered on the learn-then-test (LTT) paradigm. LTT formulates the hyperparameter selection problem as multiple hypothesis testing over a candidate set of hyperparameters. The framework enables the choice of hyperparameters that provably satisfy application-specific reliability requirements---such as bounds on average risk, quantile risk, or information-theoretic constraints---with explicit, finite-sample control of error probabilities. The supporting statistical machinery, namely p-values, e-values, and concentration inequalities, is developed from first principles.
Authors 2
Amirmohammad Farzaneh, Osvaldo Simeone
Specification
- Official page
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
- Arxiv announce type
- replace
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
- arXiv id
- 2606.25601
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
- Categories
- stat.ML, cs.IT, cs.LG, math.IT, math.ST, stat.TH
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
- Primary category
- stat.ML
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
- Published
- 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →
Provenance
Attributed facts
9
Source tiers
T19
Freshest observation
7 h ago
Conflicts
None
No models linked to this paper yet.
- Authors
- Amirmohammad Farzaneh, Osvaldo Simeone
As of
Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.
Claim history · Abstract
Abstractabstract1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| -cross Abstract: Post-training hyperparameter selection is a critical step in the deployment of modern artificial intelligence systems, given the need to tune degrees of freedom of pre-trained models such as inference-time parameters, implementation-level settings, and thresholds driving decision rules. Despite its practical importance, hyperparameter selection is typically performed using best-effort empirical methods such as grid search or Bayesian optimization, which provide no formal statistical guarantees on reliability or safety. This monograph, intended for an audience of signal processing and machine learning researchers, presents a unified statistical framework for reliable post-training hyperparameter selection, centered on the learn-then-test (LTT) paradigm. LTT formulates the hyperparameter selection problem as multiple hypothesis testing over a candidate set of hyperparameters. The framework enables the choice of hyperparameters that provably satisfy application-specific reliability requirements---such as bounds on average risk, quantile risk, or information-theoretic constraints---with explicit, finite-sample control of error probabilities. The supporting statistical machinery, namely p-values, e-values, and concentration inequalities, is developed from first principles. | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →
New paper: Statistically Valid Post-Training Hyperparameter Selection: From Tuning to Guarantees
arxiv
| Source | Document | Type | Tier | Last observed | Snapshots |
|---|---|---|---|---|---|
| arXiv (Atom API + RSS) | rss.arxiv.org/rss/cs.LG | feed | T1· Official | 5 min ago | 1 |
Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.