ExpTest: Loss-Curve Hypothesis Testing for Autonomous Learning-Rate Selection in Deep Neural Networks
Updated 5 h ago · first seen 11 Sept 2026
paper_01M294FRQGAF8V31V5E854P0DW
- Published
- 11 Sept 2026
- T1 · 5 h ago
- arXiv
- 2411.16975
- T1 · 5 h ago
- Category
- cs.LG
- T1 · 5 h ago
Abstract
Hyperparameter tuning remains a significant challenge in the training of deep neural networks (DNNs), requiring manual search or time-intensive grid searches that increase resource costs and limit the accessibility of machine learning. The global initial learning rate is among the most consequential of these hyperparameters. Adaptive and scheduling-based methods manage the learning rate during training but still require manual selection of an initial global value; learning-rate-free alternatives remove this selection at the cost of performance or stability on non-convex problems. We present ExpTest, an autonomous learning-rate controller that treats the training loss curve as an online signal and performs sequential statistical tests on theoretically motivated windows to detect convergent behavior and trigger learning-rate reductions. The framework combines a covariance-based initial learning-rate estimate, curvature-motivated window sizing, and two-phase test-driven decay, drawing on the approximately exponential decay behavior predicted under linearized network dynamics. We provide a mathematical motivation for ExpTest and evaluate it on regression, classification, forecasting, and natural-language tasks across fully-connected, convolutional, transformer-based, and pretrained architectures. Across these tasks, ExpTest achieves competitive performance relative to hand-tuned SGD-based baselines and recent learning-rate-free methods, without manual initial learning-rate selection or predefined scheduling.
Authors 2
Zan Chaudhry, Naoko Mizuno
Specification
- Official page
Source:arXiv (Atom API + RSS)T1observed 5 h agohigh
- Arxiv announce type
- replace
Source:arXiv (Atom API + RSS)T1observed 5 h agohigh
- arXiv id
- 2411.16975
Source:arXiv (Atom API + RSS)T1observed 5 h agohigh
- Categories
- cs.LG, cs.AI
Source:arXiv (Atom API + RSS)T1observed 5 h agohigh
Source:arXiv (Atom API + RSS)T1observed 5 h agohigh
- Primary category
- cs.LG
Source:arXiv (Atom API + RSS)T1observed 5 h agohigh
- Published
- 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 5 h agohigh
Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →
Provenance
Attributed facts
9
Source tiers
T19
Freshest observation
5 h ago
Conflicts
None
No models linked to this paper yet.
- Authors
- Zan Chaudhry, Naoko Mizuno
As of
Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.
Claim history
Official pageofficial_url1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://arxiv.org/abs/2411.16975 | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Abstractabstract1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| Hyperparameter tuning remains a significant challenge in the training of deep neural networks (DNNs), requiring manual search or time-intensive grid searches that increase resource costs and limit the accessibility of machine learning. The global initial learning rate is among the most consequential of these hyperparameters. Adaptive and scheduling-based methods manage the learning rate during training but still require manual selection of an initial global value; learning-rate-free alternatives remove this selection at the cost of performance or stability on non-convex problems. We present ExpTest, an autonomous learning-rate controller that treats the training loss curve as an online signal and performs sequential statistical tests on theoretically motivated windows to detect convergent behavior and trigger learning-rate reductions. The framework combines a covariance-based initial learning-rate estimate, curvature-motivated window sizing, and two-phase test-driven decay, drawing on the approximately exponential decay behavior predicted under linearized network dynamics. We provide a mathematical motivation for ExpTest and evaluate it on regression, classification, forecasting, and natural-language tasks across fully-connected, convolutional, transformer-based, and pretrained architectures. Across these tasks, ExpTest achieves competitive performance relative to hand-tuned SGD-based baselines and recent learning-rate-free methods, without manual initial learning-rate selection or predefined scheduling. | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Arxiv announce typearxiv_announce_type1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| replace | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
arXiv idarxiv_id1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 2411.16975 | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Categoriescategories1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| cs.LG, cs.AI | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
PDFpdf_url1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://arxiv.org/pdf/2411.16975 | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Primary categoryprimary_category1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| cs.LG | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Publishedpublished_at1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 11 Sept 2026 | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →
- New paperPaperExpTest: Loss-Curve Hypothesis Testing for Autonomous Learning-Rate Selection in Deep Neural Networks
New paper: ExpTest: Loss-Curve Hypothesis Testing for Autonomous Learning-Rate Selection in Deep Neural Networks
arxiv
| Source | Document | Type | Tier | Last observed | Snapshots |
|---|---|---|---|---|---|
| arXiv (Atom API + RSS) | rss.arxiv.org/rss/cs.LG | feed | T1· Official | 4 h ago | 1 |
Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.