Updated 28 min ago · first seen 11 Sept 2026
paper_01M294FPKESFYQH05PBRCV40MW
- Published
- 11 Sept 2026
- T1 · 28 min ago
- arXiv
- 2609.10550
- T1 · 28 min ago
- Category
- cs.SE
- T1 · 28 min ago
Abstract
AI deployment performance is shaped not by model architecture alone, but by interactions among compression, compiler transformations, and serving policies. Published benchmarks often report latency and throughput under incomparable conditions, limiting their use for deployment decisions. This paper presents a unified analytical treatment of inference optimization across the deployment stack. We introduce a three-layer taxonomy covering model-level techniques such as quantization, pruning, and distillation; compiler transformations such as graph fusion, layout optimization, and kernel autotuning; and system policies such as dynamic batching, admission control, and memory tiering. We formulate deployment as a constrained multi-objective optimization problem over accuracy, latency, throughput, memory footprint, and energy, and analyze a deployment-ranking functional with Pareto monotonicity and scale invariance. Roofline models show how memory-bandwidth hierarchies bound performance across precision regimes, while queuing models explain how service-time changes amplify response time under load. To improve comparability, we propose an evidence protocol that separates measured, derived, and analytical claims; limits numerical comparison to within-paper results; and requires reporting of hardware, software versions, batch semantics, and thermal state. We synthesize evidence from edge platforms, including Jetson AGX Orin and five inference frameworks; data center GPUs, including A100 and H100 with three LLM serving engines; and quantization studies across the Llama-3.1 family. The synthesis shows that deployment outcomes are governed by cross-layer interactions that no single-layer analysis can predict. We conclude with a constraint-aware selection procedure and open problems in compiler-serving co-optimization, cross-hardware performance prediction, and standardized energy reporting.
Authors 6
Tejinder Singh, John Pflueger, Jeebak Mitra, Robert Lincourt, Mitchell Markow, Bhavesh A. Patel
Specification
- Official page
Source:arXiv (Atom API + RSS)T1observed 28 min agohigh
- Arxiv announce type
- cross
Source:arXiv (Atom API + RSS)T1observed 28 min agohigh
- arXiv id
- 2609.10550
Source:arXiv (Atom API + RSS)T1observed 28 min agohigh
- Categories
- cs.SE, cs.LG
Source:arXiv (Atom API + RSS)T1observed 28 min agohigh
Source:arXiv (Atom API + RSS)T1observed 28 min agohigh
- Primary category
- cs.SE
Source:arXiv (Atom API + RSS)T1observed 28 min agohigh
- Published
- 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 28 min agohigh
Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →
Provenance
Attributed facts
9
Source tiers
T19
Freshest observation
28 min ago
Conflicts
None
No models linked to this paper yet.
- Authors
- Tejinder Singh, John Pflueger, Jeebak Mitra
As of
Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.
Claim history
Official pageofficial_url1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://arxiv.org/abs/2609.10550 | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Abstractabstract1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| AI deployment performance is shaped not by model architecture alone, but by interactions among compression, compiler transformations, and serving policies. Published benchmarks often report latency and throughput under incomparable conditions, limiting their use for deployment decisions. This paper presents a unified analytical treatment of inference optimization across the deployment stack. We introduce a three-layer taxonomy covering model-level techniques such as quantization, pruning, and distillation; compiler transformations such as graph fusion, layout optimization, and kernel autotuning; and system policies such as dynamic batching, admission control, and memory tiering. We formulate deployment as a constrained multi-objective optimization problem over accuracy, latency, throughput, memory footprint, and energy, and analyze a deployment-ranking functional with Pareto monotonicity and scale invariance. Roofline models show how memory-bandwidth hierarchies bound performance across precision regimes, while queuing models explain how service-time changes amplify response time under load. To improve comparability, we propose an evidence protocol that separates measured, derived, and analytical claims; limits numerical comparison to within-paper results; and requires reporting of hardware, software versions, batch semantics, and thermal state. We synthesize evidence from edge platforms, including Jetson AGX Orin and five inference frameworks; data center GPUs, including A100 and H100 with three LLM serving engines; and quantization studies across the Llama-3.1 family. The synthesis shows that deployment outcomes are governed by cross-layer interactions that no single-layer analysis can predict. We conclude with a constraint-aware selection procedure and open problems in compiler-serving co-optimization, cross-hardware performance prediction, and standardized energy reporting. | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Arxiv announce typearxiv_announce_type1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| cross | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
arXiv idarxiv_id1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 2609.10550 | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Categoriescategories1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| cs.SE, cs.LG | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
PDFpdf_url1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://arxiv.org/pdf/2609.10550 | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Primary categoryprimary_category1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| cs.SE | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Publishedpublished_at1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 11 Sept 2026 | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →
New paper: Optimizing AI Inference Across the Deployment Stack
arxiv
| Source | Document | Type | Tier | Last observed | Snapshots |
|---|---|---|---|---|---|
| arXiv (Atom API + RSS) | rss.arxiv.org/rss/cs.LG | feed | T1· Official | 28 min ago | 1 |
Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.