GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems

Published 15 Sept 2026arXiv:2609.14864

data quality89

Updated 29 h ago · first seen 15 Sept 2026

paper_01M2JK0CRN0E2JJJR4MY02GFQK

Abstract

We predict single-sequence model throughput from GGUF metadata using roofline-shaped predictors with quantization-specific scale factors fitted on reference models. The scored cohort comprises 318 phase-depth measurements from 53 host-file configurations on two Apple M4 Max systems and an NVIDIA RTX 5080. On host-specific held-out sets of four, five, and two configurations, an active-parameter decode model obtains 13.1%, 14.4%, and 36.1% mean absolute percentage error (MAPE), versus 49.4%, 55.3%, and 51.9% when charging total parameters. Leave-one-host-out coefficients fitted on the other two systems yield 11.6%, 16.8%, and 36.0% test MAPE. A low-bit model ladder changes ordering across runtime stacks. The P2 prefill baseline gives 18.7%, 22.2%, and 108.2% test MAPE. GGUF structure helps on all three systems, but fitted efficiencies are not universal.

Authors

Authors 6

Bo SuChuhong XuRuiyang XuShimeng DaiXinyu QiuZiyao Chen

Linked names open researcher pages (created from the paper's author list; name-only, no affiliation unless a source states it). Unlinked names have no researcher record yet.

Organizations

Organizations 0

No organization stated. arXiv metadata does not carry affiliations; an organization is linked only when a model card or lab page cites the paper.

Models

Models introduced or described 0

Inbound described_by relations from model cards and documentation.

No model links this paper yet

Model pages link papers through their model cards and documentation; the relation is written only when a source states it.

Datasets

Datasets used 0

No dataset relation recorded.

Benchmarks

Benchmarks used 0

No benchmark relation recorded.

Code

Repositories & frameworks 0

No repository linked.

Timeline

Timeline 2

15 Sept 2026
Property changedPaperGGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems
GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems: arxiv announce type changed from cross to new
Arxiv announce typecross→newarxiv
rss.arxiv.org
15 Sept 2026
New paperPaperGGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems
New paper: GGUF-Metadata Prediction of Single-Sequence llama.cpp Throughput Across Three Systems
arxiv
rss.arxiv.org

Full timeline →

Sources

Sources 2

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.AI	feed	T1· Official	21 h ago	4
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.LG	feed	T1· Official	21 h ago	3

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.