DCO: Dynamic Cache Orchestration for LLM Accelerators through Predictive Management — Paper

Abstract

-cross Abstract: The rapid adoption of large language models (LLMs) is pushing AI accelerators toward increasingly powerful and specialized designs. Instead of further complicating software development with deeply hierarchical scratchpad memories (SPMs) and their asynchronous management, we investigate the opposite point of the design spectrum: a multi-core AI accelerator equipped with a shared system-level cache and application-aware management policies, which keeps the programming effort modest. Our approach exploits dataflow information available in the software stack to guide cache replacement (including dead-block prediction), in concert with bypass decisions and mechanisms that alleviate cache thrashing. We assess the proposal using a cycle-accurate simulator and observe substantial performance gains (up to 1.80x speedup) compared with conventional cache architectures. In addition, we build and validate an analytical model that takes into account the actual overlapping behaviors to extend the measurement results of our policies to real-world larger-scale workloads. Experiment results show that when functioning together, our bypassing and thrashing mitigation strategies can handle scenarios both with and without inter-core data sharing and achieve remarkable speedups. Finally, we implement the design in RTL and the area of our design is $\mathbf{0.064mm^2}$ with 15nm process, which can run at 2 GHz clock frequency. Our findings explore the potential of the shared cache design to assist the development of future AI accelerator systems.

Authors 4

Chengtao Lai, Wei Zhang, Yuhang Gu, Zhongchun Zhou

Specification

Official page: arxiv.org/abs/2512.07312
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
Arxiv announce type: replace
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
arXiv id: 2512.07312
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
Categories: cs.AI, cs.AR, cs.DC
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
DOI: 10.1109/TC.2026.3733368
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
PDF: arxiv.org/pdf/2512.07312
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
Primary category: cs.AR
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
Published: 12 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

10

Source tiers

T110

Freshest observation

2 h ago

Conflicts

None

No models linked to this paper yet.

Authors: Zhongchun Zhou, Chengtao Lai, Yuhang Gu

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history · PDF

1 claims · 1 propertiesShow all properties

PDFpdf_url1

Claim history for PDF
Value	Valid from → to	Status	Source	Confidence	Extractor
https://arxiv.org/pdf/2512.07312	12 Sept 2026 04:12 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

12 Sept 2026
New paperPaperDCO: Dynamic Cache Orchestration for LLM Accelerators through Predictive Management
New paper: DCO: Dynamic Cache Orchestration for LLM Accelerators through Predictive Management
arxiv
rss.arxiv.org

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.AI	feed	T1· Official	2 h ago	2

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.