Updated 2 h ago · first seen 11 Sept 2026
paper_01M294WYE54V8EACFKEQG69B5P
- Published
- 28 Aug 2026
- T2 · 2 h ago
- arXiv
- 2608.27875
- T2 · 2 h ago
Abstract
Quantization has been widely adopted in LLM training and inference to reduce cost and improve efficiency. However, low-bit quantization of the attention module often introduces large errors at very low bit-widths, causing performance degradation. Existing methods mainly rely on smoothing techniques to handle outliers, while we propose a hybrid quantization design to better balance accuracy and efficiency. Specifically, we propose HyQuant, an efficient hybrid quantization framework for LLM attention. HyQuant quantizes most attention states into low-bit formats while retaining a small set of vertical-line tokens and local-window states in high precision. These accuracy-critical regions are selected using lightweight vertical-line-aware attention-pattern signals, reducing quantization error with limited overhead. In the Prefill stage, HyQuant uses a hybrid-precision quantized attention operator that preserves vertical-line tokens and a local sliding window in full precision while quantizing the remaining context. In the Decode stage, HyQuant applies the same principle to KV-cache compression and fuses KV dequantization with attention computation to improve memory and hardware efficiency. Across diverse tasks, models, and datasets, HyQuant maintains nearly lossless accuracy with an extremely simple design, demonstrating the efficiency and practical feasibility of hybrid quantization for LLM attention. Code is available at: https://github.com/jerrysfls/HyQuant .
Authors 12
Jiatong Ding, Bingxin Xing, Yu Zhang, Dian Ding, Xiaodong Yi, Xianbin Ouyang, Feihu Zhou, Kun Zhang, Zhenyu Guo, Hao Pan, Guangtao Xue, Yiming Zhang
Specification
- Official page
Source:Hugging Face Hub (public pages, model cards, papers)T2observed 2 h agomedium
- arXiv id
- 2608.27875
Source:Hugging Face Hub (public pages, model cards, papers)T2observed 2 h agomedium
- Github repo
- jerrysfls/HyQuant
Source:Hugging Face Hub (public pages, model cards, papers)T2observed 2 h agomedium
- Hf paper url
- https://huggingface.co/papers/2608.27875
Source:Hugging Face Hub (public pages, model cards, papers)T2observed 2 h agomedium
- Github stars
- 8
Source:Hugging Face Hub (public pages, model cards, papers)T2observed 2 h agomedium
- Hf comments
- 1
Source:Hugging Face Hub (public pages, model cards, papers)T2observed 2 h agomedium
- Upvotes
- 4
Source:Hugging Face Hub (public pages, model cards, papers)T2observed 2 h agomedium
Source:Hugging Face Hub (public pages, model cards, papers)T2observed 2 h agomedium
- Published
- 28 Aug 2026
Source:Hugging Face Hub (public pages, model cards, papers)T2observed 2 h agomedium
Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →
Provenance
Attributed facts
11
Source tiers
T211
Freshest observation
2 h ago
Conflicts
None
No models linked to this paper yet.
No relations recorded.
As of
Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.
Claim history · Authors
Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →
New paper: HyQuant: Hybrid-Precision Quantization for LLM Attention
huggingface
| Source | Document | Type | Tier | Last observed | Snapshots |
|---|---|---|---|---|---|
| Hugging Face Hub (public pages, model cards, papers) | huggingface.co/papers | listing | T2· Quality secondary | 2 h ago | 2 |
Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.