LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation — Paper

Abstract

Large language model serving costs scale directly with output sequence length, yet standard preference alignment often inflates response verbosity without improving utility. We study whether the parameterization of post-training updates affects generation length: low-rank subspaces alter sequence length without modifying the alignment loss. We present LOCUS, a method that selects a task-aware low-rank adaptation subspace to minimize output-token cost subject to a utility constraint. Within this subspace, post-training retains the native preference objective with a frozen backbone. Across Anthropic HH-RLHF dialogue preferences, we evaluate two $\sim$3B decoder backbones, Pythia-2.8B and Qwen2.5-3B, against protocol-matched full-parameter DPO and DrDPO branches and the released SamPO checkpoint. LOCUS reduces continuation length by up to 39.84\% on Pythia-2.8B and by 14.87--17.58\% on Qwen2.5-3B while updating only 0.24--0.28\% of model parameters, with no material change in the internal preference diagnostic.

Authors 1

Dongfang Zhao

Specification

Official page: arxiv.org/abs/2609.11739
Source:arXiv (Atom API + RSS)T1observed 50 min agohigh
Arxiv announce type: new
Source:arXiv (Atom API + RSS)T1observed 49 min agohigh
arXiv id: 2609.11739
Source:arXiv (Atom API + RSS)T1observed 50 min agohigh
Categories: cs.CL, cs.AI, cs.LG
Source:arXiv (Atom API + RSS)T1observed 50 min agohigh
PDF: arxiv.org/pdf/2609.11739
Source:arXiv (Atom API + RSS)T1observed 50 min agohigh
Primary category: cs.CL
Source:arXiv (Atom API + RSS)T1observed 50 min agohigh
Published: 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 50 min agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

9

Source tiers

T19

Freshest observation

49 min ago

Conflicts

None

No models linked to this paper yet.

Authors: Dongfang Zhao

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history · Abstract

1 claims · 1 propertiesShow all properties

Abstractabstract1

Claim history for Abstract
Value	Valid from → to	Status	Source	Confidence	Extractor
Large language model serving costs scale directly with output sequence length, yet standard preference alignment often inflates response verbosity without improving utility. We study whether the parameterization of post-training updates affects generation length: low-rank subspaces alter sequence length without modifying the alignment loss. We present LOCUS, a method that selects a task-aware low-rank adaptation subspace to minimize output-token cost subject to a utility constraint. Within this subspace, post-training retains the native preference objective with a frozen backbone. Across Anthropic HH-RLHF dialogue preferences, we evaluate two $\sim$3B decoder backbones, Pythia-2.8B and Qwen2.5-3B, against protocol-matched full-parameter DPO and DrDPO branches and the released SamPO checkpoint. LOCUS reduces continuation length by up to 39.84\% on Pythia-2.8B and by 14.87--17.58\% on Qwen2.5-3B while updating only 0.24--0.28\% of model parameters, with no material change in the internal preference diagnostic.	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
Property changedPaperLOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation
LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation: arxiv announce type changed from cross to new
Arxiv announce typecross→newarxiv
rss.arxiv.org
11 Sept 2026
New paperPaperLOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation
New paper: LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation
arxiv
rss.arxiv.org

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.CL	feed	T1· Official	49 min ago	1
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.LG	feed	T1· Official	50 min ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.