LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry — Paper

Abstract

Structured pruning of large language models (LLMs) offers hardware-efficient compression, yet existing methods require calibration data, gradient computation, or large auxiliary policy networks at pruning time. LILA (\emph{Latent-Informed Layer Analysis}) scores neuron importance via the Kolmogorov--Smirnov (KS) distance between empirical singular value distributions of the full and neuron-ablated feed-forward network (FFN) weight matrix, providing a closed-form spectral rule requiring no training, calibration data, or auxiliary network. Without any fine-tuning, LILA surpasses PruneNet (45M-parameter RL policy) by 1.57~pp in zero-shot accuracy on LLaMA-2-7B at 25\% sparsity, and outperforms WikiText-2-calibrated SliceGPT by up to 6.0~pp across all sparsity levels, while preserving the original architecture. After one epoch of LoRA recovery fine-tuning, LILA achieves highly competitive performance, matching the heavily calibrated SliceGPT baseline to within a 0.48~pp margin across LLaMA-2-7B and Phi-2, despite using zero calibration data. A Neural Tangent Kernel analysis confirms a 22$\times$ reduction in functional distortion versus random pruning, providing theoretical grounding for the spectral importance criterion. Finally, extending LILA to dynamically allocate sparsity budgets via KS-scores yields state-of-the-art generative preservation at moderate compression, while uncovering fundamental single-layer architectural bottlenecks at higher compression regimes.

Authors 6

Sankar Behera, Dhruv Singh, Anshika Agnihotri, Raj Kumar Choudhary, Satyadev Ahlawat, Yamuna Prasad

Specification

Official page: arxiv.org/abs/2609.11163
Source:arXiv (Atom API + RSS)T1observed 46 min agohigh
Arxiv announce type: cross
Source:arXiv (Atom API + RSS)T1observed 46 min agohigh
arXiv id: 2609.11163
Source:arXiv (Atom API + RSS)T1observed 46 min agohigh
Categories: cs.LG, cs.CL
Source:arXiv (Atom API + RSS)T1observed 46 min agohigh
PDF: arxiv.org/pdf/2609.11163
Source:arXiv (Atom API + RSS)T1observed 46 min agohigh
Primary category: cs.LG
Source:arXiv (Atom API + RSS)T1observed 46 min agohigh
Published: 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 46 min agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

9

Source tiers

T19

Freshest observation

46 min ago

Conflicts

None

No models linked to this paper yet.

Authors: Sankar Behera, Dhruv Singh, Anshika Agnihotri

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history · Official page

1 claims · 1 propertiesShow all properties

Official pageofficial_url1

Claim history for Official page
Value	Valid from → to	Status	Source	Confidence	Extractor
https://arxiv.org/abs/2609.11163	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
Property changedPaperLILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry
LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry: arxiv announce type changed from new to cross
Arxiv announce typenew→crossarxiv
rss.arxiv.org
11 Sept 2026
New paperPaperLILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry
New paper: LILA: Calibration-Free Structured Pruning of Large Language Models via Latent Spectral Geometry
arxiv
rss.arxiv.org

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.CL	feed	T1· Official	46 min ago	1
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.LG	feed	T1· Official	46 min ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.