Quantifying Logical Consistency in Transformers via Query-Key Alignment — Paper

Abstract

Large language models (LLMs) have demonstrated impressive performance in various natural language processing tasks, yet their ability to perform multi-step logical reasoning remains an open challenge. Although Chain-of-Thought prompting has improved logical reasoning by enabling models to generate intermediate steps, it lacks mechanisms to assess the coherence of these logical transitions. In this paper, we propose a novel, lightweight evaluation strategy for logical reasoning that uses query-key alignments inside transformer attention heads. By computing a single forward pass and extracting a "QK-score" from carefully chosen heads, our method reveals latent representations that reliably separate valid from invalid inferences, offering a scalable alternative to traditional ablation-based techniques. We also provide an empirical validation on multiple logical reasoning benchmarks, demonstrating improved robustness of our evaluation method against distractors and increased reasoning depth. The experiments were conducted on a diverse set of models, ranging from 1.5B to 70B parameters.

Authors 7

Eduard Tulchinskii, Anastasia Voznyuk, Laida Kushnareva, Andrei Andriiainen, Irina Piontkovskaya, Evgeny Burnaev, Serguei Barannikov

Specification

Official page: arxiv.org/abs/2502.17017
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
Arxiv announce type: cross
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
arXiv id: 2502.17017
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
Categories: cs.CL, cs.AI, cs.IT, cs.LG, math.IT, math.LO
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
PDF: arxiv.org/pdf/2502.17017
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
Primary category: cs.CL
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
Published: 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

9

Source tiers

T19

Freshest observation

7 h ago

Conflicts

None

No models linked to this paper yet.

Authors: Eduard Tulchinskii, Anastasia Voznyuk, Laida Kushnareva

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history · arXiv id

1 claims · 1 propertiesShow all properties

arXiv idarxiv_id1

Claim history for arXiv id
Value	Valid from → to	Status	Source	Confidence	Extractor
2502.17017	11 Sept 2026 21:03 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
New paperPaperQuantifying Logical Consistency in Transformers via Query-Key Alignment
New paper: Quantifying Logical Consistency in Transformers via Query-Key Alignment
arxiv
rss.arxiv.org

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.AI	feed	T1· Official	5 h ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.