Score history · Devstral Small 1.1 1 row

Not enough history to chart — a single observation (38% on 25 Jul 2025). Rows under different configurations count separately; the list below shows each one.

38%date=2025-07-25 · board=Verified · system=SWE-agent · model_tag=devstral-small-250725 Jul 2025

Back to the full leaderboard

Leaderboard 11 current results · config contains “high”

Select models with +, then open Compare.

Leaderboard
#	Model	Score	Config	Evaluated	Source	Actions
1	#1Claude Opus 4.5Anthropic	76.8%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=claude-4-5-opus	17 Feb 2026	swebench.comT2	History
2	#2MiniMax M2.5MiniMax	75.8%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=minimax-m2.5	17 Feb 2026	swebench.comT2	History
3	#3gemini-3-flashGoogle	75.8%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=gemini-3-flash-preview	17 Feb 2026	swebench.comT2	History
4	#4GLM 5Z.ai (Zhipu AI)	72.8%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=glm-5	17 Feb 2026	swebench.comT2	History
5	#5gpt-5.2OpenAI	72.8%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-2	17 Feb 2026	swebench.comT2	History
6	#6gpt-5.2OpenAI	71.8%	date=2025-12-11 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5.2-2025-12-11	11 Dec 2025	swebench.comT2	History
7	#7Claude 4.5 SonnetAnthropic	71.4%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=claude-sonnet-4-5-20250929	17 Feb 2026	swebench.comT2	History
8	#8Kimi K2.5Moonshot AI	70.8%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=kimi-k2.5	17 Feb 2026	swebench.comT2	History
9	#9DeepSeek V3.2DeepSeek	70%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=deepseek-v3.2	17 Feb 2026	swebench.comT2	History
10	#10gemini-3-proGoogle	69.6%	date=2026-02-26 · board=Verified · system=mini-SWE-agent · model_tag=gemini-3-pro-preview	26 Feb 2026	swebench.comT2	History
11	#11Claude 4.5 HaikuAnthropic	66.6%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=claude-haiku-4-5-20251001	17 Feb 2026	swebench.comT2	History

11 results

Scores are reported as published, with their evaluation configuration (harness, prompting, judge). The bar is relative to the best score on this page. Results with different configs are not directly comparable — see methodology.

The config filter matches a value inside each result's configuration (server-side, `config=` on the API). Chips are the values shared by several rows on the first page; per-model identifiers are not offered.

Definition

Category: coding
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Task: resolve real GitHub issues (500 human-validated instances)
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Metric: resolved · %
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Direction: Higher is better
Paper: https://arxiv.org/abs/2310.06770
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Known limitations: Scaffold/agent dependent; results are not comparable across harnesses.
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Website: swebench.com
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium

Each value shows its source, tier and observation time. Missing rows mean no source stated them. How results are recorded →

Evaluated models: Claude Opus 4.6, doubao-seed-code, gemini-3-flash, MiniMax M2.5, Claude Opus 4.5, Gemini 3 Pro Preview, GLM 5, GPT-5.2-Codex +74(82 total)

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history

7 claims · 7 properties

Categorycategory1

Claim history for Category
Value	Valid from → to	Status	Source	Confidence	Extractor
coding	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Known limitationsknown_limitations1

Claim history for Known limitations
Value	Valid from → to	Status	Source	Confidence	Extractor
Scaffold/agent dependent; results are not comparable across harnesses.	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Metricmetric1

Claim history for Metric
Value	Valid from → to	Status	Source	Confidence	Extractor
resolved	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Paperpaper1

Claim history for Paper
Value	Valid from → to	Status	Source	Confidence	Extractor
https://arxiv.org/abs/2310.06770	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Tasktask1

Claim history for Task
Value	Valid from → to	Status	Source	Confidence	Extractor
resolve real GitHub issues (500 human-validated instances)	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Unitunit1

Claim history for Unit
Value	Valid from → to	Status	Source	Confidence	Extractor
%	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Websitewebsite1

Claim history for Website
Value	Valid from → to	Status	Source	Confidence	Extractor
https://www.swebench.com	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
New benchmarkBenchmarkSWE-bench Verified
New benchmark: SWE-bench Verified
registry
swebench.com

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
SWE-bench leaderboards	swebench.com/	leaderboard	T2· Quality secondary	6 h ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.