Score history · Devstral 2 1 row

Not enough history to chart — a single observation (53.8% on 9 Dec 2025). Rows under different configurations count separately; the list below shows each one.

53.8%date=2025-12-09 · board=Verified · system=mini-SWE-agent · model_tag=devstral-25129 Dec 2025

Back to the full leaderboard

Leaderboard 10 current results · config contains “OpenHands”

Select models with +, then open Compare.

Leaderboard
#	Model	Score	Config	Evaluated	Source	Actions
1	#1MultipleAnthropic	73.8%	date=2025-11-03 · board=Verified · system=Salesforce AI Research SAGE · model_tag=claude-sonnet-4.5	3 Nov 2025	swebench.comT2	History
2	#2gpt-5OpenAI	71.8%	date=2025-08-07 · board=Verified · system=OpenHands · model_tag=openai/gpt-5-2025-08-07	7 Aug 2025	swebench.comT2	History
3	#3Claude 4 SonnetAnthropic	70.4%	date=2025-05-24 · board=Verified · system=OpenHands · model_tag=claude-4-sonnet-20250514	24 May 2025	swebench.comT2	History
4	#4qwen3-coder-480b-a35b-instructAlibaba Group	69.6%	date=2025-08-05 · board=Verified · system=OpenHands · model_tag=https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct	5 Aug 2025	swebench.comT2	History
5	#5Undisclosed	65.8%	date=2025-04-15 · board=Verified · system=OpenHands · submission=20250415_openhands	15 Apr 2025	swebench.comT2	History
6	#6Kimi K2 0711Moonshot AI	65.4%	date=2025-07-16 · board=Verified · system=OpenHands · model_tag=moonshot/kimi-k2-0711-preview	16 Jul 2025	swebench.comT2	History
7	#74x Scaled	60.8%	date=2025-02-03 · board=Verified · system=OpenHands · submission=20250203_openhands_4x_scaled	3 Feb 2025	swebench.comT2	History
8	#8CodeAct v2.1 (claude-3-5-sonnet-20241022)Anthropic	53%	date=2024-10-29 · board=Verified · system=OpenHands · submission=20241029_OpenHands-CodeAct-2.1-sonnet-20241022	29 Oct 2024	swebench.comT2	History
9	#9Qwen3 Coder 30B A3B InstructQwen	51.6%	date=2025-08-05 · board=Verified · system=OpenHands · model_tag=https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct	5 Aug 2025	swebench.comT2	History
10	#10Devstral Small 1.0Mistral AI	46.8%	date=2025-05-20 · board=Verified · system=OpenHands · model_tag=mistralai/Devstral-Small-2505	20 May 2025	swebench.comT2	History

10 results

Scores are reported as published, with their evaluation configuration (harness, prompting, judge). The bar is relative to the best score on this page. Results with different configs are not directly comparable — see methodology.

The config filter matches a value inside each result's configuration (server-side, `config=` on the API). Chips are the values shared by several rows on the first page; per-model identifiers are not offered.

Definition

Category: coding
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Task: resolve real GitHub issues (500 human-validated instances)
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Metric: resolved · %
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Direction: Higher is better
Paper: https://arxiv.org/abs/2310.06770
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Known limitations: Scaffold/agent dependent; results are not comparable across harnesses.
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Website: swebench.com
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium

Each value shows its source, tier and observation time. Missing rows mean no source stated them. How results are recorded →

Evaluated models: Claude Opus 4.6, doubao-seed-code, gemini-3-flash, MiniMax M2.5, Claude Opus 4.5, Gemini 3 Pro Preview, GLM 5, GPT-5.2-Codex +74(82 total)

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history

7 claims · 7 properties

Categorycategory1

Claim history for Category
Value	Valid from → to	Status	Source	Confidence	Extractor
coding	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Known limitationsknown_limitations1

Claim history for Known limitations
Value	Valid from → to	Status	Source	Confidence	Extractor
Scaffold/agent dependent; results are not comparable across harnesses.	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Metricmetric1

Claim history for Metric
Value	Valid from → to	Status	Source	Confidence	Extractor
resolved	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Paperpaper1

Claim history for Paper
Value	Valid from → to	Status	Source	Confidence	Extractor
https://arxiv.org/abs/2310.06770	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Tasktask1

Claim history for Task
Value	Valid from → to	Status	Source	Confidence	Extractor
resolve real GitHub issues (500 human-validated instances)	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Unitunit1

Claim history for Unit
Value	Valid from → to	Status	Source	Confidence	Extractor
%	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Websitewebsite1

Claim history for Website
Value	Valid from → to	Status	Source	Confidence	Extractor
https://www.swebench.com	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
New benchmarkBenchmarkSWE-bench Verified
New benchmark: SWE-bench Verified
registry
swebench.com

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
SWE-bench leaderboards	swebench.com/	leaderboard	T2· Quality secondary	6 h ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.