Score history · claude-3-opus 2 rows

Not enough history to chart — 2 observations, all dated 2 Apr 2024. Rows under different configurations count separately; the list below shows each one.

7%date=2024-04-02 · board=Verified · system=RAG baseline · model_tag=claude-3-opus-202402292 Apr 2024
15.8%date=2024-04-02 · board=Verified · system=SWE-agent · model_tag=claude-3-opus-202402292 Apr 2024

Back to the full leaderboard

Leaderboard 47 current results · config contains “mini-SWE-agent”

Select models with +, then open Compare.

Leaderboard
#	Model	Score	Config	Evaluated	Source	Actions
1	#1Claude Opus 4.5Anthropic	76.8%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=claude-4-5-opus	17 Feb 2026	swebench.comT2	History
2	#2MiniMax M2.5MiniMax	75.8%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=minimax-m2.5	17 Feb 2026	swebench.comT2	History
3	#3gemini-3-flashGoogle	75.8%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=gemini-3-flash-preview	17 Feb 2026	swebench.comT2	History
4	#4Claude Opus 4.6Anthropic	75.6%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=claude-opus-4-6	17 Feb 2026	swebench.comT2	History
5	#5Claude Opus 4.5Anthropic	74.4%	date=2025-11-24 · board=Verified · system=mini-SWE-agent · model_tag=claude-opus-4-5-20251101	24 Nov 2025	swebench.comT2	History
6	#6Gemini 3 Pro PreviewGoogle	74.2%	date=2025-11-18 · board=Verified · system=mini-SWE-agent · model_tag=gemini-3-pro-preview	18 Nov 2025	swebench.comT2	History
7	#7GLM 5Z.ai (Zhipu AI)	72.8%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=glm-5	17 Feb 2026	swebench.comT2	History
8	#8GPT-5.2-CodexOpenAI	72.8%	date=2026-02-19 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-2-codex	19 Feb 2026	swebench.comT2	History
9	#9gpt-5.2OpenAI	72.8%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-2	17 Feb 2026	swebench.comT2	History
10	#10gpt-5.2OpenAI	71.8%	date=2025-12-11 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5.2-2025-12-11	11 Dec 2025	swebench.comT2	History
11	#11Claude 4.5 SonnetAnthropic	71.4%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=claude-sonnet-4-5-20250929	17 Feb 2026	swebench.comT2	History
12	#12Kimi K2.5Moonshot AI	70.8%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=kimi-k2.5	17 Feb 2026	swebench.comT2	History
13	#13Claude 4.5 SonnetAnthropic	70.6%	date=2025-09-29 · board=Verified · system=mini-SWE-agent · model_tag=claude-sonnet-4-5-20250929	29 Sept 2025	swebench.comT2	History
14	#14DeepSeek V3.2DeepSeek	70%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=deepseek-v3.2	17 Feb 2026	swebench.comT2	History
15	#15gemini-3-proGoogle	69.6%	date=2026-02-26 · board=Verified · system=mini-SWE-agent · model_tag=gemini-3-pro-preview	26 Feb 2026	swebench.comT2	History
16	#16gpt-5.2OpenAI	69%	date=2025-12-11 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5.2-2025-12-11	11 Dec 2025	swebench.comT2	History
17	#17claude-4-opusAnthropic	67.6%	date=2025-08-02 · board=Verified · system=mini-SWE-agent · model_tag=claude-4-opus-20250514	2 Aug 2025	swebench.comT2	History
18	#18Claude 4.5 HaikuAnthropic	66.6%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=claude-haiku-4-5-20251001	17 Feb 2026	swebench.comT2	History
19	#19gpt-5.1OpenAI	66%	date=2025-11-20 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5.1-2025-11-13	20 Nov 2025	swebench.comT2	History
20	#20GPT-5.1-CodexOpenAI	66%	date=2025-11-24 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5.1-codex	24 Nov 2025	swebench.comT2	History
21	#21gpt-5OpenAI	65%	date=2025-08-07 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-2025-08-07	7 Aug 2025	swebench.comT2	History
22	#22Claude 4 SonnetAnthropic	64.93%	date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=claude-4-sonnet-20250514	26 Jul 2025	swebench.comT2	History
23	#23Kimi K2 ThinkingMoonshot AI	63.4%	date=2025-12-10 · board=Verified · system=mini-SWE-agent · model_tag=Kimi-K2-Thinking	10 Dec 2025	swebench.comT2	History
24	#24MiniMax M2MiniMax	61%	date=2025-11-24 · board=Verified · system=mini-SWE-agent · model_tag=minimax-m2	24 Nov 2025	swebench.comT2	History
25	#25DeepSeek V3.2 ReasonerDeepSeek	60%	date=2025-12-01 · board=Verified · system=mini-SWE-agent · model_tag=deepseek-v3.2-reasoner	1 Dec 2025	swebench.comT2	History
26	#26gpt-5-miniOpenAI	59.8%	date=2025-08-07 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-mini-2025-08-07	7 Aug 2025	swebench.comT2	History
27	#27o3OpenAI	58.4%	date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=o3-20250416	26 Jul 2025	swebench.comT2	History
28	#28Devstral Small (2512)Mistral AI	56.4%	date=2025-12-09 · board=Verified · system=mini-SWE-agent · model_tag=devstral-small-2512	9 Dec 2025	swebench.comT2	History
29	#29gpt-5-miniOpenAI	56.2%	date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-mini-2025-08-07	17 Feb 2026	swebench.comT2	History
30	#30qwen3-coder-480b-a35b-instructAlibaba Group	55.4%	date=2025-08-02 · board=Verified · system=mini-SWE-agent · model_tag=Qwen3-Coder-480B-A35B-Instruct	2 Aug 2025	swebench.comT2	History
31	#31GLM 4.6Z.ai (Zhipu AI)	55.4%	date=2025-12-01 · board=Verified · system=mini-SWE-agent · model_tag=glm-4.6	1 Dec 2025	swebench.comT2	History
32	#32GLM 4.5Z.ai (Zhipu AI)	54.2%	date=2025-08-22 · board=Verified · system=mini-SWE-agent · model_tag=GLM-4.5	22 Aug 2025	swebench.comT2	History
33	#33Devstral 2Mistral AI	53.8%	date=2025-12-09 · board=Verified · system=mini-SWE-agent · model_tag=devstral-2512	9 Dec 2025	swebench.comT2	History
34	#34Gemini 2.5 ProGoogle	53.6%	date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=gemini-2.5-pro	26 Jul 2025	swebench.comT2	History
35	#35Claude 3.7 SonnetAnthropic	52.8%	date=2025-07-20 · board=Verified · system=mini-SWE-agent · model_tag=claude-3-7-sonnet-20250219	20 Jul 2025	swebench.comT2	History
36	#36o4-miniOpenAI	45%	date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=o4-mini-20250416	26 Jul 2025	swebench.comT2	History
37	#37Kimi K2 0711Moonshot AI	43.8%	date=2025-08-07 · board=Verified · system=mini-SWE-agent · model_tag=Kimi-K2-Instruct	7 Aug 2025	swebench.comT2	History
38	#38gpt-4.1OpenAI	39.58%	date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=gpt-4.1-20250414	26 Jul 2025	swebench.comT2	History
39	#39gpt-5-nanoOpenAI	34.8%	date=2025-08-07 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-nano-2025-08-07	7 Aug 2025	swebench.comT2	History
40	#40Gemini 2.5 FlashGoogle	28.73%	date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=gemini-2.5-flash	26 Jul 2025	swebench.comT2	History
41	#41gpt-oss-120bOpenAI	26%	date=2025-08-07 · board=Verified · system=mini-SWE-agent · model_tag=gpt-oss-120b	7 Aug 2025	swebench.comT2	History
42	#42gpt-4.1-miniOpenAI	23.94%	date=2025-07-20 · board=Verified · system=mini-SWE-agent · model_tag=gpt-4.1-mini-20250414	20 Jul 2025	swebench.comT2	History
43	#43gpt-4oOpenAI	21.62%	date=2025-07-20 · board=Verified · system=mini-SWE-agent · model_tag=gpt-4o-20241120	20 Jul 2025	swebench.comT2	History
44	#44Llama 4 Maverick InstructMeta AI	21.04%	date=2025-07-20 · board=Verified · system=mini-SWE-agent · model_tag=llama-4-maverick-instruct	20 Jul 2025	swebench.comT2	History
45	#45Gemini 2.0 FlashGoogle	13.52%	date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=gemini-2.0-flash	26 Jul 2025	swebench.comT2	History
46	#46Llama 4 Scout InstructMeta AI	9.06%	date=2025-07-20 · board=Verified · system=mini-SWE-agent · model_tag=llama-4-scout-instruct	20 Jul 2025	swebench.comT2	History
47	#47Qwen2.5 Coder 32B InstructQwen	9%	date=2025-08-03 · board=Verified · system=mini-SWE-agent · model_tag=Qwen2.5-Coder-32B-Instruct	3 Aug 2025	swebench.comT2	History

47 results

Scores are reported as published, with their evaluation configuration (harness, prompting, judge). The bar is relative to the best score on this page. Results with different configs are not directly comparable — see methodology.

The config filter matches a value inside each result's configuration (server-side, `config=` on the API). Chips are the values shared by several rows on the first page; per-model identifiers are not offered.

Definition

Category: coding
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Task: resolve real GitHub issues (500 human-validated instances)
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Metric: resolved · %
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Direction: Higher is better
Paper: https://arxiv.org/abs/2310.06770
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Known limitations: Scaffold/agent dependent; results are not comparable across harnesses.
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Website: swebench.com
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium

Each value shows its source, tier and observation time. Missing rows mean no source stated them. How results are recorded →

Evaluated models: Claude Opus 4.6, doubao-seed-code, gemini-3-flash, MiniMax M2.5, Claude Opus 4.5, Gemini 3 Pro Preview, GLM 5, GPT-5.2-Codex +74(82 total)

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history

7 claims · 7 properties

Categorycategory1

Claim history for Category
Value	Valid from → to	Status	Source	Confidence	Extractor
coding	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Known limitationsknown_limitations1

Claim history for Known limitations
Value	Valid from → to	Status	Source	Confidence	Extractor
Scaffold/agent dependent; results are not comparable across harnesses.	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Metricmetric1

Claim history for Metric
Value	Valid from → to	Status	Source	Confidence	Extractor
resolved	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Paperpaper1

Claim history for Paper
Value	Valid from → to	Status	Source	Confidence	Extractor
https://arxiv.org/abs/2310.06770	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Tasktask1

Claim history for Task
Value	Valid from → to	Status	Source	Confidence	Extractor
resolve real GitHub issues (500 human-validated instances)	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Unitunit1

Claim history for Unit
Value	Valid from → to	Status	Source	Confidence	Extractor
%	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Websitewebsite1

Claim history for Website
Value	Valid from → to	Status	Source	Confidence	Extractor
https://www.swebench.com	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
New benchmarkBenchmarkSWE-bench Verified
New benchmark: SWE-bench Verified
registry
swebench.com

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
SWE-bench leaderboards	swebench.com/	leaderboard	T2· Quality secondary	6 h ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.