Skip to content
AI Atlas
BenchmarkActivecategory · coding

SWE-bench Verified

swebench.com

resolve real GitHub issues (500 human-validated instances)

quality57

Updated 8 h ago · first seen 11 Sept 2026

bench_01M293SPEGA1XMKH0SMTFTCK2R

Metric
resolved · %
Direction
Higher is better
Results
180 · 61 filtered
Leader
Claude Opus 4.5 79.2%

Score history · CodeAct v2.1 (claude-3-5-sonnet-20241022) 1 row

Not enough history to chart — a single observation (53% on 29 Oct 2024). Rows under different configurations count separately; the list below shows each one.

  • 53%date=2024-10-29 · board=Verified · system=OpenHands · submission=20241029_OpenHands-CodeAct-2.1-sonnet-2024102229 Oct 2024

Back to the full leaderboard

Leaderboard 61 current results · config contains “SWE-agent”

Select models with +, then open Compare.

Leaderboard
#ModelScoreConfigEvaluatedSourceActions
1#1Claude Opus 4.5Anthropic79.2%date=2025-12-15 · board=Verified · system=live-SWE-agent · model_tag=claude-opus-4-5-2025110115 Dec 2025swebench.comT2 History
2#2Gemini 3 Pro PreviewGoogle77.4%date=2025-11-20 · board=Verified · system=live-SWE-agent · model_tag=gemini-3-pro-preview20 Nov 2025swebench.comT2 History
3#3Claude Opus 4.5Anthropic76.8%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=claude-4-5-opus17 Feb 2026swebench.comT2 History
4#4MiniMax M2.5MiniMax75.8%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=minimax-m2.517 Feb 2026swebench.comT2 History
5#5gemini-3-flashGoogle75.8%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=gemini-3-flash-preview17 Feb 2026swebench.comT2 History
6#6Claude Opus 4.6Anthropic75.6%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=claude-opus-4-617 Feb 2026swebench.comT2 History
7#7Claude Opus 4.5Anthropic74.4%date=2025-11-24 · board=Verified · system=mini-SWE-agent · model_tag=claude-opus-4-5-2025110124 Nov 2025swebench.comT2 History
8#8Gemini 3 Pro PreviewGoogle74.2%date=2025-11-18 · board=Verified · system=mini-SWE-agent · model_tag=gemini-3-pro-preview18 Nov 2025swebench.comT2 History
9#9gpt-5.2OpenAI72.8%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-217 Feb 2026swebench.comT2 History
10#10GPT-5.2-CodexOpenAI72.8%date=2026-02-19 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-2-codex19 Feb 2026swebench.comT2 History
11#11GLM 5Z.ai (Zhipu AI)72.8%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=glm-517 Feb 2026swebench.comT2 History
12#12gpt-5.2OpenAI71.8%date=2025-12-11 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5.2-2025-12-1111 Dec 2025swebench.comT2 History
13#13Claude 4.5 SonnetAnthropic71.4%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=claude-sonnet-4-5-2025092917 Feb 2026swebench.comT2 History
14#14Kimi K2.5Moonshot AI70.8%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=kimi-k2.517 Feb 2026swebench.comT2 History
15#15Claude 4.5 SonnetAnthropic70.6%date=2025-09-29 · board=Verified · system=mini-SWE-agent · model_tag=claude-sonnet-4-5-2025092929 Sept 2025swebench.comT2 History
16#16DeepSeek V3.2DeepSeek70%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=deepseek-v3.217 Feb 2026swebench.comT2 History
17#17gemini-3-proGoogle69.6%date=2026-02-26 · board=Verified · system=mini-SWE-agent · model_tag=gemini-3-pro-preview26 Feb 2026swebench.comT2 History
18#18gpt-5.2OpenAI69%date=2025-12-11 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5.2-2025-12-1111 Dec 2025swebench.comT2 History
19#19claude-4-opusAnthropic67.6%date=2025-08-02 · board=Verified · system=mini-SWE-agent · model_tag=claude-4-opus-202505142 Aug 2025swebench.comT2 History
20#20Claude 4.5 HaikuAnthropic66.6%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=claude-haiku-4-5-2025100117 Feb 2026swebench.comT2 History
21#21Claude 4 SonnetAnthropic66.6%date=2025-05-22 · board=Verified · system=SWE-agent · model_tag=claude-4-sonnet-2025051422 May 2025swebench.comT2 History
22#22gpt-5.1OpenAI66%date=2025-11-20 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5.1-2025-11-1320 Nov 2025swebench.comT2 History
23#23GPT-5.1-CodexOpenAI66%date=2025-11-24 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5.1-codex24 Nov 2025swebench.comT2 History
24#24gpt-5OpenAI65%date=2025-08-07 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-2025-08-077 Aug 2025swebench.comT2 History
25#25Claude 4 SonnetAnthropic64.93%date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=claude-4-sonnet-2025051426 Jul 2025swebench.comT2 History
26#26Kimi K2 ThinkingMoonshot AI63.4%date=2025-12-10 · board=Verified · system=mini-SWE-agent · model_tag=Kimi-K2-Thinking10 Dec 2025swebench.comT2 History
27#27Claude 3.7 Sonnet w/ Review HeavyAnthropic62.4%date=2025-02-25 · board=Verified · system=SWE-agent · model_tag=claude-3-7-sonnet-2025021925 Feb 2025swebench.comT2 History
28#28MiniMax M2MiniMax61%date=2025-11-24 · board=Verified · system=mini-SWE-agent · model_tag=minimax-m224 Nov 2025swebench.comT2 History
29#29DeepSeek V3.2 ReasonerDeepSeek60%date=2025-12-01 · board=Verified · system=mini-SWE-agent · model_tag=deepseek-v3.2-reasoner1 Dec 2025swebench.comT2 History
30#30gpt-5-miniOpenAI59.8%date=2025-08-07 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-mini-2025-08-077 Aug 2025swebench.comT2 History
31#31o3OpenAI58.4%date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=o3-2025041626 Jul 2025swebench.comT2 History
32#32Devstral Small (2512)Mistral AI56.4%date=2025-12-09 · board=Verified · system=mini-SWE-agent · model_tag=devstral-small-25129 Dec 2025swebench.comT2 History
33#33gpt-5-miniOpenAI56.2%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-mini-2025-08-0717 Feb 2026swebench.comT2 History
34#34GLM 4.6Z.ai (Zhipu AI)55.4%date=2025-12-01 · board=Verified · system=mini-SWE-agent · model_tag=glm-4.61 Dec 2025swebench.comT2 History
35#35qwen3-coder-480b-a35b-instructAlibaba Group55.4%date=2025-08-02 · board=Verified · system=mini-SWE-agent · model_tag=Qwen3-Coder-480B-A35B-Instruct2 Aug 2025swebench.comT2 History
36#36GLM 4.5Z.ai (Zhipu AI)54.2%date=2025-08-22 · board=Verified · system=mini-SWE-agent · model_tag=GLM-4.522 Aug 2025swebench.comT2 History
37#37Devstral 2Mistral AI53.8%date=2025-12-09 · board=Verified · system=mini-SWE-agent · model_tag=devstral-25129 Dec 2025swebench.comT2 History
38#38Gemini 2.5 ProGoogle53.6%date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=gemini-2.5-pro26 Jul 2025swebench.comT2 History
39#39Kimi K2 0711Moonshot AI53.4%date=2025-08-04 · board=Verified · system=CodeSweep - SWE-agent · model_tag=kimi-k2-instruct4 Aug 2025swebench.comT2 History
40#40Claude 3.7 SonnetAnthropic52.8%date=2025-07-20 · board=Verified · system=mini-SWE-agent · model_tag=claude-3-7-sonnet-2025021920 Jul 2025swebench.comT2 History
41#41o4-miniOpenAI45%date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=o4-mini-2025041626 Jul 2025swebench.comT2 History
42#42Kimi K2 0711Moonshot AI43.8%date=2025-08-07 · board=Verified · system=mini-SWE-agent · model_tag=Kimi-K2-Instruct7 Aug 2025swebench.comT2 History
43#43SWE-agent-LM-32BQwen40.2%date=2025-05-11 · board=Verified · system=SWE-agent · model_tag=Qwen 2.511 May 2025swebench.comT2 History
44#44gpt-4.1OpenAI39.58%date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=gpt-4.1-2025041426 Jul 2025swebench.comT2 History
45#45Devstral Small 1.1Mistral AI38%date=2025-07-25 · board=Verified · system=SWE-agent · model_tag=devstral-small-250725 Jul 2025swebench.comT2 History
46#46gpt-5-nanoOpenAI34.8%date=2025-08-07 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-nano-2025-08-077 Aug 2025swebench.comT2 History
47#47claude-35-sonnetAnthropic33.6%date=2024-06-20 · board=Verified · system=SWE-agent · model_tag=claude-3-5-sonnet-2024102220 Jun 2024swebench.comT2 History
48#48Gemini 2.5 FlashGoogle28.73%date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=gemini-2.5-flash26 Jul 2025swebench.comT2 History
49#49gpt-oss-120bOpenAI26%date=2025-08-07 · board=Verified · system=mini-SWE-agent · model_tag=gpt-oss-120b7 Aug 2025swebench.comT2 History
50#50gpt-4.1-miniOpenAI23.94%date=2025-07-20 · board=Verified · system=mini-SWE-agent · model_tag=gpt-4.1-mini-2025041420 Jul 2025swebench.comT2 History
51#51gpt-4oOpenAI23.2%date=2024-07-28 · board=Verified · system=SWE-agent · model_tag=gpt-4o-2024-05-1328 Jul 2024swebench.comT2 History
52#52GPT-4 (1106)OpenAI22.4%date=2024-04-02 · board=Verified · system=SWE-agent · model_tag=gpt-4-1106-preview2 Apr 2024swebench.comT2 History
53#53gpt-4oOpenAI21.62%date=2025-07-20 · board=Verified · system=mini-SWE-agent · model_tag=gpt-4o-2024112020 Jul 2025swebench.comT2 History
54#54Llama 4 Maverick InstructMeta AI21.04%date=2025-07-20 · board=Verified · system=mini-SWE-agent · model_tag=llama-4-maverick-instruct20 Jul 2025swebench.comT2 History
55#55claude-3-opusAnthropic15.8%date=2024-04-02 · board=Verified · system=SWE-agent · model_tag=claude-3-opus-202402292 Apr 2024swebench.comT2 History
56#56Gemini 2.0 FlashGoogle13.52%date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=gemini-2.0-flash26 Jul 2025swebench.comT2 History
57#57Llama 4 Scout InstructMeta AI9.06%date=2025-07-20 · board=Verified · system=mini-SWE-agent · model_tag=llama-4-scout-instruct20 Jul 2025swebench.comT2 History
58#58Qwen2.5 Coder 32B InstructQwen9%date=2025-08-03 · board=Verified · system=mini-SWE-agent · model_tag=Qwen2.5-Coder-32B-Instruct3 Aug 2025swebench.comT2 History
59#59SWE-Llama 7BMeta AI1.4%date=2023-10-10 · board=Verified · system=RAG baseline · model_tag=SWE-Llama10 Oct 2023swebench.comT2 History
60#60SWE-Llama 13BMeta AI1.2%date=2023-10-10 · board=Verified · system=RAG baseline · submission=20231010_rag_swellama13b10 Oct 2023swebench.comT2 History
61#61GPT-3.5OpenAI0.4%date=2023-10-10 · board=Verified · system=RAG baseline · submission=20231010_rag_gpt3510 Oct 2023swebench.comT2 History

61 results

Scores are reported as published, with their evaluation configuration (harness, prompting, judge). The bar is relative to the best score on this page. Results with different configs are not directly comparable — see methodology.

The config filter matches a value inside each result's configuration (server-side, `config=` on the API). Chips are the values shared by several rows on the first page; per-model identifiers are not offered.