Skip to content
AI Atlas
BenchmarkActivecategory · coding

SWE-bench Verified

swebench.com

resolve real GitHub issues (500 human-validated instances)

quality57

Updated 8 h ago · first seen 11 Sept 2026

bench_01M293SPEGA1XMKH0SMTFTCK2R

Metric
resolved · %
Direction
Higher is better
Results
180 · 117 filtered
Leader
Claude Opus 4.5 79.2%

Score history · Qwen3 Coder 30B A3B Instruct 3 rows

0%20%40%60%80%Aug 25Aug 25Aug 25Aug 25
  • Qwen3 Coder 30B A3B Instruct
  • 52.2%date=2025-09-01 · board=Verified · system=EntroPO + R2E · model_tag=Qwen3-Coder-30B-A3B-Instruct1 Sept 2025
  • 60.4%date=2025-09-01 · board=Verified · system=EntroPO + R2E · model_tag=Qwen3-Coder-30B-A3B-Instruct1 Sept 2025
  • 51.6%date=2025-08-05 · board=Verified · system=OpenHands · model_tag=https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct5 Aug 2025

Back to the full leaderboard

Leaderboard 117 current results · config contains “true”

Select models with +, then open Compare.

Leaderboard
#ModelScoreConfigEvaluatedSourceActions
1#1Claude Opus 4.5Anthropic79.2%date=2025-12-15 · board=Verified · system=live-SWE-agent · model_tag=claude-opus-4-5-2025110115 Dec 2025swebench.comT2 History
2#2doubao-seed-codeByteDance78.8%date=2025-09-28 · board=Verified · system=TRAE · model_tag=Doubao-Seed-Code28 Sept 2025swebench.comT2 History
3#3Gemini 3 Pro PreviewGoogle77.4%date=2025-11-20 · board=Verified · system=live-SWE-agent · model_tag=gemini-3-pro-preview20 Nov 2025swebench.comT2 History
4#4Claude Opus 4.5Anthropic76.8%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=claude-4-5-opus17 Feb 2026swebench.comT2 History
5#5gemini-3-flashGoogle75.8%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=gemini-3-flash-preview17 Feb 2026swebench.comT2 History
6#6MiniMax M2.5MiniMax75.8%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=minimax-m2.517 Feb 2026swebench.comT2 History
7#7Claude Opus 4.6Anthropic75.6%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=claude-opus-4-617 Feb 2026swebench.comT2 History
8#8MultipleAnthropic75.2%date=2025-06-12 · board=Verified · system=TRAE · model_tag=claude-4-sonnet-2025052212 Jun 2025swebench.comT2 History
9#9MultipleAnthropic74.6%date=2025-09-15 · board=Verified · system=JoyCode · model_tag=claude-4-sonnet15 Sept 2025swebench.comT2 History
10#10Claude 4 SonnetAnthropic74.6%date=2025-07-20 · board=Verified · system=Lingxi-v1.5 · model_tag=claude-4-sonnet-2025051420 Jul 2025swebench.comT2 History
11#11MultipleAnthropic74.4%date=2025-06-03 · board=Verified · system=Refact.ai Agent · model_tag=claude-4-sonnet3 Jun 2025swebench.comT2 History
12#12Claude Opus 4.5Anthropic74.4%date=2025-11-24 · board=Verified · system=mini-SWE-agent · model_tag=claude-opus-4-5-2025110124 Nov 2025swebench.comT2 History
13#13gpt-5OpenAI74.4%date=2025-10-15 · board=Verified · system=Prometheus-v1.2.1 · model_tag=gpt-5-2025-08-0715 Oct 2025swebench.comT2 History
14#14Gemini 3 Pro PreviewGoogle74.2%date=2025-11-18 · board=Verified · system=mini-SWE-agent · model_tag=gemini-3-pro-preview18 Nov 2025swebench.comT2 History
15#15gpt-5.2OpenAI72.8%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-217 Feb 2026swebench.comT2 History
16#16GPT-5.2-CodexOpenAI72.8%date=2026-02-19 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-2-codex19 Feb 2026swebench.comT2 History
17#17GLM 5Z.ai (Zhipu AI)72.8%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=glm-517 Feb 2026swebench.comT2 History
18#18gpt-5OpenAI71.8%date=2025-08-07 · board=Verified · system=OpenHands · model_tag=openai/gpt-5-2025-08-077 Aug 2025swebench.comT2 History
19#19gpt-5.2OpenAI71.8%date=2025-12-11 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5.2-2025-12-1111 Dec 2025swebench.comT2 History
20#20Claude 4.5 SonnetAnthropic71.4%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=claude-sonnet-4-5-2025092917 Feb 2026swebench.comT2 History
21#21Kimi K2 0711Moonshot AI71.2%date=2025-10-14 · board=Verified · system=Lingxi v1.5 · model_tag=kimi-k2-0905-preview14 Oct 2025swebench.comT2 History
22#22gpt-5OpenAI71.2%date=2025-09-29 · board=Verified · system=Prometheus-v1.2 · model_tag=gpt-5-2025-08-0729 Sept 2025swebench.comT2 History
23#23Claude 4 SonnetAnthropic70.8%date=2025-06-11 · board=Verified · system=Moatless Tools · model_tag=claude-4-sonnet-2025051411 Jun 2025swebench.comT2 History
24#24Kimi K2.5Moonshot AI70.8%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=kimi-k2.517 Feb 2026swebench.comT2 History
25#25Claude 4.5 SonnetAnthropic70.6%date=2025-09-29 · board=Verified · system=mini-SWE-agent · model_tag=claude-sonnet-4-5-2025092929 Sept 2025swebench.comT2 History
26#26Claude 4 SonnetAnthropic70.4%date=2025-05-24 · board=Verified · system=OpenHands · model_tag=claude-4-sonnet-2025051424 May 2025swebench.comT2 History
27#27Undisclosed70.4%date=2025-06-10 · board=Verified · system=Augment Agent v1 · submission=20250610_augment_agent_v110 Jun 2025swebench.comT2 History
28#28MultipleAnthropic70.4%date=2025-05-15 · board=Verified · system=Refact.ai Agent · model_tag=claude-3-7-sonnet-2025021915 May 2025swebench.comT2 History
29#29DeepSeek V3.2DeepSeek70%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=deepseek-v3.217 Feb 2026swebench.comT2 History
30#30gemini-3-proGoogle69.6%date=2026-02-26 · board=Verified · system=mini-SWE-agent · model_tag=gemini-3-pro-preview26 Feb 2026swebench.comT2 History
31#31qwen3-coder-480b-a35b-instructAlibaba Group69.6%date=2025-08-05 · board=Verified · system=OpenHands · model_tag=https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct5 Aug 2025swebench.comT2 History
32#32gpt-5.2OpenAI69%date=2025-12-11 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5.2-2025-12-1111 Dec 2025swebench.comT2 History
33#33GLM 4.6Z.ai (Zhipu AI)68.2%date=2025-09-30 · board=Verified · system=Undisclosed · model_tag=https://huggingface.co/zai-org/GLM-4.630 Sept 2025swebench.comT2 History
34#34MultipleAnthropic68.2%date=2025-05-16 · board=Verified · system=Nemotron-CORTEXA · model_tag=NV-EmbedCode16 May 2025swebench.comT2 History
35#35claude-4-opusAnthropic67.6%date=2025-08-02 · board=Verified · system=mini-SWE-agent · model_tag=claude-4-opus-202505142 Aug 2025swebench.comT2 History
36#36Claude 4.5 HaikuAnthropic66.6%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=claude-haiku-4-5-2025100117 Feb 2026swebench.comT2 History
37#37Claude 4 SonnetAnthropic66.6%date=2025-05-22 · board=Verified · system=SWE-agent · model_tag=claude-4-sonnet-2025051422 May 2025swebench.comT2 History
38#38gpt-5.1OpenAI66%date=2025-11-20 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5.1-2025-11-1320 Nov 2025swebench.comT2 History
39#39GPT-5.1-CodexOpenAI66%date=2025-11-24 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5.1-codex24 Nov 2025swebench.comT2 History
40#40Undisclosed65.8%date=2025-04-15 · board=Verified · system=OpenHands · submission=20250415_openhands15 Apr 2025swebench.comT2 History
41#41Kimi K2 0711Moonshot AI65.4%date=2025-07-16 · board=Verified · system=OpenHands · model_tag=moonshot/kimi-k2-0711-preview16 Jul 2025swebench.comT2 History
42#42Undisclosed65.4%date=2025-03-16 · board=Verified · system=Augment Agent v0 · submission=20250316_augment_agent_v016 Mar 2025swebench.comT2 History
43#43gpt-5OpenAI65%date=2025-08-07 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-2025-08-077 Aug 2025swebench.comT2 History
44#44Claude 4 SonnetAnthropic64.93%date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=claude-4-sonnet-2025051426 Jul 2025swebench.comT2 History
45#45o4-miniOpenAI64.6%date=2025-05-03 · board=Verified · system=PatchPilot-v1.1 · model_tag=o4-mini3 May 2025swebench.comT2 History
46#46GLM 4.5Z.ai (Zhipu AI)64.2%date=2025-07-28 · board=Verified · system=Undisclosed · model_tag=https://huggingface.co/zai-org/GLM-4.528 Jul 2025swebench.comT2 History
47#47Kimi K2 ThinkingMoonshot AI63.4%date=2025-12-10 · board=Verified · system=mini-SWE-agent · model_tag=Kimi-K2-Thinking10 Dec 2025swebench.comT2 History
48#48Claude 3.7 Sonnet w/ Review HeavyAnthropic62.4%date=2025-02-25 · board=Verified · system=SWE-agent · model_tag=claude-3-7-sonnet-2025021925 Feb 2025swebench.comT2 History
49#49MiniMax M2MiniMax61%date=2025-11-24 · board=Verified · system=mini-SWE-agent · model_tag=minimax-m224 Nov 2025swebench.comT2 History
50#504x Scaled60.8%date=2025-02-03 · board=Verified · system=OpenHands · submission=20250203_openhands_4x_scaled3 Feb 2025swebench.comT2 History
51#51Qwen3 Coder 30B A3B InstructQwen60.4%date=2025-09-01 · board=Verified · system=EntroPO + R2E · model_tag=Qwen3-Coder-30B-A3B-Instruct1 Sept 2025swebench.comT2 History
52#52DeepSeek V3.2 ReasonerDeepSeek60%date=2025-12-01 · board=Verified · system=mini-SWE-agent · model_tag=deepseek-v3.2-reasoner1 Dec 2025swebench.comT2 History
53#53gpt-5-miniOpenAI59.8%date=2025-08-07 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-mini-2025-08-077 Aug 2025swebench.comT2 History
54#54TTS(Bo16)OpenAI58.8%date=2025-06-29 · board=Verified · system=DeepSWE-Preview · model_tag=https://huggingface.co/agentica-org/DeepSWE-Preview29 Jun 2025swebench.comT2 History
55#55o3OpenAI58.4%date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=o3-2025041626 Jul 2025swebench.comT2 History
56#56MultipleAnthropic58.2%date=2025-04-10 · board=Verified · system=Nemotron-CORTEXA · model_tag=NV-EmbedCode10 Apr 2025swebench.comT2 History
57#57Undisclosed56.6%date=2025-04-05 · board=Verified · system=SWE-Rizzo · submission=20250405_swe-rizzo_claude375 Apr 2025swebench.comT2 History
58#58Devstral Small (2512)Mistral AI56.4%date=2025-12-09 · board=Verified · system=mini-SWE-agent · model_tag=devstral-small-25129 Dec 2025swebench.comT2 History
59#59gpt-5-miniOpenAI56.2%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-mini-2025-08-0717 Feb 2026swebench.comT2 History
60#60GLM 4.6Z.ai (Zhipu AI)55.4%date=2025-12-01 · board=Verified · system=mini-SWE-agent · model_tag=glm-4.61 Dec 2025swebench.comT2 History
61#61qwen3-coder-480b-a35b-instructAlibaba Group55.4%date=2025-08-02 · board=Verified · system=mini-SWE-agent · model_tag=Qwen3-Coder-480B-A35B-Instruct2 Aug 2025swebench.comT2 History
62#62GLM 4.5Z.ai (Zhipu AI)54.2%date=2025-08-22 · board=Verified · system=mini-SWE-agent · model_tag=GLM-4.522 Aug 2025swebench.comT2 History
63#63Devstral 2Mistral AI53.8%date=2025-12-09 · board=Verified · system=mini-SWE-agent · model_tag=devstral-25129 Dec 2025swebench.comT2 History
64#64Frogboss 32B 251053.6%date=2025-11-10 · board=Verified · system=FrogBoss-32B-2510 · model_tag=FrogBoss-32B-251010 Nov 2025swebench.comT2 History
65#65Gemini 2.5 ProGoogle53.6%date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=gemini-2.5-pro26 Jul 2025swebench.comT2 History
66#66Kimi K2 0711Moonshot AI53.4%date=2025-08-04 · board=Verified · system=CodeSweep - SWE-agent · model_tag=kimi-k2-instruct4 Aug 2025swebench.comT2 History
67#67CodeAct v2.1 (claude-3-5-sonnet-20241022)Anthropic53%date=2024-10-29 · board=Verified · system=OpenHands · submission=20241029_OpenHands-CodeAct-2.1-sonnet-2024102229 Oct 2024swebench.comT2 History
68#68Claude 3.7 SonnetAnthropic52.8%date=2025-07-20 · board=Verified · system=mini-SWE-agent · model_tag=claude-3-7-sonnet-2025021920 Jul 2025swebench.comT2 History
69#69Qwen3 Coder 30B A3B InstructQwen52.2%date=2025-09-01 · board=Verified · system=EntroPO + R2E · model_tag=Qwen3-Coder-30B-A3B-Instruct1 Sept 2025swebench.comT2 History
70#70Qwen3 Coder 30B A3B InstructQwen51.6%date=2025-08-05 · board=Verified · system=OpenHands · model_tag=https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct5 Aug 2025swebench.comT2 History
71#71claude-35-sonnetAnthropic50.8%date=2024-12-02 · board=Verified · system=Agentless-1.5 · model_tag=claude-3-5-sonnet-202410222 Dec 2024swebench.comT2 History
72#72MultipleAnthropic48.6%date=2024-10-25 · board=Verified · system=Composio SWE-Kit · model_tag=claude-3-5-sonnet-2024102225 Oct 2024swebench.comT2 History
73#73MultipleAnthropic47.2%date=2024-11-06 · board=Verified · system=AppMap Navie v2 · model_tag=claude-3-5-sonnet-202410226 Nov 2024swebench.comT2 History
74#74TTS(Bo8)Qwen47%date=2025-06-16 · board=Verified · system=Skywork-SWE-32B · model_tag=Qwen2.5-Coder-32B-Instruct16 Jun 2025swebench.comT2 History
75#75Devstral Small 1.0Mistral AI46.8%date=2025-05-20 · board=Verified · system=OpenHands · model_tag=mistralai/Devstral-Small-250520 May 2025swebench.comT2 History
76#76claude-3-5-sonnet-20241022Anthropic46.2%date=2024-11-08 · board=Verified · system=AutoCodeRover-v2.0 · model_tag=claude-3-5-sonnet-202410228 Nov 2024swebench.comT2 History
77#77Co-PatcheR46%date=2025-05-28 · board=Verified · system=PatchPilot · submission=20250528_patchpilot_Co-PatcheR28 May 2025swebench.comT2 History
78#78o4-miniOpenAI45%date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=o4-mini-2025041626 Jul 2025swebench.comT2 History
79#79Frogmini 14B 251045%date=2025-11-10 · board=Verified · system=FrogMini-14B-2510 · model_tag=FrogMini-14B-251010 Nov 2025swebench.comT2 History
80#80Kimi K2 0711Moonshot AI43.8%date=2025-08-07 · board=Verified · system=mini-SWE-agent · model_tag=Kimi-K2-Instruct7 Aug 2025swebench.comT2 History
81#81o3-miniOpenAI42.4%date=2025-02-14 · board=Verified · system=Agentless Lite · model_tag=o3-mini14 Feb 2025swebench.comT2 History
82#82DeepSWE-Preview42.2%date=2025-06-29 · board=Verified · system=R2E-Gym · model_tag=https://huggingface.co/agentica-org/DeepSWE-Preview29 Jun 2025swebench.comT2 History
83#83DeepSeek V3 0324DeepSeek42%date=2025-08-06 · board=Verified · system=SWE-Exp · model_tag=DeepSeek-V3-03246 Aug 2025swebench.comT2 History
84#84Llama3-SWE-RL-70BMeta AI41.2%date=2025-02-26 · board=Verified · system=Agentless Mini · model_tag=Llama3-SWE-RL-70B26 Feb 2025swebench.comT2 History
85#85claude-35-sonnetAnthropic40.6%date=2024-10-16 · board=Verified · system=Composio SWEkit · model_tag=claude-3-5-sonnet-2024102216 Oct 2024swebench.comT2 History
86#86SWE-agent-LM-32BQwen40.2%date=2025-05-11 · board=Verified · system=SWE-agent · model_tag=Qwen 2.511 May 2025swebench.comT2 History
87#87gpt-4.1OpenAI39.58%date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=gpt-4.1-2025041426 Jul 2025swebench.comT2 History
88#88gpt-4oOpenAI38.8%date=2024-10-28 · board=Verified · system=Agentless-1.5 · model_tag=gpt-4o-2024-05-1328 Oct 2024swebench.comT2 History
89#89Devstral Small 1.1Mistral AI38%date=2025-07-25 · board=Verified · system=SWE-agent · model_tag=devstral-small-250725 Jul 2025swebench.comT2 History
90#90Qwen2.5 Coder 32B InstructQwen38%date=2025-06-16 · board=Verified · system=Skywork-SWE-32B · model_tag=Qwen2.5-Coder-32B-Instruct16 Jun 2025swebench.comT2 History
91#91gpt-5-nanoOpenAI34.8%date=2025-08-07 · board=Verified · system=mini-SWE-agent · model_tag=gpt-5-nano-2025-08-077 Aug 2025swebench.comT2 History
92#92claude-35-sonnetAnthropic33.6%date=2024-06-20 · board=Verified · system=SWE-agent · model_tag=claude-3-5-sonnet-2024102220 Jun 2024swebench.comT2 History
93#93Qwen2.5 (7B + 72B)Qwen32.8%date=2025-03-06 · board=Verified · system=SWE-Fixer · model_tag=Qwen 2.56 Mar 2025swebench.comT2 History
94#94Qwen2.5 (7B + 72B)Qwen30.2%date=2024-11-28 · board=Verified · system=SWE-Fixer · model_tag=Qwen 2.528 Nov 2024swebench.comT2 History
95#95Lingma SWE-GPT 72b (v0925)OpenAI28.8%date=2024-10-02 · board=Verified · system=Lingma Agent · submission=20241002_lingma-agent_lingma-swe-gpt-72b2 Oct 2024swebench.comT2 History
96#96Gemini 2.5 FlashGoogle28.73%date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=gemini-2.5-flash26 Jul 2025swebench.comT2 History
97#97gpt-4oOpenAI26.2%date=2024-06-15 · board=Verified · system=AppMap Navie · model_tag=gpt-4o-2024-05-1315 Jun 2024swebench.comT2 History
98#98gpt-oss-120bOpenAI26%date=2025-08-07 · board=Verified · system=mini-SWE-agent · model_tag=gpt-oss-120b7 Aug 2025swebench.comT2 History
99#99Lingma SWE-GPT 72b (v0918)OpenAI25%date=2024-09-18 · board=Verified · system=Lingma Agent · submission=20240918_lingma-agent_lingma-swe-gpt-72b18 Sept 2024swebench.comT2 History
100#100gpt-4.1-miniOpenAI23.94%date=2025-07-20 · board=Verified · system=mini-SWE-agent · model_tag=gpt-4.1-mini-2025041420 Jul 2025swebench.comT2 History

Scores are reported as published, with their evaluation configuration (harness, prompting, judge). The bar is relative to the best score on this page. Results with different configs are not directly comparable — see methodology.

The config filter matches a value inside each result's configuration (server-side, `config=` on the API). Chips are the values shared by several rows on the first page; per-model identifiers are not offered.