Skip to content
AI Atlas
BenchmarkActivecategory · coding

SWE-bench Verified

swebench.com

resolve real GitHub issues (500 human-validated instances)

quality57

Updated 8 h ago · first seen 11 Sept 2026

bench_01M293SPEGA1XMKH0SMTFTCK2R

Metric
resolved · %
Direction
Higher is better
Results
180 · 104 filtered
Leader
Claude Opus 4.5 79.2%

Score history · claude-3-5-sonnet-20241022 2 rows

0%20%40%60%Dec 24Jan 25
  • claude-3-5-sonnet-20241022
  • 51.6%date=2025-01-22 · board=Verified · system=AutoCodeRover-v2.1 · model_tag=claude-3-5-sonnet-2024102222 Jan 2025
  • 46.2%date=2024-11-08 · board=Verified · system=AutoCodeRover-v2.0 · model_tag=claude-3-5-sonnet-202410228 Nov 2024

Back to the full leaderboard

Leaderboard 104 current results · config contains “false”

Select models with +, then open Compare.

Leaderboard
#ModelScoreConfigEvaluatedSourceActions
1#1Claude Opus 4.5Anthropic79.2%date=2025-12-05 · board=Verified · system=Sonar Foundation Agent · model_tag=claude-opus-4-55 Dec 2025swebench.comT2 History
2#2Claude Opus 4.5Anthropic79.2%date=2025-12-15 · board=Verified · system=live-SWE-agent · model_tag=claude-opus-4-5-2025110115 Dec 2025swebench.comT2 History
3#3doubao-seed-codeByteDance78.8%date=2025-09-28 · board=Verified · system=TRAE · model_tag=Doubao-Seed-Code28 Sept 2025swebench.comT2 History
4#4Gemini 3 Pro PreviewGoogle77.4%date=2025-11-20 · board=Verified · system=live-SWE-agent · model_tag=gemini-3-pro-preview20 Nov 2025swebench.comT2 History
5#5MultipleAnthropic76.8%date=2025-09-02 · board=Verified · system=Atlassian Rovo Dev · model_tag=claude-sonnet-4-202505142 Sept 2025swebench.comT2 History
6#6Claude 4 SonnetAnthropic76.8%date=2025-08-04 · board=Verified · system=EPAM AI/Run Developer Agent · model_tag=claude-sonnet-4-202505144 Aug 2025swebench.comT2 History
7#7MultipleAnthropic76.4%date=2025-08-19 · board=Verified · system=ACoder · model_tag=claude-4-sonnet19 Aug 2025swebench.comT2 History
8#8MultipleAnthropic75.6%date=2025-09-01 · board=Verified · system=Warp · model_tag=gpt-51 Sept 2025swebench.comT2 History
9#9MultipleAnthropic75.2%date=2025-06-12 · board=Verified · system=TRAE · model_tag=claude-4-sonnet-2025052212 Jun 2025swebench.comT2 History
10#10Claude 4.5 SonnetAnthropic74.8%date=2025-11-03 · board=Verified · system=Sonar Foundation Agent · model_tag=claude-sonnet-4-53 Nov 2025swebench.comT2 History
11#11Claude Sonnet 4Anthropic74.8%date=2025-07-31 · board=Verified · system=Harness AI · model_tag=claude-sonnet-4-2025051431 Jul 2025swebench.comT2 History
12#12MultipleAnthropic74.6%date=2025-09-15 · board=Verified · system=JoyCode · model_tag=claude-4-sonnet15 Sept 2025swebench.comT2 History
13#13Claude 4 SonnetAnthropic74.6%date=2025-07-20 · board=Verified · system=Lingxi-v1.5 · model_tag=claude-4-sonnet-2025051420 Jul 2025swebench.comT2 History
14#14gpt-5OpenAI74.4%date=2025-10-15 · board=Verified · system=Prometheus-v1.2.1 · model_tag=gpt-5-2025-08-0715 Oct 2025swebench.comT2 History
15#15MultipleAnthropic74.4%date=2025-06-03 · board=Verified · system=Refact.ai Agent · model_tag=claude-4-sonnet3 Jun 2025swebench.comT2 History
16#16MultipleAnthropic73.8%date=2025-11-03 · board=Verified · system=Salesforce AI Research SAGE · model_tag=claude-sonnet-4.53 Nov 2025swebench.comT2 History
17#17claude-4-opusAnthropic73.2%date=2025-05-22 · board=Verified · system=Tools · model_tag=claude-4-opus-2025051422 May 2025swebench.comT2 History
18#18MultipleAnthropic73%date=2025-10-21 · board=Verified · system=Salesforce AI Research SAGE · model_tag=claude-sonnet-4.521 Oct 2025swebench.comT2 History
19#19Claude 4 SonnetAnthropic72.4%date=2025-05-22 · board=Verified · system=Tools · model_tag=claude-4-sonnet-2025051422 May 2025swebench.comT2 History
20#20Claude 4 SonnetAnthropic71.2%date=2025-07-10 · board=Verified · system=Bloop · model_tag=claude-4-sonnet-2025051410 Jul 2025swebench.comT2 History
21#21Kimi K2 0711Moonshot AI71.2%date=2025-10-14 · board=Verified · system=Lingxi v1.5 · model_tag=kimi-k2-0905-preview14 Oct 2025swebench.comT2 History
22#22gpt-5OpenAI71.2%date=2025-09-29 · board=Verified · system=Prometheus-v1.2 · model_tag=gpt-5-2025-08-0729 Sept 2025swebench.comT2 History
23#23MultipleAnthropic71.2%date=2025-07-15 · board=Verified · system=Qodo Command · model_tag=claude-sonnet-4-2025051415 Jul 2025swebench.comT2 History
24#24MultipleAnthropic71%date=2025-06-23 · board=Verified · system=Warp · model_tag=claude-sonnet-4-2025051423 Jun 2025swebench.comT2 History
25#25Undisclosed70.6%date=2025-05-19 · board=Verified · system=TRAE · submission=20250519_trae19 May 2025swebench.comT2 History
26#26Undisclosed70.4%date=2025-06-10 · board=Verified · system=Augment Agent v1 · submission=20250610_augment_agent_v110 Jun 2025swebench.comT2 History
27#27MultipleAnthropic70.4%date=2025-05-15 · board=Verified · system=Refact.ai Agent · model_tag=claude-3-7-sonnet-2025021915 May 2025swebench.comT2 History
28#28MultipleAnthropic70.2%date=2025-05-19 · board=Verified · system=devlo · model_tag=claude-3-7-sonnet-2025021919 May 2025swebench.comT2 History
29#29MultipleAnthropic70%date=2025-04-30 · board=Verified · system=Zencoder · model_tag=claude-3-7-sonnet-2025021930 Apr 2025swebench.comT2 History
30#30qwen3-coder-480b-a35b-instructAlibaba Group69.6%date=2025-08-05 · board=Verified · system=OpenHands · model_tag=https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct5 Aug 2025swebench.comT2 History
31#31MultipleAnthropic68.2%date=2025-05-16 · board=Verified · system=Nemotron-CORTEXA · model_tag=NV-EmbedCode16 May 2025swebench.comT2 History
32#32GLM 4.6Z.ai (Zhipu AI)68.2%date=2025-09-30 · board=Verified · system=Undisclosed · model_tag=https://huggingface.co/zai-org/GLM-4.630 Sept 2025swebench.comT2 History
33#33Claude 3.7 SonnetAnthropic66.4%date=2025-05-14 · board=Verified · system=Aime-coder v1 · model_tag=claude-3-7-sonnet-2025021914 May 2025swebench.comT2 History
34#34Undisclosed65.4%date=2025-04-05 · board=Verified · system=Amazon Q Developer Agent · submission=20250405_amazon-q-developer-agent-20250405-dev5 Apr 2025swebench.comT2 History
35#35Undisclosed65.4%date=2025-03-16 · board=Verified · system=Augment Agent v0 · submission=20250316_augment_agent_v016 Mar 2025swebench.comT2 History
36#36o1 PreviewOpenAI64.6%date=2025-01-17 · board=Verified · system=W&B Programmer O1 crosscheck5 · model_tag=o1-preview17 Jan 2025swebench.comT2 History
37#37o4-miniOpenAI64.6%date=2025-05-03 · board=Verified · system=PatchPilot-v1.1 · model_tag=o4-mini3 May 2025swebench.comT2 History
38#38GLM 4.5Z.ai (Zhipu AI)64.2%date=2025-07-28 · board=Verified · system=Undisclosed · model_tag=https://huggingface.co/zai-org/GLM-4.528 Jul 2025swebench.comT2 History
39#39claude-35-sonnetAnthropic63.4%date=2025-02-06 · board=Verified · system=AgentScope · model_tag=claude-3-5-sonnet-202410226 Feb 2025swebench.comT2 History
40#40Claude 3.7 SonnetAnthropic63.2%date=2025-02-24 · board=Verified · system=Tools · submission=20250224_tools_claude-3-7-sonnet24 Feb 2025swebench.comT2 History
41#41claude-35-sonnetAnthropic62.8%date=2025-02-28 · board=Verified · system=EPAM AI/Run Developer Agent · model_tag=claude-3-5-sonnet-2024102228 Feb 2025swebench.comT2 History
42#42Undisclosed62.8%date=2025-01-10 · board=Verified · system=Blackbox AI Agent · submission=20250110_blackboxai_agent_v1.110 Jan 2025swebench.comT2 History
43#43swe-searchAnthropic62.2%date=2024-12-21 · board=Verified · system=CodeStory Midwit Agent · model_tag=claude-3-5-sonnet-2024102221 Dec 2024swebench.comT2 History
44#44Qwen3 Coder 30B A3B InstructQwen60.4%date=2025-09-01 · board=Verified · system=EntroPO + R2E · model_tag=Qwen3-Coder-30B-A3B-Instruct1 Sept 2025swebench.comT2 History
45#45claude-35-sonnetAnthropic60.2%date=2025-01-10 · board=Verified · system=Learn-by-interact · model_tag=claude-3-5-sonnet-2024102210 Jan 2025swebench.comT2 History
46#46TTS(Bo16)OpenAI58.8%date=2025-06-29 · board=Verified · system=DeepSWE-Preview · model_tag=https://huggingface.co/agentica-org/DeepSWE-Preview29 Jun 2025swebench.comT2 History
47#47Undisclosed58.2%date=2024-12-13 · board=Verified · system=devlo · submission=20241213_devlo13 Dec 2024swebench.comT2 History
48#48MultipleAnthropic58.2%date=2025-04-10 · board=Verified · system=Nemotron-CORTEXA · model_tag=NV-EmbedCode10 Apr 2025swebench.comT2 History
49#49MultipleAnthropic57.2%date=2024-12-23 · board=Verified · system=Emergent E1 · model_tag=claude-3-5-sonnet-2024102223 Dec 2024swebench.comT2 History
50#50Claude Sonnet 4Anthropic57%date=2025-09-24 · board=Verified · system=Artemis Agent v2 · model_tag=claude-sonnet-4-2025051424 Sept 2025swebench.comT2 History
51#51Undisclosed57%date=2024-12-08 · board=Verified · system=Gru · submission=20241208_gru8 Dec 2024swebench.comT2 History
52#52Undisclosed56.6%date=2025-04-05 · board=Verified · system=SWE-Rizzo · submission=20250405_swe-rizzo_claude375 Apr 2025swebench.comT2 History
53#53claude-35-sonnetAnthropic55.4%date=2024-12-12 · board=Verified · system=EPAM AI/Run Developer Agent · model_tag=claude-3-5-sonnet-2024102212 Dec 2024swebench.comT2 History
54#54Undisclosed55%date=2024-12-02 · board=Verified · system=Amazon Q Developer Agent · submission=20241202_amazon-q-developer-agent-20241202-dev2 Dec 2024swebench.comT2 History
55#55claude-35-sonnetAnthropic54.2%date=2024-11-08 · board=Verified · system=devlo · model_tag=claude-3-5-sonnet-202410228 Nov 2024swebench.comT2 History
56#56Frogboss 32B 251053.6%date=2025-11-10 · board=Verified · system=FrogBoss-32B-2510 · model_tag=FrogBoss-32B-251010 Nov 2025swebench.comT2 History
57#57Kimi K2 0711Moonshot AI53.4%date=2025-08-04 · board=Verified · system=CodeSweep - SWE-agent · model_tag=kimi-k2-instruct4 Aug 2025swebench.comT2 History
58#58Undisclosed53.2%date=2025-01-20 · board=Verified · system=Bracket.sh · submission=20250120_Bracket20 Jan 2025swebench.comT2 History
59#59Gemini 2.0 Flash (v20241212-experimental)Google DeepMind52.2%date=2024-12-12 · board=Verified · system=Google Jules · submission=20241212_google_jules_gemini_2.0_flash_experimental12 Dec 2024swebench.comT2 History
60#60Qwen3 Coder 30B A3B InstructQwen52.2%date=2025-09-01 · board=Verified · system=EntroPO + R2E · model_tag=Qwen3-Coder-30B-A3B-Instruct1 Sept 2025swebench.comT2 History
61#61claude-35-sonnetAnthropic51.8%date=2024-11-25 · board=Verified · system=Engine Labs · model_tag=claude-3-5-sonnet-2024102225 Nov 2024swebench.comT2 History
62#62claude-3-5-sonnet-20241022Anthropic51.6%date=2025-01-22 · board=Verified · system=AutoCodeRover-v2.1 · model_tag=claude-3-5-sonnet-2024102222 Jan 2025swebench.comT2 History
63#63Qwen3 Coder 30B A3B InstructQwen51.6%date=2025-08-05 · board=Verified · system=OpenHands · model_tag=https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct5 Aug 2025swebench.comT2 History
64#64claude-35-sonnetAnthropic50.8%date=2024-12-02 · board=Verified · system=Agentless-1.5 · model_tag=claude-3-5-sonnet-202410222 Dec 2024swebench.comT2 History
65#65Undisclosed50%date=2024-10-28 · board=Verified · system=Solver · submission=20241028_solver28 Oct 2024swebench.comT2 History
66#66Undisclosed50%date=2024-11-25 · board=Verified · system=Bytedance MarsCode Agent · submission=20241125_marscode-agent-dev25 Nov 2024swebench.comT2 History
67#67MultipleAnthropic49.2%date=2024-11-05 · board=Verified · system=nFactorial · model_tag=claude-3-5-sonnet-202410225 Nov 2024swebench.comT2 History
68#68claude-35-sonnetAnthropic49%date=2024-10-22 · board=Verified · system=Tools · model_tag=claude-3-5-sonnet-2024102222 Oct 2024swebench.comT2 History
69#69MultipleAnthropic46.6%date=2024-10-23 · board=Verified · system=Emergent E1 · model_tag=claude-3-5-sonnet-2024102223 Oct 2024swebench.comT2 History
70#70claude-3-5-sonnet-20241022Anthropic46.2%date=2024-11-08 · board=Verified · system=AutoCodeRover-v2.0 · model_tag=claude-3-5-sonnet-202410228 Nov 2024swebench.comT2 History
71#71Co-PatcheR46%date=2025-05-28 · board=Verified · system=PatchPilot · submission=20250528_patchpilot_Co-PatcheR28 May 2025swebench.comT2 History
72#72Undisclosed45.4%date=2024-09-24 · board=Verified · system=Solver · submission=20240924_solver24 Sept 2024swebench.comT2 History
73#73Undisclosed45.2%date=2024-08-24 · board=Verified · system=Gru · submission=20240824_gru24 Aug 2024swebench.comT2 History
74#74Frogmini 14B 251045%date=2025-11-10 · board=Verified · system=FrogMini-14B-2510 · model_tag=FrogMini-14B-251010 Nov 2025swebench.comT2 History
75#75gemini-2.0-flash-expGoogle44.2%date=2025-01-18 · board=Verified · system=CodeShellAgent · model_tag=gemini-2.0-flash-exp18 Jan 2025swebench.comT2 History
76#76Undisclosed43.6%date=2024-09-20 · board=Verified · system=Solver · submission=20240920_solver20 Sept 2024swebench.comT2 History
77#77Amazon.nova Premier v1:042.4%date=2025-05-27 · board=Verified · system=Amazon Nova Premier 1.0 · model_tag=amazon.nova-premier-v1:027 May 2025swebench.comT2 History
78#78DeepSWE-Preview42.2%date=2025-06-29 · board=Verified · system=R2E-Gym · model_tag=https://huggingface.co/agentica-org/DeepSWE-Preview29 Jun 2025swebench.comT2 History
79#79DeepSeek V3 0324DeepSeek42%date=2025-08-06 · board=Verified · system=SWE-Exp · model_tag=DeepSeek-V3-03246 Aug 2025swebench.comT2 History
80#80Undisclosed41.6%date=2025-01-12 · board=Verified · system=ugaiforge · submission=20250112_ugaiforge12 Jan 2025swebench.comT2 History
81#81MultipleAnthropic41.6%date=2024-10-30 · board=Verified · system=nFactorial · model_tag=claude-3-5-sonnet-2024102230 Oct 2024swebench.comT2 History
82#82Llama3-SWE-RL-70BMeta AI41.2%date=2025-02-26 · board=Verified · system=Agentless Mini · model_tag=Llama3-SWE-RL-70B26 Feb 2025swebench.comT2 History
83#83Undisclosed40.6%date=2024-08-20 · board=Verified · system=Honeycomb · submission=20240820_honeycomb20 Aug 2024swebench.comT2 History
84#84MultipleAnthropic40.6%date=2024-11-13 · board=Verified · system=Nebius AI · model_tag=Llama 3.113 Nov 2024swebench.comT2 History
85#85Claude Haiku 3.5Anthropic40.6%date=2024-10-22 · board=Verified · system=Tools · model_tag=claude-3-haiku-2024030722 Oct 2024swebench.comT2 History
86#86claude-35-sonnetAnthropic40.6%date=2024-10-16 · board=Verified · system=Composio SWEkit · model_tag=claude-3-5-sonnet-2024102216 Oct 2024swebench.comT2 History
87#87claude-35-sonnetAnthropic39.6%date=2024-10-29 · board=Verified · system=EPAM AI/Run Developer Agent · model_tag=claude-3-5-sonnet-2024102229 Oct 2024swebench.comT2 History
88#88Undisclosed38.8%date=2024-07-21 · board=Verified · system=Amazon Q Developer Agent · submission=20240721_amazon-q-developer-agent-20240719-dev21 Jul 2024swebench.comT2 History
89#89gpt-4oOpenAI38.8%date=2024-10-28 · board=Verified · system=Agentless-1.5 · model_tag=gpt-4o-2024-05-1328 Oct 2024swebench.comT2 History
90#90gpt-4oOpenAI38.4%date=2024-06-28 · board=Verified · system=AutoCodeRover · model_tag=gpt-4o-2024-05-1328 Jun 2024swebench.comT2 History
91#91Undisclosed37%date=2024-06-17 · board=Verified · system=Factory Code Droid · submission=20240617_factory_code_droid17 Jun 2024swebench.comT2 History
92#92gpt-4oOpenAI32.6%date=2024-06-12 · board=Verified · system=MASAI · model_tag=gpt-4o-2024-08-0612 Jun 2024swebench.comT2 History
93#93Undisclosed32%date=2024-11-20 · board=Verified · system=Artemis Agent v1 · submission=20241120_artemis_agent20 Nov 2024swebench.comT2 History
94#94Undisclosed31.6%date=2024-10-07 · board=Verified · system=nFactorial · submission=20241007_nfactorial7 Oct 2024swebench.comT2 History
95#95Qwen2.5 (7B + 72B)Qwen30.2%date=2024-11-28 · board=Verified · system=SWE-Fixer · model_tag=Qwen 2.528 Nov 2024swebench.comT2 History
96#96Lingma SWE-GPT 72b (v0925)OpenAI28.8%date=2024-10-02 · board=Verified · system=Lingma Agent · submission=20241002_lingma-agent_lingma-swe-gpt-72b2 Oct 2024swebench.comT2 History
97#97gpt-4oOpenAI27%date=2024-10-16 · board=Verified · system=EPAM AI/Run Developer Agent · model_tag=gpt-4o-2024-08-0616 Oct 2024swebench.comT2 History
98#98Undisclosed25.8%date=2024-10-01 · board=Verified · system=nFactorial · submission=20241001_nfactorial1 Oct 2024swebench.comT2 History
99#99Undisclosed25.6%date=2024-05-09 · board=Verified · system=Amazon Q Developer Agent · submission=20240509_amazon-q-developer-agent-20240430-dev9 May 2024swebench.comT2 History
100#100Lingma SWE-GPT 72b (v0918)OpenAI25%date=2024-09-18 · board=Verified · system=Lingma Agent · submission=20240918_lingma-agent_lingma-swe-gpt-72b18 Sept 2024swebench.comT2 History

Scores are reported as published, with their evaluation configuration (harness, prompting, judge). The bar is relative to the best score on this page. Results with different configs are not directly comparable — see methodology.

The config filter matches a value inside each result's configuration (server-side, `config=` on the API). Chips are the values shared by several rows on the first page; per-model identifiers are not offered.