Skip to content
AI Atlas
BenchmarkActivecategory · codingfamily · swe-bench · variant Verified

SWE-bench Verified

swebench.com

resolve real GitHub issues (500 human-validated instances)

data quality57

Updated 6 h ago · first seen 11 Sept 2026

Metric
resolved · % ↑
Current results
158
Models
82
Current leader
Claude 4 Sonnet 66.6%

Score history · GPT-4 (1106) 2 rows

Not enough history to chart — 2 observations, all dated 2 Apr 2024. Rows under different configurations count separately; the list below shows each one.

  • 2.8%date=2024-04-02 · board=Verified · system=RAG baseline · model_tag=gpt-4-1106-preview2 Apr 2024
  • 22.4%date=2024-04-02 · board=Verified · system=SWE-agent · model_tag=gpt-4-1106-preview2 Apr 2024

Back to the leaderboard

Frontier over time · resolved · board=Verified · system=SWE-agent

Frontier of SWE-bench Verified0%20%40%60%80%Jul 24Oct 24Jan 25Apr 25
  1. 66.6%Claude 4 Sonnet Anthropic Official board22 May 2025
  2. 62.4%Claude 3.7 Sonnet w/ Review Heavy Anthropic Official board25 Feb 2025
  3. 33.6%claude-35-sonnet Anthropic Official board20 Jun 2024
  4. 22.4%GPT-4 (1106) OpenAI Official board2 Apr 2024

Includes closed rows (history). A point is emitted whenever a result beats every earlier result of the same group, ordered by evaluated_at when the source publishes it, else observed_at.

Leaderboard 8 models

Select models with +, then Compare.

Leaderboard
#ModelScoreTrustConfigurationvs leaderEvaluatedSourceActions
1Claude 4 SonnetAnthropic · Claude 466.6%Official boardgroup defaultsleader22 May 2025swebench.comT2History
2Claude 3.7 Sonnet w/ Review HeavyAnthropic62.4%Official boardgroup defaultsPartially comparable-4.20 pt25 Feb 2025swebench.comT2History
3SWE-agent-LM-32BQwen40.2%Official boardgroup defaultsPartially comparable-26.4 pt11 May 2025swebench.comT2History
4Devstral Small 1.1Mistral AI · Devstral38%Official boardgroup defaultsPartially comparable-28.6 pt25 Jul 2025swebench.comT2History
5claude-35-sonnetClosedAnthropic · Claude 3533.6%Official boardgroup defaultsPartially comparable-33.0 pt20 Jun 2024swebench.comT2History
6gpt-4oClosedOpenAI · GPT 423.2%Official boardgroup defaultsPartially comparable-43.4 pt28 Jul 2024swebench.comT2History
7GPT-4 (1106)OpenAI · GPT 422.4%Official boardgroup defaultsPartially comparable-44.2 pt2 Apr 2024swebench.comT2History
8claude-3-opusClosedAnthropic · Claude 315.8%Official boardgroup defaultsPartially comparable-50.8 pt2 Apr 2024swebench.comT2History

8 results

One row per canonical model — its best current row inside this comparability group (effort variants are folded into the model). Bars are relative to the page's best score. “vs leader” reads comparability: partially comparable = same task, conditions differ (reasoning effort, temperature, judge). Rules →