BenchmarkActivecategory · codingfamily · swe-bench · variant Verified

SWE-bench Verified

swebench.com

resolve real GitHub issues (500 human-validated instances)

quality57

Updated 12 min ago · first seen 11 Sept 2026

Metric: resolved · % ↑
Current results: 158
Models: 82
Current leader: Claude Opus 4.5 76.8%

Filters

Comparability group

Trust level

Views

Score history · Claude 4 Sonnet 8 rows

Claude 4 Sonnet

76.8%date=2025-08-04 · board=Verified · system=EPAM AI/Run Developer Agent · model_tag=claude-sonnet-4-202505144 Aug 2025
64.93%date=2025-07-26 · board=Verified · system=mini-SWE-agent · model_tag=claude-4-sonnet-2025051426 Jul 2025
74.6%date=2025-07-20 · board=Verified · system=Lingxi-v1.5 · model_tag=claude-4-sonnet-2025051420 Jul 2025
71.2%date=2025-07-10 · board=Verified · system=Bloop · model_tag=claude-4-sonnet-2025051410 Jul 2025
70.8%date=2025-06-11 · board=Verified · system=Moatless Tools · model_tag=claude-4-sonnet-2025051411 Jun 2025
70.4%date=2025-05-24 · board=Verified · system=OpenHands · model_tag=claude-4-sonnet-2025051424 May 2025
66.6%date=2025-05-22 · board=Verified · system=SWE-agent · model_tag=claude-4-sonnet-2025051422 May 2025
72.4%date=2025-05-22 · board=Verified · system=Tools · model_tag=claude-4-sonnet-2025051422 May 2025

Back to the leaderboard

Frontier over time · resolved · board=Verified · system=mini-SWE-agent

76.8%Claude Opus 4.5 Anthropic Official board17 Feb 2026
74.4%Claude Opus 4.5 Anthropic Official board24 Nov 2025
74.2%Gemini 3 Pro Preview Google Official board18 Nov 2025
70.6%Claude 4.5 Sonnet Anthropic Official board29 Sept 2025
67.6%claude-4-opus Anthropic Official board2 Aug 2025
64.9%Claude 4 Sonnet Anthropic Official board26 Jul 2025
52.8%Claude 3.7 Sonnet Anthropic Official board20 Jul 2025

Includes closed rows (history). A point is emitted whenever a result beats every earlier result of the same group, ordered by evaluated_at when the source publishes it, else observed_at.

Leaderboard 42 models

Select models with +, then Compare.

No result in this group with these filters

Relax the trust / organization filters or pick another comparability group.

Definition

Task: Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 10 h agomedium
Metric: Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 10 h agomedium
Metric range: Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 12 min agomedium
Variant: Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 12 min agomedium
Harness: Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 12 min agomedium
Comparability note: Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 12 min agomedium
Known limitations: Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 10 h agomedium
Paper: Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 10 h agomedium
Website: swebench.com
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 10 h agomedium

Active group

resolved · board=Verified · system=mini-SWE-agent

42 models · 42 current rows

Comparability

Comparable same task and conditions · Partially comparable same task, conditions differ (effort, temperature, judge) · Not comparable different variant or metric

Also known as

SWE-bench, SWE-bench Verified

slug swe-bench-verified · bench_01M293SPEGA1XMKH0SMTFTCK2R

Rows are grouped by comparability group = canonical metric × config_key (hash of the task-defining configuration keys: variant, evaluator, harness, shots, pass regime…). A leaderboard shows one row per canonical model: its best current row inside the group. Reasoning effort, temperature or judge differences keep rows in the same group but mark them partially comparable. delta_rank compares with the ranking built from the closed (previous) rows of the same group.