Score history · claude-35-sonnet 11 rows

claude-35-sonnet

62.8%date=2025-02-28 · board=Verified · system=EPAM AI/Run Developer Agent · model_tag=claude-3-5-sonnet-2024102228 Feb 2025
63.4%date=2025-02-06 · board=Verified · system=AgentScope · model_tag=claude-3-5-sonnet-202410226 Feb 2025
60.2%date=2025-01-10 · board=Verified · system=Learn-by-interact · model_tag=claude-3-5-sonnet-2024102210 Jan 2025
55.4%date=2024-12-12 · board=Verified · system=EPAM AI/Run Developer Agent · model_tag=claude-3-5-sonnet-2024102212 Dec 2024
50.8%date=2024-12-02 · board=Verified · system=Agentless-1.5 · model_tag=claude-3-5-sonnet-202410222 Dec 2024
51.8%date=2024-11-25 · board=Verified · system=Engine Labs · model_tag=claude-3-5-sonnet-2024102225 Nov 2024
54.2%date=2024-11-08 · board=Verified · system=devlo · model_tag=claude-3-5-sonnet-202410228 Nov 2024
39.6%date=2024-10-29 · board=Verified · system=EPAM AI/Run Developer Agent · model_tag=claude-3-5-sonnet-2024102229 Oct 2024
49%date=2024-10-22 · board=Verified · system=Tools · model_tag=claude-3-5-sonnet-2024102222 Oct 2024
40.6%date=2024-10-16 · board=Verified · system=Composio SWEkit · model_tag=claude-3-5-sonnet-2024102216 Oct 2024
33.6%date=2024-06-20 · board=Verified · system=SWE-agent · model_tag=claude-3-5-sonnet-2024102220 Jun 2024

Leaderboard 2 current results · config contains “Warp”

Select models with +, then open Compare.

Leaderboard
#	Model	Score	Config	Evaluated	Source	Actions
1	#1MultipleAnthropic	75.6%	date=2025-09-01 · board=Verified · system=Warp · model_tag=gpt-5	1 Sept 2025	swebench.comT2	History
2	#2MultipleAnthropic	71%	date=2025-06-23 · board=Verified · system=Warp · model_tag=claude-sonnet-4-20250514	23 Jun 2025	swebench.comT2	History

2 results

Scores are reported as published, with their evaluation configuration (harness, prompting, judge). The bar is relative to the best score on this page. Results with different configs are not directly comparable — see methodology.

The config filter matches a value inside each result's configuration (server-side, `config=` on the API). Chips are the values shared by several rows on the first page; per-model identifiers are not offered.

Definition

Category: coding
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Task: resolve real GitHub issues (500 human-validated instances)
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Metric: resolved · %
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Direction: Higher is better
Paper: https://arxiv.org/abs/2310.06770
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Known limitations: Scaffold/agent dependent; results are not comparable across harnesses.
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Website: swebench.com
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium

Each value shows its source, tier and observation time. Missing rows mean no source stated them. How results are recorded →

Evaluated models: Claude Opus 4.6, doubao-seed-code, gemini-3-flash, MiniMax M2.5, Claude Opus 4.5, Gemini 3 Pro Preview, GLM 5, GPT-5.2-Codex +74(82 total)

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history

7 claims · 7 properties

Categorycategory1

Claim history for Category
Value	Valid from → to	Status	Source	Confidence	Extractor
coding	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Known limitationsknown_limitations1

Claim history for Known limitations
Value	Valid from → to	Status	Source	Confidence	Extractor
Scaffold/agent dependent; results are not comparable across harnesses.	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Metricmetric1

Claim history for Metric
Value	Valid from → to	Status	Source	Confidence	Extractor
resolved	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Paperpaper1

Claim history for Paper
Value	Valid from → to	Status	Source	Confidence	Extractor
https://arxiv.org/abs/2310.06770	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Tasktask1

Claim history for Task
Value	Valid from → to	Status	Source	Confidence	Extractor
resolve real GitHub issues (500 human-validated instances)	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Unitunit1

Claim history for Unit
Value	Valid from → to	Status	Source	Confidence	Extractor
%	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Websitewebsite1

Claim history for Website
Value	Valid from → to	Status	Source	Confidence	Extractor
https://www.swebench.com	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
New benchmarkBenchmarkSWE-bench Verified
New benchmark: SWE-bench Verified
registry
swebench.com

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
SWE-bench leaderboards	swebench.com/	leaderboard	T2· Quality secondary	6 h ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.