Updated 8 h ago · first seen 11 Sept 2026
bench_01M293SPEGA1XMKH0SMTFTCK2R
- Metric
- resolved · %
- Direction
- Higher is better
- Results
- 180 · 10 filtered
- Leader
- Claude Opus 4.5 79.2%
Score history · Claude 4.5 Sonnet 3 rows
- Claude 4.5 Sonnet
- 71.4%date=2026-02-17 · board=Verified · system=mini-SWE-agent · model_tag=claude-sonnet-4-5-2025092917 Feb 2026
- 74.8%date=2025-11-03 · board=Verified · system=Sonar Foundation Agent · model_tag=claude-sonnet-4-53 Nov 2025
- 70.6%date=2025-09-29 · board=Verified · system=mini-SWE-agent · model_tag=claude-sonnet-4-5-2025092929 Sept 2025
Leaderboard 10 current results · config contains “OpenHands”
Select models with +, then open Compare.
| # | Model | Score | Config | Evaluated | Source | Actions |
|---|---|---|---|---|---|---|
| #1MultipleAnthropic | 73.8% | date=2025-11-03 · board=Verified · system=Salesforce AI Research SAGE · model_tag=claude-sonnet-4.5 | 3 Nov 2025 | swebench.comT2 | History | |
| #2gpt-5OpenAI | 71.8% | date=2025-08-07 · board=Verified · system=OpenHands · model_tag=openai/gpt-5-2025-08-07 | 7 Aug 2025 | swebench.comT2 | History | |
| #3Claude 4 SonnetAnthropic | 70.4% | date=2025-05-24 · board=Verified · system=OpenHands · model_tag=claude-4-sonnet-20250514 | 24 May 2025 | swebench.comT2 | History | |
| #4qwen3-coder-480b-a35b-instructAlibaba Group | 69.6% | date=2025-08-05 · board=Verified · system=OpenHands · model_tag=https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct | 5 Aug 2025 | swebench.comT2 | History | |
| #5Undisclosed | 65.8% | date=2025-04-15 · board=Verified · system=OpenHands · submission=20250415_openhands | 15 Apr 2025 | swebench.comT2 | History | |
| #6Kimi K2 0711Moonshot AI | 65.4% | date=2025-07-16 · board=Verified · system=OpenHands · model_tag=moonshot/kimi-k2-0711-preview | 16 Jul 2025 | swebench.comT2 | History | |
| #74x Scaled | 60.8% | date=2025-02-03 · board=Verified · system=OpenHands · submission=20250203_openhands_4x_scaled | 3 Feb 2025 | swebench.comT2 | History | |
| #8CodeAct v2.1 (claude-3-5-sonnet-20241022)Anthropic | 53% | date=2024-10-29 · board=Verified · system=OpenHands · submission=20241029_OpenHands-CodeAct-2.1-sonnet-20241022 | 29 Oct 2024 | swebench.comT2 | History | |
| #9Qwen3 Coder 30B A3B InstructQwen | 51.6% | date=2025-08-05 · board=Verified · system=OpenHands · model_tag=https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct | 5 Aug 2025 | swebench.comT2 | History | |
| #10Devstral Small 1.0Mistral AI | 46.8% | date=2025-05-20 · board=Verified · system=OpenHands · model_tag=mistralai/Devstral-Small-2505 | 20 May 2025 | swebench.comT2 | History |
10 results
Scores are reported as published, with their evaluation configuration (harness, prompting, judge). The bar is relative to the best score on this page. Results with different configs are not directly comparable — see methodology.
The config filter matches a value inside each result's configuration (server-side, `config=` on the API). Chips are the values shared by several rows on the first page; per-model identifiers are not offered.
Definition
- Category
- coding
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
- Task
- resolve real GitHub issues (500 human-validated instances)
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
- Metric
- resolved · %
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
- Direction
- Higher is better
- Paper
- https://arxiv.org/abs/2310.06770
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
- Known limitations
- Scaffold/agent dependent; results are not comparable across harnesses.
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
- Website
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Each value shows its source, tier and observation time. Missing rows mean no source stated them. How results are recorded →
- Evaluated models
- Claude Opus 4.6, doubao-seed-code, gemini-3-flash, MiniMax M2.5, Claude Opus 4.5, Gemini 3 Pro Preview, GLM 5, GPT-5.2-Codex +74(82 total)
As of
Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.
Claim history
Categorycategory1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| coding | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Known limitationsknown_limitations1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| Scaffold/agent dependent; results are not comparable across harnesses. | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Metricmetric1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| resolved | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Paperpaper1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://arxiv.org/abs/2310.06770 | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Tasktask1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| resolve real GitHub issues (500 human-validated instances) | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Unitunit1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| % | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Websitewebsite1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://www.swebench.com | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →
| Source | Document | Type | Tier | Last observed | Snapshots |
|---|---|---|---|---|---|
| SWE-bench leaderboards | swebench.com/ | leaderboard | T2· Quality secondary | 6 h ago | 1 |
Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.