Updated 3 h ago · first seen 11 Sept 2026
bench_01M293SPEMVE6W55699099NES3
- Metric
- pass rate (2 attempts) · %
- Direction
- Higher is better
- Results
- 138 · 10 filtered
Leaderboard 10 current results · config contains “224”
Select models with +, then open Compare.
| # | Model | Score | Config | Evaluated | Source | Actions |
|---|---|---|---|---|---|---|
| #1o3 (high) + gpt-4.1 | 100% | date=2025-06-27 · command=aider --model o3 · dirname=2025-06-27-23-53-57--o3-mini-high-diff-arch · versions=0.85.1.dev | 27 Jun 2025 | aider.chatT2 | History | |
| #2GPT-4.5 PreviewOpenAI | 97.3% | date=2025-02-27 · command=aider --model openai/gpt-4.5-preview · dirname=2025-02-27-20-26-15--gpt45-diff3 · versions=0.75.2.dev | 27 Feb 2025 | aider.chatT2 | History | |
| #3R1 0528DeepSeek | 94.6% | date=2025-06-06 · command=aider --model deepseek/deepseek-reasoner · dirname=2025-06-06-16-47-07--r1-diff · versions=0.84.1.dev | 6 Jun 2025 | aider.chatT2 | History | |
| #4o3 Mini HighOpenAI | 93.3% | date=2025-01-31 · command=aider --model o3-mini --reasoning-effort high · dirname=2025-01-31-20-42-47--o3-mini-diff-high · versions=0.72.4.dev | 31 Jan 2025 | aider.chatT2 | History | |
| #5o1-2024-12-17 (high) | 91.5% | date=2024-12-21 · command=aider --model openrouter/openai/o1 · dirname=2024-12-21-19-23-03--polyglot-o1-hard-diff · versions=0.69.2.dev | 21 Dec 2024 | aider.chatT2 | History | |
| #6o3 (high) + gpt-4.1 | 78.2% | date=2025-06-27 · command=aider --model o3 · dirname=2025-06-27-23-53-57--o3-mini-high-diff-arch · versions=0.85.1.dev | 27 Jun 2025 | aider.chatT2 | History | |
| #7R1 0528DeepSeek | 71.4% | date=2025-06-06 · command=aider --model deepseek/deepseek-reasoner · dirname=2025-06-06-16-47-07--r1-diff · versions=0.84.1.dev | 6 Jun 2025 | aider.chatT2 | History | |
| #8o1-2024-12-17 (high) | 61.7% | date=2024-12-21 · command=aider --model openrouter/openai/o1 · dirname=2024-12-21-19-23-03--polyglot-o1-hard-diff · versions=0.69.2.dev | 21 Dec 2024 | aider.chatT2 | History | |
| #9o3 Mini HighOpenAI | 60.4% | date=2025-01-31 · command=aider --model o3-mini --reasoning-effort high · dirname=2025-01-31-20-42-47--o3-mini-diff-high · versions=0.72.4.dev | 31 Jan 2025 | aider.chatT2 | History | |
| #10GPT-4.5 PreviewOpenAI | 44.9% | date=2025-02-27 · command=aider --model openai/gpt-4.5-preview · dirname=2025-02-27-20-26-15--gpt45-diff3 · versions=0.75.2.dev | 27 Feb 2025 | aider.chatT2 | History |
10 results
Scores are reported as published, with their evaluation configuration (harness, prompting, judge). The bar is relative to the best score on this page. Results with different configs are not directly comparable — see methodology.
The config filter matches a value inside each result's configuration (server-side, `config=` on the API). Chips are the values shared by several rows on the first page; per-model identifiers are not offered.
Definition
- Category
- coding
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 3 h agomedium
- Task
- 225 Exercism exercises in 6 languages, edit-format aware
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 3 h agomedium
- Metric
- pass rate (2 attempts) · %
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 3 h agomedium
- Direction
- Higher is better
- Known limitations
- Depends on aider's edit format and prompting; cost column depends on provider pricing.
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 3 h agomedium
- Website
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 3 h agomedium
Each value shows its source, tier and observation time. Missing rows mean no source stated them. How results are recorded →
- Evaluated models
- o1-2024-12-17 (high), DeepSeek Chat V2.5, Gemini 2.0 Pro exp-02-05, GPT-4o-mini (2024-07-18), claude-3-5-sonnet-20241022, GPT-4o (2024-11-20), GPT-4o (2024-08-06), Claude Haiku 3.5 +60(68 total)
As of
Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.
Claim history
Categorycategory1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| coding | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Known limitationsknown_limitations1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| Depends on aider's edit format and prompting; cost column depends on provider pricing. | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Metricmetric1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| pass rate (2 attempts) | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Tasktask1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| 225 Exercism exercises in 6 languages, edit-format aware | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Unitunit1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| % | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Websitewebsite1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://aider.chat/docs/leaderboards/ | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →
| Source | Document | Type | Tier | Last observed | Snapshots |
|---|---|---|---|---|---|
| Aider polyglot leaderboard (data file on GitHub) | raw.githubusercontent.com/Aider-AI/aider/main/aider/website/_data/polyglot_leaderboard.yml | leaderboard | T2· Quality secondary | 41 min ago | 1 |
Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.