Leaderboard 8 current results · config contains “diff-fenced”

Select models with +, then open Compare.

Leaderboard
#	Model	Score	Config	Evaluated	Source	Actions
1	#1gemini-2.5-pro-preview-06-05 (default think)	100%	date=2025-06-06 · command=aider --model gemini/gemini-2.5-pro-preview-06-05 · dirname=2025-06-06-18-38-56--gemini0605-diff-fenced · versions=0.84.1.dev	6 Jun 2025	aider.chatT2	History
2	#2gemini-2.5-pro-preview-06-05 (32k think)	99.6%	date=2025-06-06 · command=aider --model gemini/gemini-2.5-pro-preview-06-05 --thinking-tokens 32k · dirname=2025-06-06-16-36-21--gemini0605-32k-think-diff-fenced · versions=0.84.1.dev	6 Jun 2025	aider.chatT2	History
3	#3Gemini 2.5 Pro Preview 05-06Google	97.3%	date=2025-05-07 · command=aider --model gemini/gemini-2.5-pro-preview-05-06 · dirname=2025-05-07-19-32-40--gemini0506-diff-fenced-completion_cost · versions=0.82.4.dev	7 May 2025	aider.chatT2	History
4	#4Gemini 2.5 Pro Preview 03-25	92.4%	date=2025-04-12 · command=aider --model gemini/gemini-2.5-pro-preview-03-25 · dirname=2025-04-12-04-55-50--gemini-25-pro-diff-fenced · versions=0.81.3.dev	12 Apr 2025	aider.chatT2	History
5	#5gemini-2.5-pro-preview-06-05 (32k think)	83.1%	date=2025-06-06 · command=aider --model gemini/gemini-2.5-pro-preview-06-05 --thinking-tokens 32k · dirname=2025-06-06-16-36-21--gemini0605-32k-think-diff-fenced · versions=0.84.1.dev	6 Jun 2025	aider.chatT2	History
6	#6gemini-2.5-pro-preview-06-05 (default think)	79.1%	date=2025-06-06 · command=aider --model gemini/gemini-2.5-pro-preview-06-05 · dirname=2025-06-06-18-38-56--gemini0605-diff-fenced · versions=0.84.1.dev	6 Jun 2025	aider.chatT2	History
7	#7Gemini 2.5 Pro Preview 05-06Google	76.9%	date=2025-05-07 · command=aider --model gemini/gemini-2.5-pro-preview-05-06 · dirname=2025-05-07-19-32-40--gemini0506-diff-fenced-completion_cost · versions=0.82.4.dev	7 May 2025	aider.chatT2	History
8	#8Gemini 2.5 Pro Preview 03-25	72.9%	date=2025-04-12 · command=aider --model gemini/gemini-2.5-pro-preview-03-25 · dirname=2025-04-12-04-55-50--gemini-25-pro-diff-fenced · versions=0.81.3.dev	12 Apr 2025	aider.chatT2	History

8 results

Scores are reported as published, with their evaluation configuration (harness, prompting, judge). The bar is relative to the best score on this page. Results with different configs are not directly comparable — see methodology.

The config filter matches a value inside each result's configuration (server-side, `config=` on the API). Chips are the values shared by several rows on the first page; per-model identifiers are not offered.

Definition

Category: coding
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 3 h agomedium
Task: 225 Exercism exercises in 6 languages, edit-format aware
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 3 h agomedium
Metric: pass rate (2 attempts) · %
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 3 h agomedium
Direction: Higher is better
Known limitations: Depends on aider's edit format and prompting; cost column depends on provider pricing.
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 3 h agomedium
Website: aider.chat/docs/leaderboards
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 3 h agomedium

Each value shows its source, tier and observation time. Missing rows mean no source stated them. How results are recorded →

Evaluated models: o1-2024-12-17 (high), DeepSeek Chat V2.5, Gemini 2.0 Pro exp-02-05, GPT-4o-mini (2024-07-18), claude-3-5-sonnet-20241022, GPT-4o (2024-11-20), GPT-4o (2024-08-06), Claude Haiku 3.5 +60(68 total)

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history

6 claims · 6 properties

Categorycategory1

Claim history for Category
Value	Valid from → to	Status	Source	Confidence	Extractor
coding	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Known limitationsknown_limitations1

Claim history for Known limitations
Value	Valid from → to	Status	Source	Confidence	Extractor
Depends on aider's edit format and prompting; cost column depends on provider pricing.	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Metricmetric1

Claim history for Metric
Value	Valid from → to	Status	Source	Confidence	Extractor
pass rate (2 attempts)	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Tasktask1

Claim history for Task
Value	Valid from → to	Status	Source	Confidence	Extractor
225 Exercism exercises in 6 languages, edit-format aware	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Unitunit1

Claim history for Unit
Value	Valid from → to	Status	Source	Confidence	Extractor
%	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Websitewebsite1

Claim history for Website
Value	Valid from → to	Status	Source	Confidence	Extractor
https://aider.chat/docs/leaderboards/	11 Sept 2026 20:50 UTC → current	current	AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2	medium	curated

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
New benchmarkBenchmarkAider polyglot
New benchmark: Aider polyglot
registry
aider.chat

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
Aider polyglot leaderboard (data file on GitHub)	raw.githubusercontent.com/Aider-AI/aider/main/aider/website/_data/polyglot_leaderboard.yml	leaderboard	T2· Quality secondary	41 min ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.