Skip to content
AI Atlas
BenchmarkActivecategory · codingfamily · aider-polyglot · variant well-formed

Aider polyglot — well-formed responses

aider.chat/docs/leaderboards

share of aider polyglot cases where every edit was syntactically well formed (parsable edit blocks)

data quality57

Updated 5 h ago · first seen 12 Sept 2026

Metric
percent_cases_well_formed · % ↑
Current results
67
Models
61
Current leader
Codestral 25.01 100%

Score history · o3-mini 2 rows

Not enough history to chart — 2 observations, all dated 31 Jan 2025. Rows under different configurations count separately; the list below shows each one.

  • 93.3%date=2025-01-31 · command=aider --model o3-mini --reasoning-effort high · dirname=2025-01-31-20-42-47--o3-mini-diff-high · versions=0.72.4.dev31 Jan 2025
  • 95.1%date=2025-01-31 · command=aider --model o3-mini · dirname=2025-01-31-20-27-46--o3-mini-diff2 · versions=0.72.4.dev31 Jan 2025

Back to the leaderboard

Frontier over time · percent_cases_well_formed

1 leader change recorded, all dated 21 Dec 2024 — the frontier line needs at least two distinct dates. The corpus is young: every result was first observed on the same day, so leader changes will separate in time as sources are re-crawled.

  1. 100%GPT-4o-mini (2024-07-18) OpenAI Official board21 Dec 2024

Includes closed rows (history). A point is emitted whenever a result beats every earlier result of the same group, ordered by evaluated_at when the source publishes it, else observed_at.

Leaderboard 61 models

Select models with +, then Compare.

Leaderboard
#ModelScoreTrustConfigurationvs leaderEvaluatedSourceActions
1Codestral 25.01Mistral AI · Codestral 25.01100%Official boardedit_formatwholeleader13 Jan 2025aider.chatT2History
1DeepSeek R1 + claude-3-5-sonnet-20241022 · Claude 3.5100%Official boardedit_formatarchitectleader23 Jan 2025aider.chatT2History
1GPT-4o-mini (2024-07-18)OpenAI · GPT 4100%Official boardedit_formatwholeleader21 Dec 2024aider.chatT2History
1Gemini 2.0 Pro exp-02-05 · Gemini 2.0100%Official boardedit_formatwholeleader25 Feb 2025aider.chatT2History
1Gemini 2.5 Pro Preview 06-05Google · Gemini 2.5 · best of 2 rows100%Official boardedit_formatdiff-fencedreasoningonleader6 Jun 2025aider.chatT2History
1Gemma 3 27BOpen weightsGoogle · Gemma 3100%Official boardedit_formatwholeleader15 Mar 2025aider.chatT2History
1Grok 3 Mini Beta (low) · Grok 3100%Official boardedit_formatwholeleader10 Apr 2025aider.chatT2History
1QwQ-32B + Qwen 2.5 Coder Instruct · Qwen2.5100%Official boardedit_formatarchitectleader7 Mar 2025aider.chatT2History
1gemini-2.0-flash-expClosedGoogle · Gemini 2.0100%Official boardedit_formatwholeleader22 Dec 2024aider.chatT2History
1o3 (high) + gpt-4.1 · GPT 4.1100%Official boardedit_formatarchitectleader27 Jun 2025aider.chatT2History
11DeepSeek V3 0324Open weightsDeepSeek · DeepSeek-V399.6%Official boardedit_formatdiffPartially comparable-0.40 pt24 Mar 2025aider.chatT2History
11Grok 3 Beta · Grok 399.6%Official boardedit_formatdiffPartially comparable-0.40 pt10 Apr 2025aider.chatT2History
11Grok 3 Mini Beta (high) · Grok 399.6%Official boardedit_formatwholeComparable-0.40 pt10 Apr 2025aider.chatT2History
11▼9Qwen2.5 Coder 32B InstructOpen weightsQwen · Qwen2.599.6%Official boardedit_formatwholeComparable-0.40 pt26 Dec 2024aider.chatT2History
11claude-3-5-sonnet-20241022Anthropic · Claude 3.599.6%Official boardedit_formatdiffPartially comparable-0.40 pt17 Jan 2025aider.chatT2History
11command-a-03-2025-quality · Command99.6%Official boardedit_formatwholeComparable-0.40 pt14 Mar 2025aider.chatT2History
17Llama 4 MaverickOpen weightsMeta AI · Llama 499.1%Official boardedit_formatwholeComparable-0.90 pt6 Apr 2025aider.chatT2History
18Claude Opus 4ClosedAnthropic · Claude · best of 2 rows98.7%Official boardedit_formatdiffreasoningoffPartially comparable-1.30 pt25 May 2025aider.chatT2History
18DeepSeek Chat V3 (prev) · DeepSeek98.7%Official boardedit_formatdiffPartially comparable-1.30 pt25 Dec 2024aider.chatT2History
20Claude Sonnet 4ClosedAnthropic · Claude · best of 2 rows98.2%Official boardedit_formatdiffreasoningoffPartially comparable-1.80 pt24 May 2025aider.chatT2History
20DeepSeek-V3.2-Exp (Chat) · DeepSeek98.2%Official boardedit_formatdiffPartially comparable-1.80 pt3 Oct 2025aider.chatT2History
20Quasar Alpha98.2%Official boardedit_formatdiffPartially comparable-1.80 pt4 Apr 2025aider.chatT2History
20gemini-exp-1206 · Gemini98.2%Official boardedit_formatwholeComparable-1.80 pt22 Dec 2024aider.chatT2History
20gpt-4.1ClosedOpenAI · GPT 4.198.2%Official boardedit_formatdiffPartially comparable-1.80 pt14 Apr 2025aider.chatT2History
25claude-3-7-sonnet-20250219 (32k thinking tokens) · Claude 3.797.8%Official boardedit_formatdiffPartially comparable-2.20 pt24 Feb 2025aider.chatT2History
25o3-proClosedOpenAI · OpenAI o-series97.8%Official boardedit_formatdiffreasoning_efforthighPartially comparable-2.20 pt28 Jun 2025aider.chatT2History
27DeepSeek-V3.2-Exp (Reasoner) · DeepSeek97.3%Official boardedit_formatdiffPartially comparable-2.70 pt3 Oct 2025aider.chatT2History
27GPT-4.5 PreviewClosedOpenAI · GPT 4.597.3%Official boardedit_formatdiffPartially comparable-2.70 pt27 Feb 2025aider.chatT2History
27Gemini 2.5 Pro Preview 05-06Google · Gemini 2.597.3%Official boardedit_formatdiff-fencedPartially comparable-2.70 pt7 May 2025aider.chatT2History
27Optimus Alpha97.3%Official boardedit_formatdiffPartially comparable-2.70 pt10 Apr 2025aider.chatT2History
27grok-4ClosedSpaceXAI · Grok 497.3%Official boardedit_formatdiffreasoning_efforthighPartially comparable-2.70 pt11 Jul 2025aider.chatT2History
32R1Open weightsDeepSeek96.9%Official boardedit_formatdiffPartially comparable-3.10 pt20 Jan 2025aider.chatT2History
32o1-mini-2024-09-12 · OpenAI o-series96.9%Official boardedit_formatwholeComparable-3.10 pt22 Dec 2024aider.chatT2History
34gemini-2.5-flash-preview-05-20 (24k think) · Gemini 2.595.6%Official boardedit_formatdiffPartially comparable-4.40 pt25 May 2025aider.chatT2History
35GPT-4o (2024-11-20)OpenAI · GPT 495.1%Official boardedit_formatdiffPartially comparable-4.90 pt30 Dec 2024aider.chatT2History
35o3-miniClosedOpenAI · OpenAI o-series · best of 2 rows95.1%Official boardedit_formatdiffreasoning_effortmediumPartially comparable-4.90 pt31 Jan 2025aider.chatT2History
35openhands-lm-32b-v0.195.1%Official boardedit_formatwholeComparable-4.90 pt19 Apr 2025aider.chatT2History
38o3ClosedOpenAI · OpenAI o-series · best of 2 rows94.7%Official boardedit_formatdiffreasoning_efforthighPartially comparable-5.30 pt25 Jun 2025aider.chatT2History
39R1 0528Open weightsDeepSeek94.6%Official boardedit_formatdiffPartially comparable-5.40 pt6 Jun 2025aider.chatT2History
40GPT-4o (2024-08-06)ClosedOpenAI · GPT 494.2%Official boardedit_formatdiffPartially comparable-5.80 pt30 Dec 2024aider.chatT2History
40gpt-4.1-nanoClosedOpenAI · GPT 4.194.2%Official boardedit_formatwholeComparable-5.80 pt14 Apr 2025aider.chatT2History
42gemini-2.5-flash-preview-05-20 (no think) · Gemini 2.593.8%Official boardedit_formatdiffPartially comparable-6.20 pt26 May 2025aider.chatT2History
43Claude 3.7 SonnetClosedAnthropic · Claude93.3%Official boardedit_formatdiffreasoningoffPartially comparable-6.70 pt24 Feb 2025aider.chatT2History
43chatgpt-4o-latest (2025-02-15) · GPT 493.3%Official boardedit_formatdiffPartially comparable-6.70 pt15 Feb 2025aider.chatT2History
45DeepSeek Chat V2.5 · DeepSeek92.9%Official boardedit_formatdiffPartially comparable-7.10 pt21 Dec 2024aider.chatT2History
45Kimi K2 0711Open weightsMoonshot AI · Kimi92.9%Official boardedit_formatdiffPartially comparable-7.10 pt17 Jul 2025aider.chatT2History
45Qwen3 235B A22B diff, no think, Alibaba API · Qwen392.9%Official boardedit_formatdiffPartially comparable-7.10 pt9 May 2025aider.chatT2History
45yi-lightning · Yi92.9%Official boardedit_formatwholeComparable-7.10 pt23 Dec 2024aider.chatT2History
49Gemini 2.5 Pro Preview 03-25 · Gemini 2.592.4%Official boardedit_formatdiff-fencedPartially comparable-7.60 pt12 Apr 2025aider.chatT2History
49gpt-4.1-miniClosedOpenAI · GPT 4.192.4%Official boardedit_formatdiffPartially comparable-7.60 pt14 Apr 2025aider.chatT2History
51o1ClosedOpenAI · OpenAI o-series91.5%Official boardedit_formatdiffreasoning_efforthighPartially comparable-8.50 pt21 Dec 2024aider.chatT2History
52Claude Haiku 3.5ClosedAnthropic · Claude91.1%Official boardedit_formatdiffPartially comparable-8.90 pt21 Dec 2024aider.chatT2History
53o4-miniClosedOpenAI · OpenAI o-series90.7%Official boardedit_formatdiffreasoning_efforthighPartially comparable-9.30 pt16 Apr 2025aider.chatT2History
54qwen-max-2025-01-25 · Qwen90.2%Official boardedit_formatdiffPartially comparable-9.80 pt28 Jan 2025aider.chatT2History
55▼54gpt-5ClosedOpenAI · GPT 5 · best of 2 rows88.4%Official boardedit_formatdiffreasoning_effortmediumPartially comparable-11.6 pt25 Aug 2025aider.chatT2History
56gemini-2.5-flash-preview-04-17 (default) · Gemini 2.585.3%Official boardedit_formatdiffPartially comparable-14.7 pt20 Apr 2025aider.chatT2History
57Qwen3 32BOpen weightsQwen · Qwen383.6%Official boardedit_formatdiffPartially comparable-16.4 pt8 May 2025aider.chatT2History
58gpt-oss-120bOpen weightsOpenAI · gpt-oss79.1%Official boardedit_formatdiffreasoning_efforthighPartially comparable-20.9 pt6 Aug 2025aider.chatT2History
59gemini-2.0-flash-thinking-exp-01-21ClosedGoogle · Gemini 2.077.8%Official boardedit_formatdiffPartially comparable-22.2 pt21 Jan 2025aider.chatT2History
60QwQ-32BOpen weightsAlibaba Group · Qwen67.6%Official boardedit_formatdiffPartially comparable-32.4 pt6 Mar 2025aider.chatT2History
61chatgpt-4o-latest (2025-03-29) · GPT 464.4%Official boardedit_formatdiffPartially comparable-35.6 pt29 Mar 2025aider.chatT2History

61 results

One row per canonical model — its best current row inside this comparability group (effort variants are folded into the model). Bars are relative to the page's best score. “vs leader” reads comparability: partially comparable = same task, conditions differ (reasoning effort, temperature, judge). Rules →