BenchmarkActivecategory · agentic
τ²-bench
dual-control tool-agent-user interaction (telecom, retail, airline)
quality51
Updated 9 h ago · first seen 11 Sept 2026
bench_01M293SPH15XRN6880KKHSKM5P
- Metric
- pass^1 · %
- Direction
- Higher is better
- Results
- 440
- Leader
- Z.ai GLM 5.2 99.12%
Score history · o1 1 row
Not enough history to chart — a single observation (62.57% on 11 Sept 2026). Rows under different configurations count separately; the list below shows each one.
- 62.57%aa_slug=o1 · evaluator=Artificial Analysis · index_version=4.311 Sept 2026
Leaderboard 440 current results
Select models with +, then open Compare.
| # | Model | Score | Config | Evaluated | Source | Actions |
|---|---|---|---|---|---|---|
| #201nemotron-cascade-2-30b-a3bNVIDIA | 53.22% | aa_slug=nemotron-cascade-2-30b-a3b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #202Qwen3 235B A22B Instruct 2507Qwen | 53.22% | aa_slug=qwen3-235b-a22b-instruct-2507-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #203gpt-4.1-miniOpenAI | 52.92% | aa_slug=gpt-4-1-mini · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #204gpt-5-4-nano-mediumOpenAI | 52.63% | aa_slug=gpt-5-4-nano-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #205Claude Sonnet 4Anthropic | 52.34% | aa_slug=claude-4-sonnet · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #206Magistral Medium 1.2Mistral AI | 52.05% | aa_slug=magistral-medium-2509 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #207gpt-oss-20b-lowOpenAI | 50.29% | aa_slug=gpt-oss-20b-low · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #208Claude 3.7 SonnetAnthropic | 50% | aa_slug=claude-3-7-sonnet · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #209Seed-OSS-36B-InstructByteDance | 49.42% | aa_slug=seed-oss-36b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #210midm-250-pro-rsnsftKorea Telecom | 49.42% | aa_slug=midm-250-pro-rsnsft · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #211gpt-5-5-instant-05-26OpenAI | 49.42% | aa_slug=gpt-5-5-instant-05-26 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #212grok-3SpaceXAI | 48.83% | aa_slug=grok-3 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #213solar-open-100b-reasoningUpstage | 48.25% | aa_slug=solar-open-100b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #214Qwen3.5-0.8BQwen | 47.66% | aa_slug=qwen3-5-0-8b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #215gpt-4.1OpenAI | 47.08% | aa_slug=gpt-4-1 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #216DeepSeek V3 0324DeepSeek | 47.08% | aa_slug=deepseek-v3-0324 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #217sarvam-105bSarvam | 46.78% | aa_slug=sarvam-105b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #218gpt-5-1-non-reasoningOpenAI | 46.49% | aa_slug=gpt-5-1-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #219gpt-5-2-non-reasoningOpenAI | 46.49% | aa_slug=gpt-5-2-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #220motif-2-12-7bMotif Technologies | 46.49% | aa_slug=motif-2-12-7b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #221GLM 4.5 AirZ.ai (Zhipu AI) | 46.49% | aa_slug=glm-4-5-air · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #222gemini-2-5-flash-preview-09-2025-reasoningGoogle | 45.61% | aa_slug=gemini-2-5-flash-preview-09-2025-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #223qwen3-vl-32b-reasoningAlibaba Group | 45.61% | aa_slug=qwen3-vl-32b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #224nemotron-3-nano-omni-30b-a3bNVIDIA | 45.32% | aa_slug=nemotron-3-nano-omni-30b-a3b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #225gpt-oss-120b-lowOpenAI | 45.03% | aa_slug=gpt-oss-120b-low · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #226nova-2-0-omniAmazon Web Services | 44.74% | aa_slug=nova-2-0-omni · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #227qwen3-coder-480b-a35b-instructAlibaba Group | 43.57% | aa_slug=qwen3-coder-480b-a35b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #228Gemma 4 26B A4BGoogle | 43.57% | aa_slug=gemma-4-26b-a4b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #229gemini-3-flashGoogle | 43.27% | aa_slug=gemini-3-flash · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #230GLM 4.5Z.ai (Zhipu AI) | 42.98% | aa_slug=glm-4.5 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #231granite-4.1-30bIBM | 42.11% | aa_slug=granite-4-1-30b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #232qwen3-next-80b-a3b-reasoningAlibaba Group | 41.52% | aa_slug=qwen3-next-80b-a3b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #233Mistral Small 4Mistral AI | 41.23% | aa_slug=mistral-small-4 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #234nvidia-nemotron-3-nano-30b-a3b-reasoningNVIDIA | 40.94% | aa_slug=nvidia-nemotron-3-nano-30b-a3b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #235Mistral Medium 3.1Mistral AI | 40.64% | aa_slug=mistral-medium-3-1 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #236gemma-4-26b-a4b-non-reasoningGoogle | 40.35% | aa_slug=gemma-4-26b-a4b-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #237nova-premierAmazon Web Services | 38.3% | aa_slug=nova-premier · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #238Devstral Small 1.0Mistral AI | 38.01% | aa_slug=devstral-small-2505 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #239deepseek-v3-1-reasoningDeepSeek | 37.43% | aa_slug=deepseek-v3-1-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #240North Mini Code (free)Cohere | 37.43% | aa_slug=north-mini-code · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #241DeepSeek V3.1 TerminusDeepSeek | 37.13% | aa_slug=deepseek-v3-1-terminus · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #242deepseek-v3-1-terminus-reasoningDeepSeek | 37.13% | aa_slug=deepseek-v3-1-terminus-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #243Pixtral LargeMistral AI | 36.55% | aa_slug=pixtral-large-2411 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #244gpt-5-nanoOpenAI | 36.55% | aa_slug=gpt-5-nano · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #245gpt-5-4-mini-mediumOpenAI | 36.55% | aa_slug=gpt-5-4-mini-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #246deepseek-r1DeepSeek | 36.55% | aa_slug=deepseek-r1 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #247gemma-4-12BGoogle | 36.26% | aa_slug=gemma-4-12b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #248gpt-5-4-non-reasoningOpenAI | 35.96% | aa_slug=gpt-5-4-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #249Qwen3 VL 235B A22B InstructQwen | 35.09% | aa_slug=qwen3-vl-235b-a22b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #250DeepSeek V3.1DeepSeek | 34.8% | aa_slug=deepseek-v3-1 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #251gpt-5-4-nano-non-reasoningOpenAI | 34.8% | aa_slug=gpt-5-4-nano-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #252qwen2-5-72b-instructAlibaba Group | 34.5% | aa_slug=qwen2-5-72b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #253Qwen3 Coder 30B A3B InstructQwen | 34.5% | aa_slug=qwen3-coder-30b-a3b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #254Qwen3 14BQwen | 34.5% | aa_slug=qwen3-14b-instruct-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #255sarvam-30bSarvam | 34.5% | aa_slug=sarvam-30b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #256MiniMax-M1-80kMiniMax | 34.21% | aa_slug=minimax-m1-80k · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #257deepseek-v3-2-0925DeepSeek | 33.92% | aa_slug=deepseek-v3-2-0925 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #258DeepSeek V3.2 ExpDeepSeek | 33.92% | aa_slug=deepseek-v3-2-reasoning-0925 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #259qwen3-235b-a22b-instruct-2507Alibaba Group | 33.33% | aa_slug=qwen3-235b-a22b-instruct-2507 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #260Mistral Large 2.0Mistral AI | 33.04% | aa_slug=mistral-large-2407 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #261ling-1tinclusionAI | 32.75% | aa_slug=ling-1t · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #262qwen3-max-previewAlibaba Group | 32.75% | aa_slug=qwen3-max-preview · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #263Claude Haiku 4.5Anthropic | 32.46% | aa_slug=claude-4-5-haiku · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #264qwen3-14b-instructAlibaba Group | 32.16% | aa_slug=qwen3-14b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #265gpt-5-mini-minimalOpenAI | 31.87% | aa_slug=gpt-5-mini-minimal · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #266solar-pro-2Upstage | 31.87% | aa_slug=solar-pro-2 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #267gemma-4-12b-non-reasoningGoogle | 31.87% | aa_slug=gemma-4-12b-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #268MiniMax-M1-40kMiniMax | 31.58% | aa_slug=minimax-m1-40k · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #269GLM 4.6VZ.ai (Zhipu AI) | 31.58% | aa_slug=glm-4-6v-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #270gemini-2-5-flash-reasoningGoogle | 31.58% | aa_slug=gemini-2-5-flash-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #271o3 Mini HighOpenAI | 31.29% | aa_slug=o3-mini-high · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #272gemini-3-1-flash-lite-previewGoogle | 31.29% | aa_slug=gemini-3-1-flash-lite-preview · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #273gemini-2-5-flash-lite-preview-09-2025-reasoningGoogle | 30.7% | aa_slug=gemini-2-5-flash-lite-preview-09-2025-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #274glm-4-6vZ.ai (Zhipu AI) | 30.7% | aa_slug=glm-4-6v · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #275mistral-large-2Mistral AI | 30.7% | aa_slug=mistral-large-2 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #276gpt-5-nano-mediumOpenAI | 30.41% | aa_slug=gpt-5-nano-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #277gemini-2-5-flash-lite-preview-09-2025Google | 30.41% | aa_slug=gemini-2-5-flash-lite-preview-09-2025 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #278Qwen3 32BQwen | 29.82% | aa_slug=qwen3-32b-instruct-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #279Mistral Small 3.2Mistral AI | 29.53% | aa_slug=mistral-small-3-2 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #280Gemini 2.0 FlashGoogle | 29.53% | aa_slug=gemini-2-0-flash · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #281Qwen3 VL 32B InstructQwen | 29.24% | aa_slug=qwen3-vl-32b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #282Qwen3 VL 8B InstructQwen | 29.24% | aa_slug=qwen3-vl-8b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #283GPT-4o (2024-08-06)OpenAI | 28.95% | aa_slug=gpt-4o-2024-08-06 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #284o3-miniOpenAI | 28.65% | aa_slug=o3-mini · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #285gemini-2-5-flash-preview-09-2025Google | 28.36% | aa_slug=gemini-2-5-flash-preview-09-2025 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #286devstral-smallMistral AI | 28.36% | aa_slug=devstral-small · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #287solar-pro-2-reasoningUpstage | 28.07% | aa_slug=solar-pro-2-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #288nvidia-nemotron-3-nano-4bNVIDIA | 28.07% | aa_slug=nvidia-nemotron-3-nano-4b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #289qwen3-30b-a3b-2507-reasoningAlibaba Group | 28.07% | aa_slug=qwen3-30b-a3b-2507-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #290llama-nemotron-super-49b-v1-5-reasoningNVIDIA | 28.07% | aa_slug=llama-nemotron-super-49b-v1-5-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #291falcon-h1r-7bTII UAE | 27.78% | aa_slug=falcon-h1r-7b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #292granite-4.1-8bIBM | 27.78% | aa_slug=granite-4-1-8b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #293k2-v2MBZUAI Institute of Foundation Models | 27.78% | aa_slug=k2-v2 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #294Qwen3 8BQwen | 27.78% | aa_slug=qwen3-8b-instruct-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #295Magistral Small 1.2Mistral AI | 27.78% | aa_slug=magistral-small-2509 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #296Ministral 3 14BMistral AI | 27.19% | aa_slug=ministral-3-14b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #297qwen3-235b-a22b-instructAlibaba Group | 27.19% | aa_slug=qwen3-235b-a22b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #298llama-3-3-nemotron-super-49b-reasoningNVIDIA | 26.9% | aa_slug=llama-3-3-nemotron-super-49b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #299Llama 3.3 70BMeta AI | 26.61% | aa_slug=llama-3-3-instruct-70b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #300intellect-3Prime Intellect | 26.61% | aa_slug=intellect-3 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History |
Scores are reported as published, with their evaluation configuration (harness, prompting, judge). The bar is relative to the best score on this page. Results with different configs are not directly comparable — see methodology.
Definition
- Category
- agentic
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 9 h agomedium
- Task
- dual-control tool-agent-user interaction (telecom, retail, airline)
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 9 h agomedium
- Metric
- pass^1 · %
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 9 h agomedium
- Direction
- Higher is better
- Paper
- https://arxiv.org/abs/2506.07982
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 9 h agomedium
Each value shows its source, tier and observation time. Missing rows mean no source stated them. How results are recorded →
- Evaluated models
- grok-3-mini-reasoning, gemma-4-12B, grok-4-3-low, GPT-5.1-Codex Mini, Qwen3 Coder Next, KAT-Coder-Pro V2, nova-2-0-lite-reasoning-medium, lfm2-24b-a2b +432(440 total)
As of
Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.
Claim history
5 claims · 5 properties
Categorycategory1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| agentic | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Metricmetric1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| pass^1 | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Paperpaper1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://arxiv.org/abs/2506.07982 | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Tasktask1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| dual-control tool-agent-user interaction (telecom, retail, airline) | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Unitunit1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| % | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →
| Source | Document | Type | Tier | Last observed | Snapshots |
|---|---|---|---|---|---|
| Artificial Analysis | artificialanalysis.ai/leaderboards/models | leaderboard | T2· Quality secondary | 7 h ago | 1 |
Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.