BenchmarkActivecategory · agentic
τ²-bench
dual-control tool-agent-user interaction (telecom, retail, airline)
quality51
Updated 9 h ago · first seen 11 Sept 2026
bench_01M293SPH15XRN6880KKHSKM5P
- Metric
- pass^1 · %
- Direction
- Higher is better
- Results
- 440
- Leader
- Z.ai GLM 5.2 99.12%
Score history · Qwen3 VL 235B A22B Instruct 1 row
Not enough history to chart — a single observation (35.09% on 11 Sept 2026). Rows under different configurations count separately; the list below shows each one.
- 35.09%aa_slug=qwen3-vl-235b-a22b-instruct · evaluator=Artificial Analysis · index_version=4.311 Sept 2026
Leaderboard 440 current results
Select models with +, then open Compare.
| # | Model | Score | Config | Evaluated | Source | Actions |
|---|---|---|---|---|---|---|
| #301Ministral 3 8BMistral AI | 26.61% | aa_slug=ministral-3-8b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #302magistral-smallMistral AI | 26.61% | aa_slug=magistral-small · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #303hermes-4-llama-3-1-405bNous Research | 26.61% | aa_slug=hermes-4-llama-3-1-405b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #304qwen3-4b-2507-instructAlibaba Group | 26.61% | aa_slug=qwen3-4b-2507-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #305ring-1tinclusionAI | 26.32% | aa_slug=ring-1t · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #306qwen3-30b-a3b-instruct-reasoningAlibaba Group | 26.02% | aa_slug=qwen3-30b-a3b-instruct-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #307gemma-4-e4b-non-reasoningGoogle | 26.02% | aa_slug=gemma-4-e4b-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #308qwen3-1.7b-instruct-reasoningAlibaba Group | 26.02% | aa_slug=qwen3-1.7b-instruct-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #309gpt-5-nano-minimalOpenAI | 25.73% | aa_slug=gpt-5-nano-minimal · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #310qwen3-4b-2507-instruct-reasoningAlibaba Group | 25.44% | aa_slug=qwen3-4b-2507-instruct-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #311Nemotron 3 Nano 30B A3BNVIDIA | 25.44% | aa_slug=nvidia-nemotron-3-nano-30b-a3b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #312k2-think-v2MBZUAI Institute of Foundation Models | 25.44% | aa_slug=k2-think-v2 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #313llama-nemotron-super-49b-v1-5NVIDIA | 25.15% | aa_slug=llama-nemotron-super-49b-v1-5 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #314gpt-4oOpenAI | 25.15% | aa_slug=gpt-4o · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #315Mistral Small 3.1Mistral AI | 25.15% | aa_slug=mistral-small-3-1 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #316Devstral 2Mistral AI | 24.85% | aa_slug=devstral-2 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #317k2-v2-mediumMBZUAI Institute of Foundation Models | 24.85% | aa_slug=k2-v2-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #318Ministral 3 3BMistral AI | 24.85% | aa_slug=ministral-3-3b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #319qwen3-8b-instructAlibaba Group | 24.85% | aa_slug=qwen3-8b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #320Claude Haiku 3.5Anthropic | 24.56% | aa_slug=claude-3-5-haiku · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #321Mistral Large 3Mistral AI | 24.56% | aa_slug=mistral-large-3 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #322Mistral Medium 3Mistral AI | 24.27% | aa_slug=mistral-medium-3 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #323qwen3-235b-a22b-instruct-reasoningAlibaba Group | 23.98% | aa_slug=qwen3-235b-a22b-instruct-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #324Devstral Small 2Mistral AI | 23.39% | aa_slug=devstral-small-2 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #325gpt-5-4-mini-non-reasoningOpenAI | 23.39% | aa_slug=gpt-5-4-mini-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #326nvidia-nemotron-nano-9b-v2NVIDIA | 23.39% | aa_slug=nvidia-nemotron-nano-9b-v2 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #327Qwen3-VL-4B-InstructQwen | 23.39% | aa_slug=qwen3-vl-4b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #328magistral-mediumMistral AI | 23.1% | aa_slug=magistral-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #329llama-3-1-nemotron-instruct-70bNVIDIA | 23.1% | aa_slug=llama-3-1-nemotron-instruct-70b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #330granite-4-0-nano-1bIBM | 22.81% | aa_slug=granite-4-0-nano-1b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #331deepseek-v3DeepSeek | 22.81% | aa_slug=deepseek-v3 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #332GLM 4.5VZ.ai (Zhipu AI) | 22.51% | aa_slug=glm-4-5v-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #333Hermes-4-70BNous Research | 22.51% | aa_slug=hermes-4-llama-3-1-70b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #334qwen3-vl-8b-reasoningAlibaba Group | 22.51% | aa_slug=qwen3-vl-8b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #335Hermes 4 405BNous Research | 22.22% | aa_slug=hermes-4-llama-3-1-405b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #336gemma-4-e2b-non-reasoningGoogle | 22.22% | aa_slug=gemma-4-e2b-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #337qwen3-30b-a3b-instructAlibaba Group | 22.22% | aa_slug=qwen3-30b-a3b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #338R1 Distill Llama 70BDeepSeek | 21.93% | aa_slug=deepseek-r1-distill-llama-70b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #339NVIDIA-Nemotron-Nano-9B-v2NVIDIA | 21.93% | aa_slug=nvidia-nemotron-nano-9b-v2-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #340Qwen3 Next 80B A3B InstructQwen | 21.64% | aa_slug=qwen3-next-80b-a3b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #341nanbeige4-1-3bNanbeige | 21.64% | aa_slug=nanbeige4-1-3b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #342hermes-4-llama-3-1-70bNous Research | 21.64% | aa_slug=hermes-4-llama-3-1-70b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #343qwen3-1.7b-instructAlibaba Group | 21.64% | aa_slug=qwen3-1.7b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #344qwen3-omni-30b-a3b-reasoningAlibaba Group | 21.35% | aa_slug=qwen3-omni-30b-a3b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #345nvidia-nemotron-nano-12b-v2-vl-reasoningNVIDIA | 21.35% | aa_slug=nvidia-nemotron-nano-12b-v2-vl-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #346olmo-3-1-32b-instructAllen Institute for AI | 21.35% | aa_slug=olmo-3-1-32b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #347Claude 3 HaikuAnthropic | 21.05% | aa_slug=claude-3-haiku · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #348Qwen3-0.6BQwen | 21.05% | aa_slug=qwen3-0.6b-instruct-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #349Llama-3.2-3BMeta AI | 21.05% | aa_slug=llama-3-2-instruct-3b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #350gemma-4-E4BGoogle | 20.76% | aa_slug=gemma-4-e4b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #351k2-v2-lowMBZUAI Institute of Foundation Models | 20.76% | aa_slug=k2-v2-low · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #352ling-flash-2-0inclusionAI | 20.76% | aa_slug=ling-flash-2-0 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #353gemma-4-E2BGoogle | 20.76% | aa_slug=gemma-4-e2b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #354exaone-4-0-1-2bLG AI Research | 20.47% | aa_slug=exaone-4-0-1-2b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #355solar-miniUpstage | 20.18% | aa_slug=solar-mini · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #356qwen3-vl-30b-a3b-reasoningAlibaba Group | 19.88% | aa_slug=qwen3-vl-30b-a3b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #357devstral-mediumMistral AI | 19.88% | aa_slug=devstral-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #358glm-4-5vZ.ai (Zhipu AI) | 19.59% | aa_slug=glm-4-5v · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #359lfm2-5-1-2b-thinkingLiquid AI | 19.59% | aa_slug=lfm2-5-1-2b-thinking · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #360granite-4.1-3bIBM | 19.59% | aa_slug=granite-4-1-3b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #361Mistral Small 3Mistral AI | 19.59% | aa_slug=mistral-small-3 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #362granite-4-0-h-nano-1bIBM | 19.59% | aa_slug=granite-4-0-h-nano-1b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #363nvidia-nemotron-nano-12b-v2-vlNVIDIA | 19.3% | aa_slug=nvidia-nemotron-nano-12b-v2-vl · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #364Qwen3 VL 30B A3B InstructQwen | 19.01% | aa_slug=qwen3-vl-30b-a3b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #365Qwen3-4BQwen | 19.01% | aa_slug=qwen3-4b-instruct-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #366Llama-3.1-405BMeta AI | 19.01% | aa_slug=llama-3-1-instruct-405b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #367gemini-2-5-flash-liteGoogle | 19.01% | aa_slug=gemini-2-5-flash-lite · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #368Gemini 2.5 Flash-LiteGoogle | 18.42% | aa_slug=gemini-2-5-flash-lite-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #369mistral-small-4-non-reasoningMistral AI | 18.42% | aa_slug=mistral-small-4-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #370Llama 4 MaverickMeta AI | 17.84% | aa_slug=llama-4-maverick · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #371nova-liteAmazon Web Services | 17.54% | aa_slug=nova-lite · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #372exaone-4-0-32b-reasoningLG AI Research | 17.25% | aa_slug=exaone-4-0-32b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #373granite-4-0-h-smallIBM | 17.25% | aa_slug=granite-4-0-h-small · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #374gpt-4.1-nanoOpenAI | 17.25% | aa_slug=gpt-4-1-nano · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #375qwen3-omni-30b-a3b-instructAlibaba Group | 16.37% | aa_slug=qwen3-omni-30b-a3b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #376exaone-4-0-1-2b-reasoningLG AI Research | 16.37% | aa_slug=exaone-4-0-1-2b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #377Llama 3.1 8BMeta AI | 16.37% | aa_slug=llama-3-1-instruct-8b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #378LFM2.5-8B-A1BLiquid AI | 16.08% | aa_slug=lfm2-5-8b-a1b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #379step-3-vl-10bStepFun | 16.08% | aa_slug=step-3-vl-10b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #380jamba-reasoning-3bAI21 Labs | 15.79% | aa_slug=jamba-reasoning-3b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #381qwen3-vl-4b-reasoningAlibaba Group | 15.5% | aa_slug=qwen3-vl-4b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #382Llama 4 ScoutMeta AI | 15.5% | aa_slug=llama-4-scout · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #383Llama-3.1-70BMeta AI | 15.2% | aa_slug=llama-3-1-instruct-70b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #384Command ACohere | 15.2% | aa_slug=command-a · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #385Gemini 2.5 FlashGoogle | 14.91% | aa_slug=gemini-2-5-flash · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #386llama-3-2-instruct-11b-visionMeta AI | 14.62% | aa_slug=llama-3-2-instruct-11b-vision · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #387qwen3-0.6b-instructAlibaba Group | 14.62% | aa_slug=qwen3-0.6b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #388granite-4-0-h-350mIBM | 14.62% | aa_slug=granite-4-0-h-350m · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #389nova-microAmazon Web Services | 14.04% | aa_slug=nova-micro · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #390nova-proAmazon Web Services | 14.04% | aa_slug=nova-pro · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #391lfm2-2-6bLiquid AI | 13.45% | aa_slug=lfm2-2-6b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #392jamba-1-7-largeAI21 Labs | 13.45% | aa_slug=jamba-1-7-large · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #393granite-4-0-350mIBM | 13.16% | aa_slug=granite-4-0-350m · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #394ling-mini-2-0inclusionAI | 13.16% | aa_slug=ling-mini-2-0 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #395apertus-70b-instructSwiss AI Initiative | 12.87% | aa_slug=apertus-70b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #396Olmo-3-7B-InstructAllen Institute for AI | 12.57% | aa_slug=olmo-3-7b-instruct · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #397Granite 4.0 MicroIBM | 12.57% | aa_slug=granite-4-0-micro · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #398jamba-1-7-miniAI21 Labs | 12.57% | aa_slug=jamba-1-7-mini · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #399LFM2-1.2BLiquid AI | 12.57% | aa_slug=lfm2-1-2b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #400llama-3-1-nemotron-nano-4b-reasoningNVIDIA | 11.7% | aa_slug=llama-3-1-nemotron-nano-4b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History |
Scores are reported as published, with their evaluation configuration (harness, prompting, judge). The bar is relative to the best score on this page. Results with different configs are not directly comparable — see methodology.
Definition
- Category
- agentic
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 9 h agomedium
- Task
- dual-control tool-agent-user interaction (telecom, retail, airline)
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 9 h agomedium
- Metric
- pass^1 · %
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 9 h agomedium
- Direction
- Higher is better
- Paper
- https://arxiv.org/abs/2506.07982
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 9 h agomedium
Each value shows its source, tier and observation time. Missing rows mean no source stated them. How results are recorded →
- Evaluated models
- grok-3-mini-reasoning, gemma-4-12B, grok-4-3-low, GPT-5.1-Codex Mini, Qwen3 Coder Next, KAT-Coder-Pro V2, nova-2-0-lite-reasoning-medium, lfm2-24b-a2b +432(440 total)
As of
Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.
Claim history
5 claims · 5 properties
Categorycategory1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| agentic | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Metricmetric1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| pass^1 | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Paperpaper1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://arxiv.org/abs/2506.07982 | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Tasktask1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| dual-control tool-agent-user interaction (telecom, retail, airline) | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Unitunit1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| % | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →
| Source | Document | Type | Tier | Last observed | Snapshots |
|---|---|---|---|---|---|
| Artificial Analysis | artificialanalysis.ai/leaderboards/models | leaderboard | T2· Quality secondary | 7 h ago | 1 |
Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.