BenchmarkActivecategory · instruction-following
IFBench
precise instruction following with novel constraints
quality51
Updated 5 h ago · first seen 11 Sept 2026
bench_01M293SPGX8Q1PSHHMPKC9R37H
- Metric
- accuracy · %
- Direction
- Higher is better
- Results
- 450
- Leader
- grok-4-3-medium 83.33%
Leaderboard 450 current results
Select models with +, then open Compare.
| # | Model | Score | Config | Evaluated | Source | Actions |
|---|---|---|---|---|---|---|
| #1grok-4-3-mediumSpaceXAI | 83.33% | aa_slug=grok-4-3-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #2grok-4-20-0309SpaceXAI | 82.93% | aa_slug=grok-4-20-0309 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #3MiniMax M3MiniMax | 82.86% | aa_slug=minimax-m3 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #4Nemotron 3 UltraNVIDIA | 81.36% | aa_slug=nvidia-nemotron-3-ultra-550b-a55b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #5Grok 4.3xAI | 81.29% | aa_slug=grok-4-3 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #6Grok 4.20xAI | 81.22% | aa_slug=grok-4-20 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #7grok-4-3-lowSpaceXAI | 80.95% | aa_slug=grok-4-3-low · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #8Qwen3.7 MaxQwen | 80.54% | aa_slug=qwen3-7-max · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #9nemotron-cascade-2-30b-a3bNVIDIA | 80.41% | aa_slug=nemotron-cascade-2-30b-a3b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #10MiMo-V2.5-ProXiaomi | 79.86% | aa_slug=mimo-v2-5-pro · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #11nova-2-0-pro-reasoning-lowAmazon Web Services | 79.59% | aa_slug=nova-2-0-pro-reasoning-low · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #12deepseek-v4-flash-0420DeepSeek | 79.18% | aa_slug=deepseek-v4-flash-0420 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #13nova-2-0-pro-reasoning-mediumAmazon Web Services | 79.05% | aa_slug=nova-2-0-pro-reasoning-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #14Qwen3.5 397B A17BQwen | 78.78% | aa_slug=qwen3-5-397b-a17b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #15Gemini 3 Flash PreviewGoogle | 77.96% | aa_slug=gemini-3-flash-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #16Qwen 3.7 PlusQwen | 77.96% | aa_slug=qwen3-7-plus · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #17GPT-5.2-CodexOpenAI | 77.62% | aa_slug=gpt-5-2-codex · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #18gemini-3-1-flash-lite-previewGoogle | 77.21% | aa_slug=gemini-3-1-flash-lite-preview · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #19Gemini 3.1 Pro PreviewGoogle | 77.14% | aa_slug=gemini-3-1-pro-preview · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #20Qwen3.6 Max PreviewQwen | 76.6% | aa_slug=qwen3-6-max · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #21deepseek-v4-pro-0424DeepSeek | 76.46% | aa_slug=deepseek-v4-pro-0424 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #22Gemini 3.5 FlashGoogle | 76.33% | aa_slug=gemini-3-5-flash · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #23GLM 5.1Z.ai (Zhipu AI) | 76.26% | aa_slug=glm-5-1 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #24Kimi K2.6Moonshot AI | 75.99% | aa_slug=kimi-k2-6 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #25gpt-5.4-nanoOpenAI | 75.92% | aa_slug=gpt-5-4-nano · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #26muse-sparkMeta AI | 75.92% | aa_slug=muse-spark · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #27gpt-5.5OpenAI | 75.85% | aa_slug=gpt-5-5 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #28MiniMax M2.7MiniMax | 75.71% | aa_slug=minimax-m2-7 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #29Qwen3.5-122B-A10BQwen | 75.71% | aa_slug=qwen3-5-122b-a10b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #30Qwen3.5-27BQwen | 75.58% | aa_slug=qwen3-5-27b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #31Gemma 4 31BGoogle | 75.58% | aa_slug=gemma-4-31b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #32gpt-5-miniOpenAI | 75.44% | aa_slug=gpt-5-mini · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #33gpt-5.2OpenAI | 75.44% | aa_slug=gpt-5-2 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #34gpt-5.3-codexOpenAI | 75.37% | aa_slug=gpt-5-3-codex · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #35Qwen3.6 PlusQwen | 75.17% | aa_slug=qwen3-6-plus · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #36gemini-3-5-flash-mediumGoogle | 74.56% | aa_slug=gemini-3-5-flash-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #37GPT-5-CodexOpenAI | 74.15% | aa_slug=gpt-5-codex · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #38command-a-plusCohere | 73.95% | aa_slug=command-a-plus · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #39gpt-5.4OpenAI | 73.95% | aa_slug=gpt-5-4 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #40gemma-4-12BGoogle | 73.54% | aa_slug=gemma-4-12b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #41deepseek-v4-flash-0420-highDeepSeek | 73.47% | aa_slug=deepseek-v4-flash-0420-high · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #42Z.ai GLM 5.2Z.ai (Zhipu AI) | 73.33% | aa_slug=glm-5-2 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #43gpt-5.4-miniOpenAI | 73.27% | aa_slug=gpt-5-4-mini · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #44GLM 5 TurboZ.ai (Zhipu AI) | 73.2% | aa_slug=glm-5-turbo · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #45gpt-5OpenAI | 73.06% | aa_slug=gpt-5 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #46gpt-5.1OpenAI | 72.86% | aa_slug=gpt-5-1 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #47gpt-5-6-solOpenAI | 72.65% | aa_slug=gpt-5-6-sol · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #48Qwen3.5-35B-A3BQwen | 72.52% | aa_slug=qwen3-5-35b-a3b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #49Gemma 4 26B A4BGoogle | 72.45% | aa_slug=gemma-4-26b-a4b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #50GLM 5Z.ai (Zhipu AI) | 72.31% | aa_slug=glm-5 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #51MiniMax M2MiniMax | 72.31% | aa_slug=minimax-m2 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #52mimo-v2-0206Xiaomi | 71.84% | aa_slug=mimo-v2-0206 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #53MiniMax M2.5MiniMax | 71.63% | aa_slug=minimax-m2-5 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #54gpt-5-5-highOpenAI | 71.63% | aa_slug=gpt-5-5-high · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #55Nemotron 3 SuperNVIDIA | 71.5% | aa_slug=nvidia-nemotron-3-super-120b-a12b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #56gpt-5-5-instant-05-26OpenAI | 71.5% | aa_slug=gpt-5-5-instant-05-26 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #57o3OpenAI | 71.43% | aa_slug=o3 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #58deepseek-v4-pro-0424-highDeepSeek | 71.29% | aa_slug=deepseek-v4-pro-0424-high · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #59gpt-5.6-terraOpenAI | 71.22% | aa_slug=gpt-5-6-terra · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #60Solar Pro 3Upstage | 71.2% | aa_slug=solar-pro-3 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #61gpt-5-mini-mediumOpenAI | 71.16% | aa_slug=gpt-5-mini-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #62nvidia-nemotron-3-nano-30b-a3b-reasoningNVIDIA | 71.09% | aa_slug=nvidia-nemotron-3-nano-30b-a3b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #63gpt-5-6-sol-xhighOpenAI | 71.02% | aa_slug=gpt-5-6-sol-xhigh · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #64gpt-5-5-mediumOpenAI | 70.95% | aa_slug=gpt-5-5-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #65Qwen3 Max ThinkingQwen | 70.75% | aa_slug=qwen3-max-thinking · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #66nova-2-0-lite-reasoningAmazon Web Services | 70.75% | aa_slug=nova-2-0-lite-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #67gpt-5 (medium)OpenAI | 70.61% | aa_slug=gpt-5-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #68gemini-3-proGoogle | 70.41% | aa_slug=gemini-3-pro · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #69o1OpenAI | 70.34% | aa_slug=o1 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #70Kimi K2.5Moonshot AI | 70.2% | aa_slug=kimi-k2-5 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #71GPT-5.1-CodexOpenAI | 70% | aa_slug=gpt-5-1-codex · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #72MiniMax M2.1MiniMax | 69.86% | aa_slug=minimax-m2-1 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #73Mercury 2Inception | 69.8% | aa_slug=mercury-2 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #74gpt-5-6-sol-mediumOpenAI | 69.59% | aa_slug=gpt-5-6-sol-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #75gpt-5-6-sol-highOpenAI | 69.18% | aa_slug=gpt-5-6-sol-high · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #76apriel-v1-6-15b-thinkerServiceNow | 69.12% | aa_slug=apriel-v1-6-15b-thinker · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #77gpt-oss-120bOpenAI | 68.98% | aa_slug=gpt-oss-120b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #78mimo-v2-proXiaomi | 68.84% | aa_slug=mimo-v2-pro · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #79Mistral Medium 3.5Mistral AI | 68.78% | aa_slug=mistral-medium-3-5 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #80o4-miniOpenAI | 68.71% | aa_slug=o4-mini · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #81nova-2-0-lite-reasoning-mediumAmazon Web Services | 68.5% | aa_slug=nova-2-0-lite-reasoning-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #82kat-coder-pro-v1KwaiKAT | 68.37% | aa_slug=kat-coder-pro-v1 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #83Kimi K2 ThinkingMoonshot AI | 68.1% | aa_slug=kimi-k2-thinking · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #84GLM 4.7Z.ai (Zhipu AI) | 67.89% | aa_slug=glm-4-7 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #85GPT-5.1-Codex MiniOpenAI | 67.89% | aa_slug=gpt-5-1-codex-mini · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #86gpt-5-nanoOpenAI | 67.55% | aa_slug=gpt-5-nano · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #87Qwen3.6 27BQwen | 67.55% | aa_slug=qwen3-6-27b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #88mimo-v2-omni-0327Xiaomi | 67.35% | aa_slug=mimo-v2-omni-0327 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #89Step 3.7 FlashStepFun | 67.28% | aa_slug=step-3-7-flash · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #90MiMo-V2.5Xiaomi | 67.14% | aa_slug=mimo-v2-5-0424 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #91o3 Mini HighOpenAI | 67.14% | aa_slug=o3-mini-high · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #92Qwen3.5-9BQwen | 66.73% | aa_slug=qwen3-5-9b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #93KAT-Coder-Pro V2Kwaipilot | 66.67% | aa_slug=kat-coder-pro-v2 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #94gpt-5 (low)OpenAI | 66.6% | aa_slug=gpt-5-low · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #95step-3-5-flashStepFun | 66.53% | aa_slug=step-3-5-flash · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #96gpt-5-6-sol-lowOpenAI | 66.53% | aa_slug=gpt-5-6-sol-low · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #97hypernova-60bMultiverse Computing | 66.46% | aa_slug=hypernova-60b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #98gpt-5-6-terra-xhighOpenAI | 66.26% | aa_slug=gpt-5-6-terra-xhigh · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #99nex-n2-proNex AGI | 66.19% | aa_slug=nex-n2-pro · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #100nova-2-0-omni-reasoning-mediumAmazon Web Services | 66.19% | aa_slug=nova-2-0-omni-reasoning-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History |
Scores are reported as published, with their evaluation configuration (harness, prompting, judge). The bar is relative to the best score on this page. Results with different configs are not directly comparable — see methodology.
Definition
- Category
- instruction-following
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 5 h agomedium
- Task
- precise instruction following with novel constraints
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 5 h agomedium
- Metric
- accuracy · %
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 5 h agomedium
- Direction
- Higher is better
- Paper
- https://arxiv.org/abs/2507.02833
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 5 h agomedium
Each value shows its source, tier and observation time. Missing rows mean no source stated them. How results are recorded →
- Evaluated models
- grok-3-mini-reasoning, gemma-4-12B, grok-4-3-low, GPT-5.1-Codex Mini, Qwen3 Coder Next, KAT-Coder-Pro V2, nova-2-0-lite-reasoning-medium, lfm2-24b-a2b +442(450 total)
As of
Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.
Claim history · Unit
1 claims · 1 propertiesShow all properties
Unitunit1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| % | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →
| Source | Document | Type | Tier | Last observed | Snapshots |
|---|---|---|---|---|---|
| Artificial Analysis | artificialanalysis.ai/leaderboards/models | leaderboard | T2· Quality secondary | 3 h ago | 1 |
Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.