BenchmarkActivecategory · instruction-following
IFBench
precise instruction following with novel constraints
quality51
Updated 8 h ago · first seen 11 Sept 2026
bench_01M293SPGX8Q1PSHHMPKC9R37H
- Metric
- accuracy · %
- Direction
- Higher is better
- Results
- 450
- Leader
- grok-4-3-medium 83.33%
Score history · nvidia-nemotron-3-nano-30b-a3b-reasoning 1 row
Not enough history to chart — a single observation (71.09% on 11 Sept 2026). Rows under different configurations count separately; the list below shows each one.
- 71.09%aa_slug=nvidia-nemotron-3-nano-30b-a3b-reasoning · evaluator=Artificial Analysis · index_version=4.311 Sept 2026
Leaderboard 450 current results
Select models with +, then open Compare.
| # | Model | Score | Config | Evaluated | Source | Actions |
|---|---|---|---|---|---|---|
| #101olmo-3-1-32b-thinkAllen Institute for AI | 65.99% | aa_slug=olmo-3-1-32b-think · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #102gpt-5-4-lowOpenAI | 65.92% | aa_slug=gpt-5-4-low · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #103gpt-5-nano-mediumOpenAI | 65.92% | aa_slug=gpt-5-nano-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #104gpt-5-2-mediumOpenAI | 65.24% | aa_slug=gpt-5-2-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #105gpt-oss-20bOpenAI | 65.1% | aa_slug=gpt-oss-20b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #106gpt-5-4-mini-mediumOpenAI | 64.76% | aa_slug=gpt-5-4-mini-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #107k-exaoneLG AI Research | 64.69% | aa_slug=k-exaone · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #108Step 3.5 FlashStepFun | 64.56% | aa_slug=step-3-5-flash-0202 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #109gpt-5-4-nano-mediumOpenAI | 64.42% | aa_slug=gpt-5-4-nano-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #110gpt-5-6-terra-highOpenAI | 64.42% | aa_slug=gpt-5-6-terra-high · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #111Qwen3.6 35B A3BQwen | 64.35% | aa_slug=qwen3-6-35b-a3b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #112gpt-5-5-lowOpenAI | 64.35% | aa_slug=gpt-5-5-low · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #113mimo-v2-flash-reasoningXiaomi | 64.22% | aa_slug=mimo-v2-flash-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #114deepseek-v3-2-specialeDeepSeek | 63.88% | aa_slug=deepseek-v3-2-speciale · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #115Claude Fable 5Anthropic | 63.47% | aa_slug=claude-fable-5 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #116nemotron-3-nano-omni-30b-a3bNVIDIA | 63.2% | aa_slug=nemotron-3-nano-omni-30b-a3b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #117Hy3 previewTencent | 63.13% | aa_slug=hy3-preview · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #118Kimi K2.7 CodeMoonshot AI | 63.13% | aa_slug=kimi-k2-7-code · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #119k2-think-v2MBZUAI Institute of Foundation Models | 62.79% | aa_slug=k2-think-v2 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #120Claude Opus 4.8Anthropic | 62.24% | aa_slug=claude-opus-4-8 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #121gpt-5-6-terra-mediumOpenAI | 62.18% | aa_slug=gpt-5-6-terra-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #122nova-2-0-omni-reasoning-lowAmazon Web Services | 61.77% | aa_slug=nova-2-0-omni-reasoning-low · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #123apriel-v1-5-15b-thinkerServiceNow | 61.7% | aa_slug=apriel-v1-5-15b-thinker · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #124nova-2-0-lite-reasoning-lowAmazon Web Services | 61.22% | aa_slug=nova-2-0-lite-reasoning-low · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #125GLM 5V TurboZ.ai (Zhipu AI) | 61.09% | aa_slug=glm-5v-turbo · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #126GLM 4.7 FlashZ.ai (Zhipu AI) | 60.82% | aa_slug=glm-4-7-flash · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #127DeepSeek V3.2DeepSeek | 60.68% | aa_slug=deepseek-v3-2-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #128qwen3-next-80b-a3b-reasoningAlibaba Group | 60.68% | aa_slug=qwen3-next-80b-a3b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #129k2-v2MBZUAI Institute of Foundation Models | 60.14% | aa_slug=k2-v2 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #130gpt-5-6-terra-lowOpenAI | 59.66% | aa_slug=gpt-5-6-terra-low · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #131diffusiongemma-26b-a4bGoogle | 59.46% | aa_slug=diffusiongemma-26b-a4b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #132qwen3-vl-32b-reasoningAlibaba Group | 59.39% | aa_slug=qwen3-vl-32b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #133Claude Opus 4.7Anthropic | 58.64% | aa_slug=claude-opus-4-7 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #134gpt-oss-120b-lowOpenAI | 58.3% | aa_slug=gpt-oss-120b-low · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #135nvidia-nemotron-3-nano-4bNVIDIA | 58.23% | aa_slug=nvidia-nemotron-3-nano-4b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #136claude-opus-4-5-thinkingAnthropic | 57.96% | aa_slug=claude-opus-4-5-thinking · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #137exaone-4-5-33bLG AI Research | 57.96% | aa_slug=exaone-4-5-33b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #138gpt-oss-20b-lowOpenAI | 57.82% | aa_slug=gpt-oss-20b-low · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #139solar-open-100b-reasoningUpstage | 57.69% | aa_slug=solar-open-100b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #140North Mini Code (free)Cohere | 57.55% | aa_slug=north-mini-code · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #141ling-2-6-flashinclusionAI | 57.41% | aa_slug=ling-2-6-flash · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #142claude-4-5-sonnet-thinkingAnthropic | 57.28% | aa_slug=claude-4-5-sonnet-thinking · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #143motif-2-12-7bMotif Technologies | 57.01% | aa_slug=motif-2-12-7b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #144deepseek-v3-1-terminus-reasoningDeepSeek | 57.01% | aa_slug=deepseek-v3-1-terminus-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #145ling-2-6-1tinclusionAI | 56.87% | aa_slug=ling-2-6-1t · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #146claude-sonnet-4-6-adaptiveAnthropic | 56.6% | aa_slug=claude-sonnet-4-6-adaptive · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #147qwen3-vl-235b-a22b-reasoningAlibaba Group | 56.46% | aa_slug=qwen3-vl-235b-a22b-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #148Trinity Large ThinkingArcee AI | 56.26% | aa_slug=trinity-large-thinking · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #149LFM2.5-8B-A1BLiquid AI | 55.65% | aa_slug=lfm2-5-8b-a1b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #150claude-4-1-opus-thinkingAnthropic | 55.44% | aa_slug=claude-4-1-opus-thinking · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #151glm-5-non-reasoningZ.ai (Zhipu AI) | 55.17% | aa_slug=glm-5-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #152k2-v2-mediumMBZUAI Institute of Foundation Models | 55.1% | aa_slug=k2-v2-medium · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #153gemini-3-flashGoogle | 55.1% | aa_slug=gemini-3-flash · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #154claude-4-sonnet-thinkingAnthropic | 54.69% | aa_slug=claude-4-sonnet-thinking · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #155tri-21b-think-v0-5Trillion Labs | 54.63% | aa_slug=tri-21b-think-v0-5 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #156glm-4-7-non-reasoningZ.ai (Zhipu AI) | 54.63% | aa_slug=glm-4-7-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #157falcon-h1r-7bTII UAE | 54.42% | aa_slug=falcon-h1r-7b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #158claude-4-5-haiku-reasoningAnthropic | 54.29% | aa_slug=claude-4-5-haiku-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #159DeepSeek V3.2 ExpDeepSeek | 54.15% | aa_slug=deepseek-v3-2-reasoning-0925 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #160qwen3-max-thinking-previewAlibaba Group | 53.81% | aa_slug=qwen3-max-thinking-preview · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #161Claude Opus 4Anthropic | 53.74% | aa_slug=claude-4-opus-thinking · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #162grok-4SpaceXAI | 53.67% | aa_slug=grok-4 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #163mimo-v2-omniXiaomi | 53.54% | aa_slug=mimo-v2-omni · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #164gemma-4-31b-non-reasoningGoogle | 53.47% | aa_slug=gemma-4-31b-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #165claude-opus-4-6-adaptiveAnthropic | 53.13% | aa_slug=claude-opus-4-6-adaptive · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #166grok-4-1-fast-reasoningSpaceXAI | 52.72% | aa_slug=grok-4-1-fast-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #167gemini-2-5-flash-lite-preview-09-2025-reasoningGoogle | 52.59% | aa_slug=gemini-2-5-flash-lite-preview-09-2025-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #168jamba-reasoning-3bAI21 Labs | 52.45% | aa_slug=jamba-reasoning-3b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #169gemini-2-5-flash-preview-09-2025-reasoningGoogle | 52.31% | aa_slug=gemini-2-5-flash-preview-09-2025-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #170nova-2-0-proAmazon Web Services | 52.04% | aa_slug=nova-2-0-pro · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #171glm-5-1-non-reasoningZ.ai (Zhipu AI) | 51.97% | aa_slug=glm-5-1-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #172Qwen3.5-4BQwen | 51.97% | aa_slug=qwen3-5-4b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #173qwen3-5-397b-a17b-non-reasoningAlibaba Group | 51.63% | aa_slug=qwen3-5-397b-a17b-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #174doubao-seed-codeByteDance | 51.43% | aa_slug=doubao-seed-code · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #175Qwen3 235B A22B Instruct 2507Qwen | 51.22% | aa_slug=qwen3-235b-a22b-instruct-2507-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #176qwen3-5-omni-plusAlibaba Group | 51.16% | aa_slug=qwen3-5-omni-plus · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #177qwen3-5-122b-a10b-non-reasoningAlibaba Group | 50.82% | aa_slug=qwen3-5-122b-a10b-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #178qwen3-30b-a3b-2507-reasoningAlibaba Group | 50.68% | aa_slug=qwen3-30b-a3b-2507-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #179grok-4-fast-reasoningSpaceXAI | 50.54% | aa_slug=grok-4-fast-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #180gemini-2-5-flash-reasoningGoogle | 50.27% | aa_slug=gemini-2-5-flash-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #181step-3-vl-10bStepFun | 50.2% | aa_slug=step-3-vl-10b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #182Gemini 2.5 Flash-LiteGoogle | 49.86% | aa_slug=gemini-2-5-flash-lite-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #183qwen3-4b-2507-instruct-reasoningAlibaba Group | 49.8% | aa_slug=qwen3-4b-2507-instruct-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #184gemini-3-pro-lowGoogle | 49.66% | aa_slug=gemini-3-pro-low · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #185minicpm5-1bOpenBMB | 49.32% | aa_slug=minicpm5-1b · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #186grok-4.20-0309-non-reasoningxAI | 49.32% | aa_slug=grok-4-20-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #187mi-dm-k-2-5-pro-dec28Korea Telecom | 49.32% | aa_slug=mi-dm-k-2-5-pro-dec28 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #188olmo-3-32b-thinkAllen Institute for AI | 49.12% | aa_slug=olmo-3-32b-think · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #189DeepSeek V3DeepSeek | 49.05% | aa_slug=deepseek-v3-2 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #190Gemini 2.5 ProGoogle | 48.71% | aa_slug=gemini-2-5-pro · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #191gpt-5-4-non-reasoningOpenAI | 48.44% | aa_slug=gpt-5-4-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #192claude-3-7-sonnet-thinkingAnthropic | 48.3% | aa_slug=claude-3-7-sonnet-thinking · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #193Mistral Small 4Mistral AI | 48.16% | aa_slug=mistral-small-4 · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #194qwen3-max-previewAlibaba Group | 48.03% | aa_slug=qwen3-max-preview · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #195hy3-non-reasoningTencent | 47.96% | aa_slug=hy3-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #196grok-4-20-0309-non-reasoningSpaceXAI | 47.82% | aa_slug=grok-4-20-0309-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #197grok-4-3-non-reasoningSpaceXAI | 47.62% | aa_slug=grok-4-3-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #198gpt-5-2-non-reasoningOpenAI | 47.41% | aa_slug=gpt-5-2-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #199gemini-3-5-flash-minimalGoogle | 47.28% | aa_slug=gemini-3-5-flash-minimal · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History | |
| #200deepseek-v4-flash-0420-non-reasoningDeepSeek | 47.21% | aa_slug=deepseek-v4-flash-0420-non-reasoning · evaluator=Artificial Analysis · index_version=4.3 | — (obs. 11 Sept 2026) | artificialanalysis.aiT2 | History |
Scores are reported as published, with their evaluation configuration (harness, prompting, judge). The bar is relative to the best score on this page. Results with different configs are not directly comparable — see methodology.
Definition
- Category
- instruction-following
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
- Task
- precise instruction following with novel constraints
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
- Metric
- accuracy · %
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
- Direction
- Higher is better
- Paper
- https://arxiv.org/abs/2507.02833
Source:AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2observed 8 h agomedium
Each value shows its source, tier and observation time. Missing rows mean no source stated them. How results are recorded →
- Evaluated models
- grok-3-mini-reasoning, gemma-4-12B, grok-4-3-low, GPT-5.1-Codex Mini, Qwen3 Coder Next, KAT-Coder-Pro V2, nova-2-0-lite-reasoning-medium, lfm2-24b-a2b +442(450 total)
As of
Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.
Claim history
5 claims · 5 properties
Categorycategory1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| instruction-following | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Metricmetric1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| accuracy | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Paperpaper1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| https://arxiv.org/abs/2507.02833 | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Tasktask1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| precise instruction following with novel constraints | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Unitunit1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| % | → current | current | AI Atlas curated registry (YAML, versioned in git, every entry carries its source URL)T2 | medium | curated |
Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →
| Source | Document | Type | Tier | Last observed | Snapshots |
|---|---|---|---|---|---|
| Artificial Analysis | artificialanalysis.ai/leaderboards/models | leaderboard | T2· Quality secondary | 6 h ago | 1 |
Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.