Timeline of Mistral AI

Devstral Small 1.0 scores 46.8% on SWE-bench Verified

Devstral 2 scores 53.8% on SWE-bench Verified

Devstral Small (2512) scores 56.4% on SWE-bench Verified

Z.ai GLM 5.2 scores 62.292% on LiveBench

Z.ai GLM 5.2 scores 76.242% on LiveBench

Z.ai GLM 5.2 scores 73.74% on LiveBench

Z.ai GLM 5.2 scores 89.781% on LiveBench

Z.ai GLM 5.2 scores 51.768% on LiveBench

Z.ai GLM 5.2 scores 79.654% on LiveBench

Z.ai GLM 5.2 scores 78.625% on LiveBench

Z.ai GLM 5.2 scores 73.157% on LiveBench

Mistral Medium 3.1 scores 10.61% on Terminal-Bench

Mistral Medium 3.1 scores 13.86% on Terminal-Bench

Mistral Medium 3.1 scores 0% on Terminal-Bench

Mistral Medium 3.1 scores 40.64% on τ²-bench

Mistral Medium 3.1 scores 54.16% on MMMU-Pro

Mistral Medium 3.1 scores 39.8% on IFBench

Mistral Medium 3.1 scores 32.41% on SciCode

Mistral Medium 3.1 scores 4.68% on Humanity's Last Exam

Mistral Medium 3.1 scores 58.79% on GPQA

Mistral Medium 3.1 scores 9.89 on Artificial Analysis Intelligence Index

Devstral 2 scores 18.94% on Terminal-Bench

Devstral 2 scores 30.34% on Terminal-Bench

Devstral 2 scores 0% on Terminal-Bench

Devstral 2 scores 24.85% on τ²-bench

Devstral 2 scores 38.1% on IFBench

Devstral 2 scores 32.75% on SciCode

Devstral 2 scores 3.57% on Humanity's Last Exam

Devstral 2 scores 59.39% on GPQA

Devstral 2 scores 9.41 on Artificial Analysis Intelligence Index

magistral-small scores 4.55% on Terminal-Bench

magistral-small scores 26.61% on τ²-bench

magistral-small scores 24.76% on IFBench

magistral-small scores 7.55% on Humanity's Last Exam

magistral-small scores 64.14% on GPQA

magistral-small scores 8.22 on Artificial Analysis Intelligence Index

mistral-small-4-non-reasoning scores 10.61% on Terminal-Bench

mistral-small-4-non-reasoning scores 18.42% on τ²-bench

mistral-small-4-non-reasoning scores 46.47% on MMMU-Pro

mistral-small-4-non-reasoning scores 32.79% on IFBench

mistral-small-4-non-reasoning scores 3.8% on Humanity's Last Exam

mistral-small-4-non-reasoning scores 57.07% on GPQA

mistral-small-4-non-reasoning scores 8.99 on Artificial Analysis Intelligence Index

Mistral Large 2.0 scores 33.04% on τ²-bench

Mistral Large 2.0 scores 31.63% on IFBench

Mistral Large 2.0 scores 2.95% on Humanity's Last Exam

Mistral Large 2.0 scores 47.17% on GPQA

Mistral Large 2.0 scores 6.8 on Artificial Analysis Intelligence Index

mistral-medium scores 3.51% on Humanity's Last Exam

mistral-medium scores 34.95% on GPQA

mistral-medium scores 5.49 on Artificial Analysis Intelligence Index

Mistral 7B scores 0% on τ²-bench

Mistral 7B scores 19.93% on IFBench

Mistral 7B scores 4.55% on Humanity's Last Exam

Mistral 7B scores 17.68% on GPQA

Mistral 7B scores 5.03 on Artificial Analysis Intelligence Index

Mistral Small 3.1 scores 7.58% on Terminal-Bench

Mistral Small 3.1 scores 26.22% on Terminal-Bench

Mistral Small 3.1 scores 0% on Terminal-Bench

Mistral Small 3.1 scores 25.15% on τ²-bench

Mistral Small 3.1 scores 29.93% on IFBench

Mistral Small 3.1 scores 27.78% on SciCode

Mistral Small 3.1 scores 4.31% on Humanity's Last Exam

Mistral Small 3.1 scores 45.35% on GPQA

Mistral Small 3.1 scores 7.37 on Artificial Analysis Intelligence Index

Mistral Small 1.0 scores 4.09% on Humanity's Last Exam

Mistral Small 1.0 scores 30.2% on GPQA

Mistral Small 1.0 scores 5.5 on Artificial Analysis Intelligence Index

Mixtral 8x7B scores 4.68% on Humanity's Last Exam

Mixtral 8x7B scores 29.19% on GPQA

Mixtral 8x7B scores 5.12 on Artificial Analysis Intelligence Index

Mistral Small 4 scores 17.42% on Terminal-Bench

Mistral Small 4 scores 20.97% on Terminal-Bench

Mistral Small 4 scores 0% on Terminal-Bench

Mistral Small 4 scores 41.23% on τ²-bench

Mistral Small 4 scores 56.82% on MMMU-Pro

Mistral Small 4 scores 48.16% on IFBench

Mistral Small 4 scores 38.77% on SciCode

Mistral Small 4 scores 9.87% on Humanity's Last Exam

Mistral Small 4 scores 76.87% on GPQA

Mistral Small 4 scores 11.45 on Artificial Analysis Intelligence Index

Mistral Small 3.2 scores 6.82% on Terminal-Bench

Mistral Small 3.2 scores 5.62% on Terminal-Bench

Mistral Small 3.2 scores 0% on Terminal-Bench

Mistral Small 3.2 scores 29.53% on τ²-bench

Mistral Small 3.2 scores 47.98% on MMMU-Pro

Mistral Small 3.2 scores 33.47% on IFBench

Mistral Small 3.2 scores 28.59% on SciCode

Mistral Small 3.2 scores 4.31% on Humanity's Last Exam

Mistral Small 3.2 scores 50.51% on GPQA

Mistral Small 3.2 scores 6.96 on Artificial Analysis Intelligence Index

Mistral Large 3 scores 15.91% on Terminal-Bench

Mistral Large 3 scores 11.99% on Terminal-Bench

Mistral Large 3 scores 0% on Terminal-Bench

Mistral Large 3 scores 24.56% on τ²-bench

Mistral Large 3 scores 55.66% on MMMU-Pro

Mistral Large 3 scores 36.19% on IFBench

Mistral Large 3 scores 36.57% on SciCode

Mistral Large 3 scores 4.17% on Humanity's Last Exam

Mistral Large 3 scores 67.98% on GPQA

Mistral Large 3 scores 9.71 on Artificial Analysis Intelligence Index

mistral-large-2 scores 6.06% on Terminal-Bench

mistral-large-2 scores 30.7% on τ²-bench

mistral-large-2 scores 31.22% on IFBench

mistral-large-2 scores 3.34% on Humanity's Last Exam

mistral-large-2 scores 48.59% on GPQA

mistral-large-2 scores 7.56 on Artificial Analysis Intelligence Index

Mistral Medium 3 scores 3.79% on Terminal-Bench

Mistral Medium 3 scores 24.27% on τ²-bench

Mistral Medium 3 scores 53.01% on MMMU-Pro

Mistral Medium 3 scores 39.25% on IFBench

Mistral Medium 3 scores 4.05% on Humanity's Last Exam

Mistral Medium 3 scores 57.78% on GPQA

Mistral Medium 3 scores 9.05 on Artificial Analysis Intelligence Index

Mistral Large scores 3.5% on Humanity's Last Exam

Mistral Large scores 35.05% on GPQA

Mistral Large scores 5.76 on Artificial Analysis Intelligence Index

magistral-medium scores 9.09% on Terminal-Bench

magistral-medium scores 23.1% on τ²-bench

magistral-medium scores 25.1% on IFBench

magistral-medium scores 9.82% on Humanity's Last Exam

magistral-medium scores 67.88% on GPQA

magistral-medium scores 9.05 on Artificial Analysis Intelligence Index

Ministral 3 3B scores 0% on Terminal-Bench

Ministral 3 3B scores 0% on Terminal-Bench

Ministral 3 3B scores 0% on Terminal-Bench

Ministral 3 3B scores 24.85% on τ²-bench

Ministral 3 3B scores 38.09% on MMMU-Pro

Ministral 3 3B scores 26.8% on IFBench

Ministral 3 3B scores 15.28% on SciCode

Ministral 3 3B scores 5.38% on Humanity's Last Exam

Ministral 3 3B scores 35.76% on GPQA

Ministral 3 3B scores 4.84 on Artificial Analysis Intelligence Index

Magistral Small 1.2 scores 4.55% on Terminal-Bench

Magistral Small 1.2 scores 4.49% on Terminal-Bench

Magistral Small 1.2 scores 27.78% on τ²-bench

Magistral Small 1.2 scores 55.49% on MMMU-Pro

Magistral Small 1.2 scores 44.35% on IFBench

Magistral Small 1.2 scores 6.44% on Humanity's Last Exam

Magistral Small 1.2 scores 66.26% on GPQA

Magistral Small 1.2 scores 8.59 on Artificial Analysis Intelligence Index

Ministral 3 14B scores 4.55% on Terminal-Bench

Ministral 3 14B scores 9.74% on Terminal-Bench

Ministral 3 14B scores 0% on Terminal-Bench

Ministral 3 14B scores 27.19% on τ²-bench

Ministral 3 14B scores 49.83% on MMMU-Pro

Ministral 3 14B scores 32.04% on IFBench

Ministral 3 14B scores 23.84% on SciCode

Ministral 3 14B scores 4.63% on Humanity's Last Exam

Ministral 3 14B scores 57.17% on GPQA

Ministral 3 14B scores 6.04 on Artificial Analysis Intelligence Index

Mistral Saba scores 4.3% on Humanity's Last Exam

Mistral Saba scores 42.42% on GPQA

Mistral Saba scores 6.49 on Artificial Analysis Intelligence Index

mistral-small scores 4.25% on Humanity's Last Exam

mistral-small scores 38.08% on GPQA

mistral-small scores 5.85 on Artificial Analysis Intelligence Index

Magistral Medium 1.2 scores 12.88% on Terminal-Bench

Magistral Medium 1.2 scores 12.36% on Terminal-Bench

Magistral Medium 1.2 scores 52.05% on τ²-bench

Magistral Medium 1.2 scores 59.65% on MMMU-Pro

Magistral Medium 1.2 scores 42.99% on IFBench

Magistral Medium 1.2 scores 10.29% on Humanity's Last Exam

Magistral Medium 1.2 scores 73.94% on GPQA

Magistral Medium 1.2 scores 11.83 on Artificial Analysis Intelligence Index

Ministral 3 8B scores 4.55% on Terminal-Bench

Ministral 3 8B scores 4.12% on Terminal-Bench

Ministral 3 8B scores 0% on Terminal-Bench

Ministral 3 8B scores 26.61% on τ²-bench

Ministral 3 8B scores 45.95% on MMMU-Pro

Ministral 3 8B scores 29.12% on IFBench

Ministral 3 8B scores 20.72% on SciCode

Ministral 3 8B scores 4.26% on Humanity's Last Exam

Ministral 3 8B scores 47.07% on GPQA

Ministral 3 8B scores 5.48 on Artificial Analysis Intelligence Index

devstral-small scores 6.06% on Terminal-Bench

devstral-small scores 28.36% on τ²-bench

devstral-small scores 34.56% on IFBench

devstral-small scores 3.75% on Humanity's Last Exam

devstral-small scores 41.41% on GPQA

devstral-small scores 7.6 on Artificial Analysis Intelligence Index

devstral-medium scores 9.09% on Terminal-Bench

devstral-medium scores 19.88% on τ²-bench

devstral-medium scores 29.93% on IFBench

devstral-medium scores 3.75% on Humanity's Last Exam

devstral-medium scores 49.19% on GPQA

devstral-medium scores 9.01 on Artificial Analysis Intelligence Index

Mixtral 8x22B scores 4.01% on Humanity's Last Exam

Mixtral 8x22B scores 33.23% on GPQA

Mixtral 8x22B scores 5.74 on Artificial Analysis Intelligence Index

Devstral Small 1.0 scores 6.06% on Terminal-Bench

Devstral Small 1.0 scores 38.01% on τ²-bench

Devstral Small 1.0 scores 31.63% on IFBench

Devstral Small 1.0 scores 4.03% on Humanity's Last Exam

Devstral Small 1.0 scores 43.43% on GPQA

Devstral Small 1.0 scores 8.74 on Artificial Analysis Intelligence Index

Devstral Small 2 scores 16.67% on Terminal-Bench

Devstral Small 2 scores 29.59% on Terminal-Bench

Devstral Small 2 scores 0% on Terminal-Bench

Devstral Small 2 scores 23.39% on τ²-bench

Devstral Small 2 scores 44.62% on MMMU-Pro

Devstral Small 2 scores 31.16% on IFBench

Devstral Small 2 scores 32.41% on SciCode

Devstral Small 2 scores 3.48% on Humanity's Last Exam

Devstral Small 2 scores 53.23% on GPQA

Devstral Small 2 scores 8.36 on Artificial Analysis Intelligence Index

Mistral Medium 3.5 scores 33.33% on Terminal-Bench

Mistral Medium 3.5 scores 50.56% on Terminal-Bench

Mistral Medium 3.5 scores 0% on Terminal-Bench

Mistral Medium 3.5 scores 94.15% on τ²-bench

Mistral Medium 3.5 scores 64.86% on MMMU-Pro

Mistral Medium 3.5 scores 68.78% on IFBench

Mistral Medium 3.5 scores 40.16% on SciCode

Mistral Medium 3.5 scores 13.76% on Humanity's Last Exam

Mistral Medium 3.5 scores 74.85% on GPQA

Mistral Medium 3.5 scores 14.89 on Artificial Analysis Intelligence Index

Pixtral Large scores 36.55% on τ²-bench

Pixtral Large scores 50.58% on MMMU-Pro

Pixtral Large scores 34.49% on IFBench

Pixtral Large scores 2.8% on Humanity's Last Exam

Pixtral Large scores 50.51% on GPQA

Pixtral Large scores 7.15 on Artificial Analysis Intelligence Index

Mistral Small 3 scores 19.59% on τ²-bench

Mistral Small 3 scores 26.39% on IFBench

Mistral Small 3 scores 3.76% on Humanity's Last Exam

Mistral Small 3 scores 46.16% on GPQA

Mistral Small 3 scores 6.67 on Artificial Analysis Intelligence Index

Z.ai GLM 5.2 scores 50.76% on Terminal-Bench

Z.ai GLM 5.2 scores 77.9% on Terminal-Bench

Z.ai GLM 5.2 scores 99.12% on τ²-bench

Z.ai GLM 5.2 scores 73.33% on IFBench

Z.ai GLM 5.2 scores 51.16% on SciCode

Z.ai GLM 5.2 scores 41.15% on Humanity's Last Exam

Z.ai GLM 5.2 scores 89.49% on GPQA

Z.ai GLM 5.2 scores 38.64 on Artificial Analysis Intelligence Index

Codestral 25.01 scores 100% on Aider polyglot

Codestral 25.01 scores 11.1% on Aider polyglot