Timeline of Allen Institute for AI

Olmo-3-7B-Think scores 0% on τ²-bench

Olmo-3-7B-Think scores 41.5% on IFBench

Olmo-3-7B-Think scores 5.98% on Humanity's Last Exam

Olmo-3-7B-Think scores 51.62% on GPQA

Olmo-3-7B-Think scores 5.62 on Artificial Analysis Intelligence Index

olmo-3-1-32b-think scores 0% on Terminal-Bench

olmo-3-1-32b-think scores 0% on τ²-bench

olmo-3-1-32b-think scores 65.99% on IFBench

olmo-3-1-32b-think scores 6.26% on Humanity's Last Exam

olmo-3-1-32b-think scores 59.09% on GPQA

olmo-3-1-32b-think scores 7.12 on Artificial Analysis Intelligence Index

Molmo2-8B scores 0% on Terminal-Bench

Molmo2-8B scores 0% on τ²-bench

Molmo2-8B scores 37.46% on MMMU-Pro

Molmo2-8B scores 26.94% on IFBench

Molmo2-8B scores 4.26% on Humanity's Last Exam

Molmo2-8B scores 42.53% on GPQA

Molmo2-8B scores 5 on Artificial Analysis Intelligence Index

olmo-2-32b scores 0% on Terminal-Bench

olmo-2-32b scores 0% on τ²-bench

olmo-2-32b scores 38.1% on IFBench

olmo-2-32b scores 3.61% on Humanity's Last Exam

olmo-2-32b scores 32.83% on GPQA

olmo-2-32b scores 5.97 on Artificial Analysis Intelligence Index

Olmo-3-7B-Instruct scores 0% on Terminal-Bench

Olmo-3-7B-Instruct scores 12.57% on τ²-bench

Olmo-3-7B-Instruct scores 32.79% on IFBench

Olmo-3-7B-Instruct scores 5.79% on Humanity's Last Exam

Olmo-3-7B-Instruct scores 40% on GPQA

Olmo-3-7B-Instruct scores 5.24 on Artificial Analysis Intelligence Index

olmo-3-1-32b-instruct scores 0% on Terminal-Bench

olmo-3-1-32b-instruct scores 21.35% on τ²-bench

olmo-3-1-32b-instruct scores 39.18% on IFBench

olmo-3-1-32b-instruct scores 5% on Humanity's Last Exam

olmo-3-1-32b-instruct scores 53.94% on GPQA

olmo-3-1-32b-instruct scores 6.5 on Artificial Analysis Intelligence Index

molmo-7b-d scores 0% on Terminal-Bench

molmo-7b-d scores 0% on τ²-bench

molmo-7b-d scores 24.51% on MMMU-Pro

molmo-7b-d scores 19.66% on IFBench

molmo-7b-d scores 5.14% on Humanity's Last Exam

molmo-7b-d scores 24.04% on GPQA

molmo-7b-d scores 5.56 on Artificial Analysis Intelligence Index

olmo-3-32b-think scores 1.52% on Terminal-Bench

olmo-3-32b-think scores 0% on τ²-bench

olmo-3-32b-think scores 49.12% on IFBench

olmo-3-32b-think scores 6.44% on Humanity's Last Exam

olmo-3-32b-think scores 61.01% on GPQA

olmo-3-32b-think scores 6.47 on Artificial Analysis Intelligence Index

tulu3-405b scores 3.28% on Humanity's Last Exam

tulu3-405b scores 51.62% on GPQA

tulu3-405b scores 7.2 on Artificial Analysis Intelligence Index

olmo-2-7b scores 0% on Terminal-Bench

olmo-2-7b scores 0% on τ²-bench

olmo-2-7b scores 24.42% on IFBench

olmo-2-7b scores 5.38% on Humanity's Last Exam

olmo-2-7b scores 28.79% on GPQA

olmo-2-7b scores 5.58 on Artificial Analysis Intelligence Index