Timeline of OpenAI

gpt-4o scores 6% on SWE-bench Multimodal

gpt-4o scores 9.28% on SWE-bench Multimodal

gpt-4o scores 11.99% on SWE-bench Multimodal

gpt-4o scores 12.19% on SWE-bench Multimodal

gpt-4o scores 30.37% on SWE-bench Multimodal

gpt-4.1 scores 31.14% on SWE-bench Multimodal

o4-mini scores 33.85% on SWE-bench Multimodal

o3 scores 35.98% on SWE-bench Multimodal

o3 scores 35.98% on SWE-bench Multimodal

GPT-3.5 scores 0.33% on SWE-bench Lite

GPT-4 (1106) scores 2.67% on SWE-bench Lite

GPT-4 (1106) scores 18% on SWE-bench Lite

gpt-4o scores 18.33% on SWE-bench Lite

GPT-4 (0125) scores 19% on SWE-bench Lite

gpt-4o scores 21.67% on SWE-bench Lite

gpt-4 scores 23.67% on SWE-bench Lite

gpt-4o scores 24.67% on SWE-bench Lite

gpt-4o scores 25.33% on SWE-bench Lite

gpt-4o scores 27.33% on SWE-bench Lite

gpt-4o scores 27.33% on SWE-bench Lite

gpt-4o scores 27.67% on SWE-bench Lite

gpt-4o scores 28% on SWE-bench Lite

GPT-4 (1106) scores 28.33% on SWE-bench Lite

gpt-4o scores 29.67% on SWE-bench Lite

o3-mini_1.0 scores 30.33% on SWE-bench Lite

gpt-4o scores 30.67% on SWE-bench Lite

gpt-4o scores 31.33% on SWE-bench Lite

gpt-4o scores 32% on SWE-bench Lite

o3-mini scores 32.33% on SWE-bench Lite

gpt-4o scores 34% on SWE-bench Lite

gpt-4o scores 38% on SWE-bench Lite

GPT-4 (0806) scores 39.67% on SWE-bench Lite

GPT-3.5 scores 0.4% on SWE-bench Verified

GPT-4 (1106) scores 2.8% on SWE-bench Verified

Lingma SWE-GPT 7b (v0918) scores 10.2% on SWE-bench Verified

Lingma SWE-GPT 7b (v0925) scores 18.2% on SWE-bench Verified

gpt-4o scores 21.62% on SWE-bench Verified

GPT-4 (1106) scores 22.4% on SWE-bench Verified

gpt-4o scores 23.2% on SWE-bench Verified

gpt-4.1-mini scores 23.94% on SWE-bench Verified

gpt-4o scores 24% on SWE-bench Verified

Lingma SWE-GPT 72b (v0918) scores 25% on SWE-bench Verified

gpt-oss-120b scores 26% on SWE-bench Verified

gpt-4o scores 26.2% on SWE-bench Verified

gpt-4o scores 27% on SWE-bench Verified

Lingma SWE-GPT 72b (v0925) scores 28.8% on SWE-bench Verified

gpt-4o scores 32.6% on SWE-bench Verified

gpt-5-nano scores 34.8% on SWE-bench Verified

gpt-4o scores 38.4% on SWE-bench Verified

gpt-4o scores 38.8% on SWE-bench Verified

gpt-4.1 scores 39.58% on SWE-bench Verified

o3-mini scores 42.4% on SWE-bench Verified

o4-mini scores 45% on SWE-bench Verified

gpt-5-mini scores 56.2% on SWE-bench Verified

o3 scores 58.4% on SWE-bench Verified

TTS(Bo16) scores 58.8% on SWE-bench Verified

gpt-5-mini scores 59.8% on SWE-bench Verified

o4-mini scores 64.6% on SWE-bench Verified

o1 Preview scores 64.6% on SWE-bench Verified

gpt-5 scores 65% on SWE-bench Verified

GPT-5.1-Codex scores 66% on SWE-bench Verified

gpt-5.1 scores 66% on SWE-bench Verified

gpt-5.2 scores 69% on SWE-bench Verified

gpt-5 scores 71.2% on SWE-bench Verified

gpt-5.2 scores 71.8% on SWE-bench Verified

gpt-5 scores 71.8% on SWE-bench Verified

GPT-5.2-Codex scores 72.8% on SWE-bench Verified

gpt-5.2 scores 72.8% on SWE-bench Verified

gpt-5 scores 74.4% on SWE-bench Verified

GPT-3.5 scores 0.17% on SWE-bench (full test split)

GPT-4 (1106) scores 1.31% on SWE-bench (full test split)

gpt-4o scores 11.99% on SWE-bench (full test split)

GPT-4 (1106) scores 12.47% on SWE-bench (full test split)

gpt-4o scores 14.6% on SWE-bench (full test split)

gpt-4o scores 18.83% on SWE-bench (full test split)

gpt-5-mini scores 39.7% on SWE-bench Multilingual

GPT-5.2-Codex scores 66.3% on SWE-bench Multilingual

gpt-5.2 scores 66.7% on SWE-bench Multilingual

gpt-6-astra scores 75.579% on LiveBench

gpt-6-astra scores 89.431% on LiveBench

gpt-6-astra scores 82.973% on LiveBench

gpt-6-astra scores 96.806% on LiveBench

gpt-6-astra scores 57.323% on LiveBench

gpt-6-astra scores 80.358% on LiveBench

gpt-6-astra scores 92.654% on LiveBench

gpt-6-astra scores 82.161% on LiveBench

gpt-5.6-luna scores 60.121% on LiveBench

gpt-5.6-luna scores 72.567% on LiveBench

gpt-5.6-luna scores 78.033% on LiveBench

gpt-5.6-luna scores 87.201% on LiveBench

gpt-5.6-luna scores 48.434% on LiveBench

gpt-5.6-luna scores 82.915% on LiveBench

gpt-5.6-luna scores 85.644% on LiveBench

gpt-5.6-luna scores 73.559% on LiveBench

gpt-5.6-terra scores 64.617% on LiveBench

gpt-5.6-terra scores 82.893% on LiveBench

gpt-5.6-terra scores 79.307% on LiveBench

gpt-5.6-terra scores 94.908% on LiveBench

gpt-5.6-terra scores 54.95% on LiveBench

gpt-5.6-terra scores 78.245% on LiveBench

gpt-5.6-terra scores 90.635% on LiveBench

gpt-5.6-terra scores 77.936% on LiveBench

gpt-5-6-sol scores 71.846% on LiveBench

gpt-5-6-sol scores 87.684% on LiveBench

gpt-5-6-sol scores 79.841% on LiveBench

gpt-5-6-sol scores 96.198% on LiveBench

gpt-5-6-sol scores 56.212% on LiveBench

gpt-5-6-sol scores 83.941% on LiveBench

gpt-5-6-sol scores 91.654% on LiveBench

gpt-5-6-sol scores 81.054% on LiveBench

gpt-5.5 scores 70.729% on LiveBench

gpt-5.5 scores 87.363% on LiveBench

gpt-5.5 scores 81.576% on LiveBench

gpt-5.5 scores 95.857% on LiveBench

gpt-5.5 scores 53.99% on LiveBench

gpt-5.5 scores 82.149% on LiveBench

gpt-5.5 scores 89.654% on LiveBench

gpt-5.5 scores 80.188% on LiveBench

gpt-5.4 scores 70.217% on LiveBench

gpt-5.4 scores 82.634% on LiveBench

gpt-5.4 scores 79.313% on LiveBench

gpt-5.4 scores 94.148% on LiveBench

gpt-5.4 scores 53.838% on LiveBench

gpt-5.4 scores 77.541% on LiveBench

gpt-5.4 scores 88.115% on LiveBench

gpt-5.4 scores 77.972% on LiveBench

gpt-5.4-nano scores 67.205% on LiveBench

gpt-5.4-nano scores 62.507% on LiveBench

gpt-5.4-nano scores 67.643% on LiveBench

gpt-5.4-nano scores 90.977% on LiveBench

gpt-5.4-nano scores 46.768% on LiveBench

gpt-5.4-nano scores 70.839% on LiveBench

gpt-5.4-nano scores 81.097% on LiveBench

gpt-5.4-nano scores 69.577% on LiveBench

gpt-5.4-mini scores 59.804% on LiveBench

gpt-5.4-mini scores 70.952% on LiveBench

gpt-5.4-mini scores 70.786% on LiveBench

gpt-5.4-mini scores 78.462% on LiveBench

gpt-5.4-mini scores 41.667% on LiveBench

gpt-5.4-mini scores 71.624% on LiveBench

gpt-5.4-mini scores 71.324% on LiveBench

gpt-5.4-mini scores 66.374% on LiveBench

GPT-5.2-Codex scores 66.446% on LiveBench

GPT-5.2-Codex scores 73.678% on LiveBench

GPT-5.2-Codex scores 78.204% on LiveBench

GPT-5.2-Codex scores 88.774% on LiveBench

GPT-5.2-Codex scores 49.394% on LiveBench

GPT-5.2-Codex scores 83.619% on LiveBench

GPT-5.2-Codex scores 77.712% on LiveBench

GPT-5.2-Codex scores 73.975% on LiveBench

GPT-5.2-Codex scores 37.12% on Terminal-Bench

GPT-5.2-Codex scores 92.11% on τ²-bench

GPT-5.2-Codex scores 76.3% on MMMU-Pro

GPT-5.2-Codex scores 77.62% on IFBench

GPT-5.2-Codex scores 35.73% on Humanity's Last Exam

GPT-5.2-Codex scores 89.9% on GPQA

GPT-5.2-Codex scores 28.5 on Artificial Analysis Intelligence Index

GPT-3.5 Turbo scores 29.7% on GPQA

GPT-3.5 Turbo scores 5.49 on Artificial Analysis Intelligence Index

GPT-5.1-Codex scores 34.85% on Terminal-Bench

GPT-5.1-Codex scores 83.04% on τ²-bench

GPT-5.1-Codex scores 72.49% on MMMU-Pro

GPT-5.1-Codex scores 70% on IFBench

GPT-5.1-Codex scores 25.72% on Humanity's Last Exam

GPT-5.1-Codex scores 85.96% on GPQA

GPT-5.1-Codex scores 23.7 on Artificial Analysis Intelligence Index

gpt-5-6-luna-low scores 43.45% on Terminal-Bench

gpt-5-6-luna-low scores 0% on Terminal-Bench

gpt-5-6-luna-low scores 73.99% on MMMU-Pro

gpt-5-6-luna-low scores 46.06% on SciCode

gpt-5-6-luna-low scores 19.83% on Humanity's Last Exam

gpt-5-6-luna-low scores 83.54% on GPQA

gpt-5-6-luna-low scores 21.55 on Artificial Analysis Intelligence Index

gpt-5-nano-minimal scores 6.82% on Terminal-Bench

gpt-5-nano-minimal scores 25.73% on τ²-bench

gpt-5-nano-minimal scores 31.79% on MMMU-Pro

gpt-5-nano-minimal scores 32.52% on IFBench

gpt-5-nano-minimal scores 3.99% on Humanity's Last Exam

gpt-5-nano-minimal scores 42.83% on GPQA

gpt-5-nano-minimal scores 7.09 on Artificial Analysis Intelligence Index

o3 Mini High scores 6.06% on Terminal-Bench

o3 Mini High scores 4.49% on Terminal-Bench

o3 Mini High scores 0% on Terminal-Bench

o3 Mini High scores 31.29% on τ²-bench

o3 Mini High scores 67.14% on IFBench

o3 Mini High scores 42.82% on SciCode

o3 Mini High scores 12.04% on Humanity's Last Exam

o3 Mini High scores 77.27% on GPQA

o3 Mini High scores 10.96 on Artificial Analysis Intelligence Index

o1 Preview scores 76.46% on GPQA

o1 Preview scores 11.38 on Artificial Analysis Intelligence Index

gpt-5-6-sol-low scores 60.61% on Terminal-Bench

gpt-5-6-sol-low scores 76.78% on Terminal-Bench

gpt-5-6-sol-low scores 1.01% on Terminal-Bench

gpt-5-6-sol-low scores 76.02% on τ²-bench

gpt-5-6-sol-low scores 80.98% on MMMU-Pro

gpt-5-6-sol-low scores 66.53% on IFBench

gpt-5-6-sol-low scores 56.37% on SciCode

gpt-5-6-sol-low scores 39.39% on Humanity's Last Exam

gpt-5-6-sol-low scores 89.8% on GPQA

gpt-5-6-sol-low scores 33.8 on Artificial Analysis Intelligence Index

gpt-oss-20b-low scores 4.55% on Terminal-Bench

gpt-oss-20b-low scores 50.29% on τ²-bench

gpt-oss-20b-low scores 57.82% on IFBench

gpt-oss-20b-low scores 5.28% on Humanity's Last Exam

gpt-oss-20b-low scores 61.11% on GPQA

gpt-oss-20b-low scores 9.95 on Artificial Analysis Intelligence Index

o3 scores 37.12% on Terminal-Bench

o3 scores 80.7% on τ²-bench

o3 scores 70.06% on MMMU-Pro

o3 scores 71.43% on IFBench

o3 scores 20.05% on Humanity's Last Exam

o3 scores 82.73% on GPQA

o3 scores 20.2 on Artificial Analysis Intelligence Index

gpt-5.5 scores 60.61% on Terminal-Bench

gpt-5.5 scores 84.27% on Terminal-Bench

gpt-5.5 scores 14.65% on Terminal-Bench

gpt-5.5 scores 93.86% on τ²-bench

gpt-5.5 scores 79.88% on MMMU-Pro

gpt-5.5 scores 75.85% on IFBench

gpt-5.5 scores 55.79% on SciCode

gpt-5.5 scores 45.78% on Humanity's Last Exam

gpt-5.5 scores 93.54% on GPQA

gpt-5.5 scores 38.63 on Artificial Analysis Intelligence Index

gpt-6-astra-xhigh scores 89.14% on Terminal-Bench

gpt-6-astra-xhigh scores 59.6% on Terminal-Bench

gpt-6-astra-xhigh scores 86.24% on MMMU-Pro

gpt-6-astra-xhigh scores 55.67% on SciCode

gpt-6-astra-xhigh scores 54.59% on Humanity's Last Exam

gpt-6-astra-xhigh scores 96.26% on GPQA

gpt-6-astra-xhigh scores 52.51 on Artificial Analysis Intelligence Index

gpt-6-astra-non-reasoning scores 89.14% on Terminal-Bench

gpt-6-astra-non-reasoning scores 50.51% on Terminal-Bench

gpt-6-astra-non-reasoning scores 82.02% on MMMU-Pro

gpt-6-astra-non-reasoning scores 53.47% on SciCode

gpt-6-astra-non-reasoning scores 37.12% on Humanity's Last Exam

gpt-6-astra-non-reasoning scores 89.49% on GPQA

gpt-6-astra-non-reasoning scores 45.17 on Artificial Analysis Intelligence Index

gpt-5-6-terra-medium scores 72.28% on Terminal-Bench

gpt-5-6-terra-medium scores 1.01% on Terminal-Bench

gpt-5-6-terra-medium scores 72.81% on τ²-bench

gpt-5-6-terra-medium scores 76.76% on MMMU-Pro

gpt-5-6-terra-medium scores 62.18% on IFBench

gpt-5-6-terra-medium scores 50.46% on SciCode

gpt-5-6-terra-medium scores 33.27% on Humanity's Last Exam

gpt-5-6-terra-medium scores 87.17% on GPQA

gpt-5-6-terra-medium scores 30.42 on Artificial Analysis Intelligence Index

gpt-5-mini scores 33.33% on Terminal-Bench

gpt-5-mini scores 3.75% on Terminal-Bench

gpt-5-mini scores 0% on Terminal-Bench

gpt-5-mini scores 68.42% on τ²-bench

gpt-5-mini scores 70.06% on MMMU-Pro

gpt-5-mini scores 75.44% on IFBench

gpt-5-mini scores 39% on SciCode

gpt-5-mini scores 21.46% on Humanity's Last Exam

gpt-5-mini scores 82.83% on GPQA

gpt-5-mini scores 17.41 on Artificial Analysis Intelligence Index

o3-mini scores 6.82% on Terminal-Bench

o3-mini scores 28.65% on τ²-bench

o3-mini scores 7.91% on Humanity's Last Exam

o3-mini scores 74.85% on GPQA

o3-mini scores 12.47 on Artificial Analysis Intelligence Index

gpt-6-astra scores 88.39% on Terminal-Bench

gpt-6-astra scores 59.09% on Terminal-Bench

gpt-6-astra scores 86.88% on MMMU-Pro

gpt-6-astra scores 56.48% on SciCode

gpt-6-astra scores 54.68% on Humanity's Last Exam

gpt-6-astra scores 96.06% on GPQA

gpt-6-astra scores 52.81 on Artificial Analysis Intelligence Index

gpt-oss-20b scores 10.61% on Terminal-Bench

gpt-oss-20b scores 13.86% on Terminal-Bench

gpt-oss-20b scores 0% on Terminal-Bench

gpt-oss-20b scores 60.23% on τ²-bench

gpt-oss-20b scores 65.1% on IFBench

gpt-oss-20b scores 38.89% on SciCode

gpt-oss-20b scores 10.98% on Humanity's Last Exam

gpt-oss-20b scores 68.79% on GPQA

gpt-oss-20b scores 9.04 on Artificial Analysis Intelligence Index

gpt-5-4-mini-medium scores 34.09% on Terminal-Bench

gpt-5-4-mini-medium scores 36.55% on τ²-bench

gpt-5-4-mini-medium scores 71.16% on MMMU-Pro