Automated page. Written and translated automatically by AI and checked against the sources by software; no human editor. SI-IQ is an index built from public benchmark results, not an IQ test. How it works
OTIS Mock AIME 2024–2025: AI model results
OTIS Mock AIME 2024–2025 is a Math benchmark run by Epoch AI. 193 models in our data have a published result.
It is part of the SI-IQ index (loading α = 1.38, chance level 0%).
- Evaluator
- Epoch AI
- Benchmark home page
- epoch.ai/benchmarks/otis-mock-aime-2024-2025
Results
| 1 | Claude Fable 5 Anthropic | 100.0% | 138134–141 | 9 |
| 2 | Claude Fable 5.1 Anthropic | 100.0% | 140136–143 | 8 |
| 3 | Claude Opus 5.5 Anthropic | 100.0% | 144141–148 | 9 |
| 4 | Claude Sonnet 5.5 Anthropic | 100.0% | 140137–143 | 9 |
| 5 | GPT-5.5 OpenAI | 100.0% | 138135–141 | 10 |
| 6 | GPT-5.5 Pro OpenAI | 100.0% | 139135–142 | 5 too little data |
| 7 | GPT-5.6 Sol OpenAI | 100.0% | 140137–143 | 9 |
| 8 | GPT-6 Astra OpenAI | 100.0% | 150146–153 | 9 |
| 9 | GPT-6 Sol OpenAI | 100.0% | 140137–144 | 9 |
| 10 | GPT-6.1 Sol OpenAI | 100.0% | 148144–152 | 9 |
| 11 | Gemini 4 Argon Google DeepMind | 100.0% | 140137–143 | 9 |
| 12 | Qwen3.8 Max (0902) Alibaba | 100.0% | 127123–130 | 6 too little data |
| 13 | GPT-5.6 Terra OpenAI | 99.7% | 134131–137 | 9 |
| 14 | Muse Wonder Meta AI | 99.4% | 136133–139 | 4 too little data |
| 15 | Qwen 3.8 Max Alibaba | 99.4% | 130127–133 | 7 too little data |
| 16 | Grok 4.6 xAI | 99.2% | 126123–129 | 9 |
| 17 | Muse Spark 1.3 Meta AI | 99.2% | 128125–131 | 5 too little data |
| 18 | Claude Haiku 5.5 Anthropic | 98.9% | 129126–132 | 9 |
| 19 | Claude Opus 5 Anthropic | 98.9% | 136133–140 | 9 |
| 20 | GPT-6 Luna OpenAI | 98.9% | 130127–133 | 9 |
| 21 | Gemini 3.8 Flash Google DeepMind | 98.9% | 130127–133 | 9 |
| 22 | DeepSeek V4 Pro 0813open DeepSeek | 98.6% | 128125–131 | 9 |
| 23 | Claude Opus 4.8 Anthropic | 98.3% | 132129–135 | 9 |
| 24 | DeepSeek V4.1 Flashopen DeepSeek | 98.3% | 128125–131 | 9 |
| 25 | GPT-5.6 Luna OpenAI | 98.3% | 130127–133 | 9 |
| 26 | Mistral Large 4 Mistral AI | 98.3% | 122119–125 | 7 too little data |
| 27 | Grok 4.7 xAI | 98.1% | 123120–127 | 7 too little data |
| 28 | Muse Spark Openopen Meta AI | 98.1% | 127124–130 | 7 |
| 29 | Claude Opus 4.7 Anthropic | 97.8% | 127124–130 | 10 |
| 30 | GPT-5.4 OpenAI | 97.8% | 131128–133 | 10 |
| 31 | Grok 4.5 xAI | 97.8% | 124121–127 | 6 too little data |
| 32 | Gemini 3.7 Flash Google DeepMind | 97.2% | 131128–134 | 9 |
| 33 | Kimi K3open Moonshot | 97.2% | 127124–130 | 9 |
| 34 | DeepSeek-V4-Proopen DeepSeek | 96.7% | 117113–120 | 10 |
| 35 | GPT-5.2 OpenAI | 96.1% | 125122–128 | 11 |
| 36 | Kimi K2.6open Moonshot | 96.1% | 121118–124 | 10 |
| 37 | Gemini 3.1 Pro Google DeepMind | 95.6% | 127124–130 | 10 |
| 38 | Gemini 3 Flash Google DeepMind | 95.6% | 122119–125 | 10 |
| 39 | Gemini 3.5 Flash Google DeepMind | 95.6% | 126123–129 | 10 |
| 40 | Kimi K2.7 Codeopen Moonshot | 95.6% | 118115–122 | 8 |
| 41 | Qwen3.7-Max Alibaba | 95.6% | 125122–128 | 10 |
| 42 | Claude Sonnet 5 Anthropic | 94.7% | 126123–129 | 9 |
| 43 | Claude Opus 4.6 Anthropic | 94.4% | 123120–126 | 10 |
| 44 | DeepSeek V4 Flash 0731open DeepSeek | 94.4% | 124121–127 | 9 |
| 45 | Gemini 3.6 Flash Google DeepMind | 94.2% | 125122–128 | 9 |
| 46 | GLM-5.3-Flashopen Z.ai (Zhipu AI) | 93.9% | 120117–124 | 8 |
| 47 | GLM-5.1open Z.ai (Zhipu AI) | 93.3% | 116112–120 | 8 |
| 48 | Grok 4.3 Beta xAI | 93.3% | 117114–121 | 6 too little data |
| 49 | Qwen 3.6 Plus Alibaba | 93.3% | 113109–117 | 7 too little data |
| 50 | Qwen3.7-Plus Alibaba | 93.3% | 115110–119 | 6 |
| 51 | Kimi K2.5open Moonshot | 92.2% | 119112–125 | 6 |
| 52 | GPT-5 OpenAI | 91.4% | 121118–124 | 12 |
| 53 | Gemini 3 Pro Google DeepMind | 91.4% | 122115–130 | 4 too little data |
| 54 | GLM-5.3open Z.ai (Zhipu AI) | 91.1% | 125122–128 | 9 |
| 55 | Qwen 3.6 Max (Preview) Alibaba | 91.1% | 118112–124 | 5 too little data |
| 56 | Qwen3.6 27Bopen Alibaba | 91.1% | 115110–119 | 6 |
| 57 | Inkling-Smallopen Thinking Machines | 90.0% | 118115–122 | 8 |
| 58 | Muse Spark Meta AI | 88.9% | 118110–126 | 2 too little data |
| 59 | GPT-5.4 Mini OpenAI | 88.9% | 117113–120 | 9 |
| 60 | Inklingopen Thinking Machines | 88.9% | 114110–117 | 8 |
| 61 | Qwen3.5 397B-A17Bopen Alibaba | 88.9% | 114109–118 | 6 |
| 62 | gpt-oss-120bopen OpenAI | 88.9% | 10699–112 | 6 |
| 63 | GPT-5.1 OpenAI | 88.6% | 120115–125 | 8 |
| 64 | GPT-5.4 Nano OpenAI | 87.8% | 118115–121 | 9 |
| 65 | GPT-5 mini OpenAI | 86.7% | 115112–118 | 12 |
| 66 | Nemotron 3 Ultraopen Nvidia | 86.7% | 114108–120 | 4 too little data |
| 67 | Qwen 3.5 Plus (hosted 397B-A17B) Alibaba | 86.7% | 109103–115 | 5 too little data |
| 68 | Qwen 3.6 35B-A3Bopen Alibaba | 86.7% | 111107–115 | 6 |
| 69 | Qwen3-235B-A22B-Thinking (Jul 2025)open Alibaba | 86.7% | 111105–118 | 6 |
| 70 | Qwen3.7 Flash Alibaba | 86.7% | 107103–112 | 4 too little data |
| 71 | GLM-5.2open Z.ai (Zhipu AI) | 86.4% | 121118–124 | 10 |
| 72 | Claude Opus 4.5 Anthropic | 86.1% | 115111–118 | 11 |
| 73 | Claude Sonnet 4.6 Anthropic | 85.8% | 113108–118 | 8 |
| 74 | Gemini 2.5 Pro (Jun 2025) Google DeepMind | 84.7% | 108104–112 | 8 |
| 75 | Qwen 3.5 Flash (hosted 35B-A3B) Alibaba | 84.4% | 107103–111 | 6 too little data |
| 76 | Qwen 3.6 Flash Alibaba | 84.4% | 107103–111 | 6 too little data |
| 77 | o3 OpenAI | 84.4% | 116113–119 | 10 |
| 78 | Grok 4 xAI | 84.0% | 114107–120 | 4 too little data |
| 79 | GLM-4.7open Z.ai (Zhipu AI) | 83.3% | 112107–118 | 6 |
| 80 | Gemma 4 26B A4Bopen Google DeepMind | 82.2% | 106100–112 | 4 too little data |
| 81 | o4-mini OpenAI | 81.7% | 112109–115 | 10 |
| 82 | GPT-5 nano OpenAI | 81.1% | 105102–109 | 11 |
| 83 | GLM-5open Z.ai (Zhipu AI) | 80.0% | 112105–118 | 5 too little data |
| 84 | Gemini 3.1 Flash-Lite Google | 80.0% | 109105–114 | 3 too little data |
| 85 | Claude Sonnet 4.5 Anthropic | 77.8% | 110107–113 | 12 |
| 86 | Grok-3 mini xAI | 77.8% | 10095–105 | 4 too little data |
| 87 | o3-mini OpenAI | 76.9% | 105101–108 | 9 |
| 88 | Gemma 4 31B ITopen Google DeepMind | 73.3% | 10498–110 | 5 too little data |
| 89 | Qwen3-Max Alibaba | 73.3% | 106103–110 | 6 too little data |
| 90 | o1 OpenAI | 73.3% | 10399–106 | 6 too little data |
| 91 | Gemini 2.5 Flash (Apr 2025) Google DeepMind | 73.1% | 10194–108 | 2 too little data |
| 92 | Claude Sonnet 4 Anthropic | 71.1% | 9793–101 | 6 |
| 93 | Gemini 3.5 Flash-Lite Google DeepMind | 71.1% | 109106–113 | 8 |
| 94 | MiniMax-M3open MiniMax | 71.1% | 108102–114 | 6 |
| 95 | Gemini 2.5 Flash (May 2025) Google DeepMind | 70.8% | 10294–109 | 2 too little data |
| 96 | Qwen3-30B-A3B-Thinking (Jul 2025)open Alibaba | 70.3% | 10093–106 | 2 too little data |
| 97 | Qwen3.5-35B-A3Bopen Alibaba | 70.0% | 106100–112 | 4 too little data |
| 98 | Claude Opus 4.1 Anthropic | 68.9% | 105102–109 | 6 too little data |
| 99 | GPT-5.5 Instant OpenAI | 68.1% | 107103–111 | 4 too little data |
| 100 | seed-oss-36b-instruct — | 67.5% | 9993–106 | 2 too little data |
| 101 | Qwen3-32Bopen Alibaba | 66.9% | 9589–100 | 6 |
| 102 | Claude Haiku 4.5 Anthropic | 66.7% | 10499–108 | 8 |
| 103 | DeepSeek-R1 (May 2025)open DeepSeek | 66.4% | 104100–109 | 5 too little data |
| 104 | Qwen3-14Bopen Alibaba | 66.4% | 9387–99 | 5 too little data |
| 105 | gpt-oss-20bopen OpenAI | 65.3% | 9589–101 | 4 too little data |
| 106 | Claude Opus 4 Anthropic | 64.4% | 9893–102 | 5 |
| 107 | Qwen3-30B-A3Bopen Alibaba | 62.8% | 9488–100 | 4 too little data |
| 108 | Qwen3-30B-A3B-Instruct (Jul 2025)open Alibaba | 62.2% | 9186–97 | 5 too little data |
| 109 | Qwen3.5-9Bopen Alibaba | 61.7% | 10195–107 | 4 too little data |
| 110 | QwQ-32Bopen Alibaba | 59.2% | 9286–98 | 3 too little data |
| 111 | glm-4.7-flash — | 58.3% | 9488–100 | 4 too little data |
| 112 | Claude 3.7 Sonnet Anthropic | 57.8% | 9995–103 | 5 |
| 113 | Gemini 2.0 Flash Thinking (Jan 2025) Google DeepMind, Google | 57.8% | 8983–95 | 3 too little data |
| 114 | Qwen3-8Bopen Alibaba | 56.1% | 9184–97 | 3 too little data |
| 115 | Qwen3.5-4Bopen Alibaba | 55.8% | 9688–103 | 1 too little data |
| 116 | DeepSeek-R1-Distill-Qwen-32Bopen DeepSeek | 55.6% | 9387–100 | 2 too little data |
| 117 | Grok 3 xAI | 55.6% | 9691–100 | 4 too little data |
| 118 | DeepSeek-R1open DeepSeek | 53.3% | 9893–102 | 4 too little data |
| 119 | qwen3-4b-instruct-2507 — | 52.2% | 8579–92 | 3 too little data |
| 120 | DeepSeek-R1-Distill-Llama-70Bopen DeepSeek | 51.4% | 9288–97 | 3 too little data |
| 121 | DeepSeek-R1-Distill-Qwen-14Bopen DeepSeek | 50.6% | 8984–93 | 3 too little data |
| 122 | o1-mini OpenAI | 46.9% | 9287–96 | 4 too little data |
| 123 | GPT-4.1 mini OpenAI | 44.7% | 9188–95 | 11 |
| 124 | deepseek-r1-0528-qwen3-8b — | 43.9% | 9083–98 | 2 too little data |
| 125 | GPT-4.1 OpenAI | 38.3% | 9187–94 | 11 |
| 126 | DeepSeek-V3 (Mar 2025)open DeepSeek | 37.8% | 8581–90 | 5 too little data |
| 127 | GPT-4.5 OpenAI | 37.8% | 8783–92 | 3 too little data |
| 128 | Mistral Medium 3 Mistral AI | 32.2% | 8681–90 | 4 too little data |
| 129 | Gemini 2.0 Flash (Feb 2025) Google DeepMind, Google | 31.1% | 8782–91 | 3 too little data |
| 130 | o1-preview OpenAI | 31.1% | 8480–89 | 3 too little data |
| 131 | Mistral Small 3.2open Mistral AI | 30.3% | 8074–87 | 3 too little data |
| 132 | Magistral Small 1.0open Mistral AI | 30.0% | 8376–89 | 2 too little data |
| 133 | GPT-4.1 nano OpenAI | 28.9% | 7672–80 | 9 |
| 134 | Magistral Small 1.2open Mistral AI | 28.1% | 7972–86 | 2 too little data |
| 135 | glm-4.7-flash_none — | 25.0% | 7770–84 | 2 too little data |
| 136 | Gemini 1.5 Pro (Sept 2024) Google DeepMind | 23.1% | 8075–85 | 3 too little data |
| 137 | Gemma 3 27Bopen Google DeepMind | 22.5% | 7772–81 | 5 |
| 138 | DeepSeek-R1-Distill-Qwen-1.5Bopen DeepSeek | 21.4% | 7365–80 | 2 too little data |
| 139 | Llama 4 Maverickopen Meta AI | 20.6% | 8177–85 | 7 |
| 140 | Qwen Plus (Jan 2025) Alibaba | 17.8% | 7671–81 | 3 too little data |
| 141 | Gemma 3 12Bopen Google DeepMind | 16.7% | 7366–79 | 5 too little data |
| 142 | Gemini 1.5 Flash (Sep 2024) Google DeepMind | 16.2% | 7470–79 | 3 too little data |
| 143 | Qwen2.5-Max Alibaba | 16.1% | 7873–82 | 4 too little data |
| 144 | DeepSeek-V3open DeepSeek | 15.8% | 7773–82 | 5 too little data |
| 145 | Phi-4open Microsoft Research | 13.8% | 7671–81 | 4 too little data |
| 146 | Grok-2 (Dec 2024) xAI | 11.5% | 7570–80 | 3 too little data |
| 147 | Llama 3.1-405Bopen Meta AI | 9.7% | 7065–75 | 3 too little data |
| 148 | Claude 3.5 Sonnet (October 2024) Anthropic | 8.5% | 7671–80 | 4 too little data |
| 149 | Mistral Large 2 (Jul 2024)open Mistral AI | 8.5% | 6863–73 | 3 too little data |
| 150 | Qwen2.5-72Bopen Alibaba | 8.1% | 7268–77 | 5 too little data |
| 151 | Qwen3-1.7Bopen Alibaba | 8.1% | 6557–73 | 3 too little data |
| 152 | Llama 4 Scoutopen Meta AI | 7.8% | 7368–78 | 4 too little data |
| 153 | Mistral Large 2 (Nov 2024)open Mistral AI | 7.8% | 7166–76 | 4 too little data |
| 154 | Gemma 3 4Bopen Google DeepMind | 7.5% | 6353–73 | 4 too little data |
| 155 | Qwen2.5-32Bopen Alibaba | 7.4% | 7166–76 | 3 too little data |
| 156 | GPT-4o mini OpenAI | 6.9% | 6864–73 | 10 |
| 157 | Gemini 1.5 Pro (May 2024) Google DeepMind | 6.8% | 6661–71 | 3 too little data |
| 158 | GPT-4 Turbo (Apr 2024) OpenAI | 6.7% | 6864–73 | 4 too little data |
| 159 | Mistral Small 3open Mistral AI | 6.7% | 6762–72 | 3 too little data |
| 160 | Claude 3.5 Sonnet Anthropic | 6.5% | 7166–76 | 3 too little data |
| 161 | GPT-4o (Aug 2024) OpenAI | 6.4% | 7368–77 | 9 |
| 162 | GPT-4o (May 2024) OpenAI | 6.2% | 7066–75 | 5 too little data |
| 163 | GPT-4o (Nov 2024) OpenAI | 6.2% | 6964–73 | 5 |
| 164 | Qwen-Turbo Alibaba | 6.1% | 6964–75 | 3 too little data |
| 165 | Mistral Small 3.1open Mistral AI | 5.8% | 6863–73 | 3 too little data |
| 166 | Llama 3.3 70Bopen Meta AI | 5.1% | 6661–71 | 6 |
| 167 | Claude 3 Opus Anthropic | 4.7% | 6560–70 | 5 too little data |
| 168 | Gemini 1.5 Flash 8B Google DeepMind | 4.6% | 5747–67 | 2 too little data |
| 169 | Claude 3.5 Haiku Anthropic | 4.3% | 6762–72 | 4 too little data |
| 170 | Llama 3-70Bopen Meta AI | 4.3% | 5752–62 | 3 too little data |
| 171 | Gemini 1.5 Flash (May 2024) Google DeepMind | 3.9% | 5853–63 | 3 too little data |
| 172 | Llama 3.1-70Bopen Meta AI | 3.6% | 6358–68 | 3 too little data |
| 173 | granite-4.0-micro — | 2.8% | 5441–67 | 2 too little data |
| 174 | Claude 2 Anthropic | 2.5% | 4842–53 | 3 too little data |
| 175 | Claude 3 Sonnet Anthropic | 2.5% | 5449–59 | 3 too little data |
| 176 | Hermes 2 Theta Llama-3 70Bopen Nous Research, Arcee AI | 2.5% | 5550–61 | 3 too little data |
| 177 | Qwen2.5-7Bopen Alibaba | 2.5% | 5141–60 | 4 too little data |
| 178 | GPT-3.5 Turbo (Jan 2024) OpenAI | 2.2% | 4742–53 | 5 too little data |
| 179 | Claude 2.1 Anthropic | 1.9% | 5240–64 | 2 too little data |
| 180 | Llama 3-8Bopen Meta AI | 1.9% | 4034–46 | 3 too little data |
| 181 | Mistral Large Mistral AI | 1.9% | 5651–62 | 3 too little data |
| 182 | Claude 3 Haiku Anthropic | 1.8% | 5045–56 | 3 too little data |
| 183 | Llama 3.1-8Bopen Meta AI | 1.7% | 5549–60 | 4 too little data |
| 184 | Gemma 2 27Bopen Google DeepMind | 1.4% | 5752–63 | 3 too little data |
| 185 | GPT-4 (Jun 2023) OpenAI | 1.1% | 5751–63 | 4 too little data |
| 186 | Gemini 1.0 Pro Google DeepMind | 1.1% | 4640–52 | 3 too little data |
| 187 | Gemma 3 1Bopen Google DeepMind | 1.1% | 4025–55 | 3 too little data |
| 188 | granite-4.0-1b — | 0.8% | 5537–74 | 2 too little data |
| 189 | GPT-4 (Mar 2023) OpenAI | 0.6% | 5240–65 | 2 too little data |
| 190 | Gemma 2 9Bopen Google DeepMind | 0.6% | 5347–59 | 3 too little data |
| 191 | Llama 3.2 1Bopen Meta AI | 0.6% | 4227–58 | 3 too little data |
| 192 | Mistral 7B v0.3open Mistral AI | 0.3% | 5434–75 | 2 too little data |
| 193 | granite-4.0-350m — | 0.3% | 5337–68 | 3 too little data |
Data sources: Aider polyglot leaderboard data (Apache-2.0); Arena leaderboard dataset on Hugging Face (CC BY 4.0); Berkeley Function Calling Leaderboard, Gorilla project (Apache-2.0); Epoch AI, ‘Capabilities & benchmarking’ (CC BY 4.0); LiteLLM, model_prices_and_context_window.json (MIT); Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter (licences and changes).