Automated page. Written and translated automatically by AI and checked against the sources by software; no human editor. SI-IQ is an index built from public benchmark results, not an IQ test. How it works
GPQA Diamond: AI model results
GPQA Diamond is a Knowledge benchmark run by Epoch AI. 205 models in our data have a published result.
It is part of the SI-IQ index (loading α = 0.75, chance level 25%).
- Evaluator
- Epoch AI
- Benchmark home page
- arxiv.org/abs/2311.12022
Results
| 1 | Gemini 4 Argon Google DeepMind | 96.4% | 140137–143 | 9 |
| 2 | GPT-6 Astra OpenAI | 95.8% | 150146–153 | 9 |
| 3 | Claude Sonnet 5.5 Anthropic | 95.6% | 140137–143 | 9 |
| 4 | GPT-6.1 Sol OpenAI | 95.4% | 148144–152 | 9 |
| 5 | Gemini 3.8 Flash Google DeepMind | 95.4% | 130127–133 | 9 |
| 6 | Gemini 3.7 Flash Google DeepMind | 94.8% | 131128–134 | 9 |
| 7 | GPT-5.4 Pro OpenAI | 94.6% | 135131–138 | 5 too little data |
| 8 | Gemini 3.1 Pro Google DeepMind | 94.4% | 127124–130 | 10 |
| 9 | GPT-6 Sol OpenAI | 94.3% | 140137–144 | 9 |
| 10 | Gemini 3.6 Flash Google DeepMind | 94.1% | 125122–128 | 9 |
| 11 | GPT-5.5 OpenAI | 94.0% | 138135–141 | 10 |
| 12 | Grok 4.6 xAI | 94.0% | 126123–129 | 9 |
| 13 | GPT-5.5 Pro OpenAI | 93.9% | 139135–142 | 5 too little data |
| 14 | Claude Opus 5 Anthropic | 93.9% | 136133–140 | 9 |
| 15 | GPT-5.6 Sol OpenAI | 93.5% | 140137–143 | 9 |
| 16 | Grok 4.5 xAI | 93.4% | 124121–127 | 6 too little data |
| 17 | GPT-5.6 Terra OpenAI | 93.3% | 134131–137 | 9 |
| 18 | GPT-5.4 OpenAI | 93.3% | 131128–133 | 10 |
| 19 | Kimi K3open Moonshot | 93.1% | 127124–130 | 9 |
| 20 | Gemini 3.5 Flash Google DeepMind | 92.8% | 126123–129 | 10 |
| 21 | Grok 4.7 xAI | 92.7% | 123120–127 | 7 too little data |
| 22 | Qwen 3.8 Max Alibaba | 92.7% | 130127–133 | 7 too little data |
| 23 | Gemini 3 Pro Google DeepMind | 92.6% | 122115–130 | 4 too little data |
| 24 | Qwen3.8 Max (0902) Alibaba | 92.3% | 127123–130 | 6 too little data |
| 25 | GLM-5.2open Z.ai (Zhipu AI) | 91.9% | 121118–124 | 10 |
| 26 | DeepSeek V4 Pro 0813open DeepSeek | 91.7% | 128125–131 | 9 |
| 27 | GPT-5.6 Luna OpenAI | 91.6% | 130127–133 | 9 |
| 28 | GPT-5.2 OpenAI | 91.4% | 125122–128 | 11 |
| 29 | Claude Opus 4.8 Anthropic | 91.0% | 132129–135 | 9 |
| 30 | DeepSeek V4 Flash 0731open DeepSeek | 91.0% | 124121–127 | 9 |
| 31 | DeepSeek-V4-Proopen DeepSeek | 90.9% | 117113–120 | 10 |
| 32 | GLM-5.3open Z.ai (Zhipu AI) | 90.9% | 125122–128 | 9 |
| 33 | MiniMax-M3open MiniMax | 90.9% | 108102–114 | 6 |
| 34 | Qwen3.7-Max Alibaba | 90.9% | 125122–128 | 10 |
| 35 | Kimi K2.6open Moonshot | 90.8% | 121118–124 | 10 |
| 36 | Claude Opus 5.5 Anthropic | 90.6% | 144141–148 | 9 |
| 37 | Claude Opus 4.6 Anthropic | 90.5% | 123120–126 | 10 |
| 38 | Claude Sonnet 5 Anthropic | 90.5% | 126123–129 | 9 |
| 39 | GPT-6 Luna OpenAI | 90.5% | 130127–133 | 9 |
| 40 | Claude Opus 4.7 Anthropic | 90.2% | 127124–130 | 10 |
| 41 | GLM-5.3-Flashopen Z.ai (Zhipu AI) | 90.2% | 120117–124 | 8 |
| 42 | GLM-5.1open Z.ai (Zhipu AI) | 89.9% | 116112–120 | 8 |
| 43 | DeepSeek V4.1 Flashopen DeepSeek | 89.8% | 128125–131 | 9 |
| 44 | Muse Spark Meta AI | 89.8% | 118110–126 | 2 too little data |
| 45 | Claude Haiku 5.5 Anthropic | 89.6% | 129126–132 | 9 |
| 46 | Gemini 3 Flash Google DeepMind | 89.4% | 122119–125 | 10 |
| 47 | Grok 4.3 Beta xAI | 88.8% | 117114–121 | 6 too little data |
| 48 | Muse Spark Openopen Meta AI | 88.6% | 127124–130 | 7 |
| 49 | Inkling-Smallopen Thinking Machines | 88.5% | 118115–122 | 8 |
| 50 | Qwen 3.6 Plus Alibaba | 88.4% | 113109–117 | 7 too little data |
| 51 | Inklingopen Thinking Machines | 88.3% | 114110–117 | 8 |
| 52 | Mistral Large 4 Mistral AI | 88.1% | 122119–125 | 7 too little data |
| 53 | Kimi K2.7 Codeopen Moonshot | 87.9% | 118115–122 | 8 |
| 54 | Qwen3.7-Plus Alibaba | 87.9% | 115110–119 | 6 |
| 55 | GLM-5open Z.ai (Zhipu AI) | 87.8% | 112105–118 | 5 too little data |
| 56 | GPT-5.1 OpenAI | 87.6% | 120115–125 | 8 |
| 57 | Kimi K2.5open Moonshot | 87.6% | 119112–125 | 6 |
| 58 | Claude Sonnet 4.6 Anthropic | 87.4% | 113108–118 | 8 |
| 59 | Qwen 3.6 Max (Preview) Alibaba | 87.4% | 118112–124 | 5 too little data |
| 60 | Grok 4 xAI | 87.0% | 114107–120 | 4 too little data |
| 61 | GPT-5.4 Mini OpenAI | 86.9% | 117113–120 | 9 |
| 62 | Qwen3.5 397B-A17Bopen Alibaba | 86.4% | 114109–118 | 6 |
| 63 | GPT-5 OpenAI | 86.2% | 121118–124 | 12 |
| 64 | Claude Opus 4.5 Anthropic | 86.0% | 115111–118 | 11 |
| 65 | Claude Fable 5 Anthropic | 85.9% | 138134–141 | 9 |
| 66 | Qwen3.6 27Bopen Alibaba | 85.9% | 115110–119 | 6 |
| 67 | Nemotron 3 Ultraopen Nvidia | 85.4% | 114108–120 | 4 too little data |
| 68 | Gemini 2.5 Pro (Jun 2025) Google DeepMind | 85.3% | 108104–112 | 8 |
| 69 | Qwen 3.5 Plus (hosted 397B-A17B) Alibaba | 84.8% | 109103–115 | 5 too little data |
| 70 | Qwen 3.6 35B-A3Bopen Alibaba | 84.8% | 111107–115 | 6 |
| 71 | Gemini 2.5 Pro (Mar 2025) Google DeepMind | 83.8% | 108102–113 | 3 too little data |
| 72 | Qwen3.5-35B-A3Bopen Alibaba | 83.5% | 106100–112 | 4 too little data |
| 73 | GLM-4.7open Z.ai (Zhipu AI) | 83.3% | 112107–118 | 6 |
| 74 | Gemini 3.5 Flash-Lite Google DeepMind | 83.3% | 109106–113 | 8 |
| 75 | Qwen 3.6 Flash Alibaba | 83.3% | 107103–111 | 6 too little data |
| 76 | GPT-5.5 Instant OpenAI | 82.5% | 107103–111 | 4 too little data |
| 77 | Claude Sonnet 4.5 Anthropic | 82.3% | 110107–113 | 12 |
| 78 | Qwen 3.5 Flash (hosted 35B-A3B) Alibaba | 82.3% | 107103–111 | 6 too little data |
| 79 | Qwen3.7 Flash Alibaba | 82.3% | 107103–112 | 4 too little data |
| 80 | Gemini 3.1 Flash-Lite Google | 81.8% | 109105–114 | 3 too little data |
| 81 | o3 OpenAI | 81.8% | 116113–119 | 10 |
| 82 | Qwen3-235B-A22B-Thinking (Jul 2025)open Alibaba | 80.1% | 111105–118 | 6 |
| 83 | Claude 3.7 Sonnet Anthropic | 79.7% | 9995–103 | 5 |
| 84 | o4-mini OpenAI | 79.6% | 112109–115 | 10 |
| 85 | Claude Sonnet 4 Anthropic | 79.2% | 9793–101 | 6 |
| 86 | Qwen3.5-9Bopen Alibaba | 79.0% | 10195–107 | 4 too little data |
| 87 | GPT-5.4 Nano OpenAI | 78.5% | 118115–121 | 9 |
| 88 | Claude Opus 4.1 Anthropic | 77.3% | 105102–109 | 6 too little data |
| 89 | o3-mini OpenAI | 77.0% | 105101–108 | 9 |
| 90 | o1 OpenAI | 76.8% | 10399–106 | 6 too little data |
| 91 | DeepSeek-R1 (May 2025)open DeepSeek | 76.3% | 104100–109 | 5 too little data |
| 92 | Claude Opus 4 Anthropic | 76.3% | 9893–102 | 5 |
| 93 | Grok-3 mini xAI | 76.3% | 10095–105 | 4 too little data |
| 94 | Gemma 4 31B ITopen Google DeepMind | 75.8% | 10498–110 | 5 too little data |
| 95 | Grok 3 xAI | 75.8% | 9691–100 | 4 too little data |
| 96 | gpt-oss-120bopen OpenAI | 75.8% | 10699–112 | 6 |
| 97 | GPT-5 mini OpenAI | 75.0% | 115112–118 | 12 |
| 98 | Gemma 4 26B A4Bopen Google DeepMind | 73.2% | 106100–112 | 4 too little data |
| 99 | Qwen3-Max Alibaba | 72.6% | 106103–110 | 6 too little data |
| 100 | DeepSeek-R1open DeepSeek | 71.7% | 9893–102 | 4 too little data |
| 101 | seed-oss-36b-instruct — | 71.5% | 9993–106 | 2 too little data |
| 102 | Claude Haiku 4.5 Anthropic | 71.2% | 10499–108 | 8 |
| 103 | Qwen3-235B-A22Bopen Alibaba | 70.7% | 8580–91 | 3 too little data |
| 104 | Qwen3-30B-A3B-Thinking (Jul 2025)open Alibaba | 70.1% | 10093–106 | 2 too little data |
| 105 | GPT-5 nano OpenAI | 69.4% | 105102–109 | 11 |
| 106 | GPT-4.5 OpenAI | 68.7% | 8783–92 | 3 too little data |
| 107 | DeepSeek-V3 (Mar 2025)open DeepSeek | 67.6% | 8581–90 | 5 too little data |
| 108 | Llama 4 Maverickopen Meta AI | 67.0% | 8177–85 | 7 |
| 109 | GPT-4.1 OpenAI | 66.9% | 9187–94 | 11 |
| 110 | Gemini 2.5 Pro (May 2025) Google DeepMind | 66.7% | 10498–110 | 3 too little data |
| 111 | GPT-4.1 mini OpenAI | 65.8% | 9188–95 | 11 |
| 112 | Qwen3-32Bopen Alibaba | 65.7% | 9589–100 | 6 |
| 113 | Gemini 2.0 Pro Google DeepMind | 65.7% | 8983–94 | 3 too little data |
| 114 | QWQ-Plus Alibaba | 65.4% | 9079–102 | 1 too little data |
| 115 | QwQ-32Bopen Alibaba | 65.3% | 9286–98 | 3 too little data |
| 116 | DeepSeek-R1-Distill-Qwen-32Bopen DeepSeek | 64.1% | 9387–100 | 2 too little data |
| 117 | Gemini 2.0 Flash (Feb 2025) Google DeepMind, Google | 64.1% | 8782–91 | 3 too little data |
| 118 | Qwen3-14Bopen Alibaba | 63.8% | 9387–99 | 5 too little data |
| 119 | o1-mini OpenAI | 62.4% | 9287–96 | 4 too little data |
| 120 | Qwen3-30B-A3Bopen Alibaba | 61.7% | 9488–100 | 4 too little data |
| 121 | gpt-oss-20bopen OpenAI | 60.8% | 9589–101 | 4 too little data |
| 122 | glm-4.7-flash — | 60.5% | 9488–100 | 4 too little data |
| 123 | Mistral Medium 3 Mistral AI | 59.5% | 8681–90 | 4 too little data |
| 124 | Gemini 1.5 Pro (Sept 2024) Google DeepMind | 57.2% | 8075–85 | 3 too little data |
| 125 | Gemini 2.0 Flash Thinking (Jan 2025) Google DeepMind, Google | 57.1% | 8983–95 | 3 too little data |
| 126 | Qwen3-8Bopen Alibaba | 56.8% | 9184–97 | 3 too little data |
| 127 | DeepSeek-V3open DeepSeek | 56.5% | 7773–82 | 5 too little data |
| 128 | Qwen2.5-Max Alibaba | 56.1% | 7873–82 | 4 too little data |
| 129 | Magistral Small 1.0open Mistral AI | 56.1% | 8376–89 | 2 too little data |
| 130 | Phi-4open Microsoft Research | 56.1% | 7671–81 | 4 too little data |
| 131 | DeepSeek-R1-Distill-Llama-70Bopen DeepSeek | 55.7% | 9288–97 | 3 too little data |
| 132 | Qwen3-30B-A3B-Instruct (Jul 2025)open Alibaba | 55.6% | 9186–97 | 5 too little data |
| 133 | Claude 3.5 Sonnet (October 2024) Anthropic | 55.3% | 7671–80 | 4 too little data |
| 134 | Claude 3.5 Sonnet Anthropic | 54.0% | 7166–76 | 3 too little data |
| 135 | Grok-2 (Dec 2024) xAI | 53.8% | 7570–80 | 3 too little data |
| 136 | Qwen3-4Bopen Alibaba | 52.3% | 7765–89 | 1 too little data |
| 137 | Llama 4 Scoutopen Meta AI | 51.8% | 7368–78 | 4 too little data |
| 138 | Mistral Large 2 (Nov 2024)open Mistral AI | 51.3% | 7166–76 | 4 too little data |
| 139 | Llama 3.1-405Bopen Meta AI | 50.9% | 7065–75 | 3 too little data |
| 140 | o1-preview OpenAI | 50.3% | 8480–89 | 3 too little data |
| 141 | GPT-4o (Aug 2024) OpenAI | 49.2% | 7368–77 | 9 |
| 142 | Qwen2.5-72Bopen Alibaba | 49.1% | 7268–77 | 5 too little data |
| 143 | Mistral Small 3.2open Mistral AI | 49.1% | 8074–87 | 3 too little data |
| 144 | Mistral Large 2 (Jul 2024)open Mistral AI | 49.0% | 6863–73 | 3 too little data |
| 145 | GPT-4.1 nano OpenAI | 48.9% | 7672–80 | 9 |
| 146 | GPT-4o (May 2024) OpenAI | 48.9% | 7066–75 | 5 too little data |
| 147 | Qwen Plus (Jan 2025) Alibaba | 48.1% | 7671–81 | 3 too little data |
| 148 | GPT-4o (Nov 2024) OpenAI | 47.9% | 6964–73 | 5 |
| 149 | Gemma 3 27Bopen Google DeepMind | 47.7% | 7772–81 | 5 |
| 150 | Magistral Small 1.2open Mistral AI | 47.6% | 7972–86 | 2 too little data |
| 151 | Mistral Small 3.1open Mistral AI | 47.5% | 6863–73 | 3 too little data |
| 152 | Llama 3.3 70Bopen Meta AI | 47.4% | 6661–71 | 6 |
| 153 | Gemini 1.5 Flash (Sep 2024) Google DeepMind | 47.3% | 7470–79 | 3 too little data |
| 154 | Mistral Small 3open Mistral AI | 47.3% | 6762–72 | 3 too little data |
| 155 | Claude 3 Opus Anthropic | 47.2% | 6560–70 | 5 too little data |
| 156 | GPT-4 Turbo (Apr 2024) OpenAI | 46.6% | 6864–73 | 4 too little data |
| 157 | Qwen2.5-32Bopen Alibaba | 46.1% | 7166–76 | 3 too little data |
| 158 | Gemini 1.5 Pro (May 2024) Google DeepMind | 45.9% | 6661–71 | 3 too little data |
| 159 | qwen3-4b-instruct-2507 — | 45.8% | 8579–92 | 3 too little data |
| 160 | glm-4.7-flash_none — | 45.1% | 7770–84 | 2 too little data |
| 161 | DeepSeek-R1-Distill-Qwen-14Bopen DeepSeek | 44.7% | 8984–93 | 3 too little data |
| 162 | Llama 3.1-70Bopen Meta AI | 44.2% | 6358–68 | 3 too little data |
| 163 | WizardLM-2 8x22Bopen Microsoft | 43.4% | 5953–64 | 2 too little data |
| 164 | Qwen-Turbo Alibaba | 41.8% | 6964–75 | 3 too little data |
| 165 | Qwen2-72Bopen Alibaba | 40.8% | 6458–69 | 2 too little data |
| 166 | Claude 3 Sonnet Anthropic | 40.6% | 5449–59 | 3 too little data |
| 167 | Llama 3-70Bopen Meta AI | 40.6% | 5752–62 | 3 too little data |
| 168 | Gemini 1.5 Flash (May 2024) Google DeepMind | 40.4% | 5853–63 | 3 too little data |
| 169 | Gemma 3 12Bopen Google DeepMind | 39.5% | 7366–79 | 5 too little data |
| 170 | Mistral Large Mistral AI | 38.8% | 5651–62 | 3 too little data |
| 171 | Claude 3.5 Haiku Anthropic | 38.1% | 6762–72 | 4 too little data |
| 172 | Qwen3-1.7Bopen Alibaba | 38.0% | 6557–73 | 3 too little data |
| 173 | GPT-4o mini OpenAI | 37.7% | 6864–73 | 10 |
| 174 | Hermes 2 Theta Llama-3 70Bopen Nous Research, Arcee AI | 37.5% | 5550–61 | 3 too little data |
| 175 | Gemma 2 27Bopen Google DeepMind | 36.5% | 5752–63 | 3 too little data |
| 176 | Claude 3 Haiku Anthropic | 36.3% | 5045–56 | 3 too little data |
| 177 | GPT-4 (Mar 2023) OpenAI | 35.7% | 5240–65 | 2 too little data |
| 178 | Qwen2.5-7Bopen Alibaba | 35.5% | 5141–60 | 4 too little data |
| 179 | Claude 2 Anthropic | 34.7% | 4842–53 | 3 too little data |
| 180 | Mixtral 8x22Bopen Mistral AI | 34.1% | 5549–61 | 2 too little data |
| 181 | Gemini 1.0 Pro Google DeepMind | 34.0% | 4640–52 | 3 too little data |
| 182 | Eurus-2-7B-PRIMEopen Tsinghua University, University of Illinois Urbana-Champaign (UIUC), Shanghai AI Lab, Peking University, Shanghai Jiao Tong University, CUHK Shenzhen Research Institute | 33.9% | 5135–67 | 1 too little data |
| 183 | DeepSeek-R1-Distill-Qwen-1.5Bopen DeepSeek | 33.6% | 7365–80 | 2 too little data |
| 184 | Claude 2.1 Anthropic | 33.0% | 5240–64 | 2 too little data |
| 185 | Gemini 1.5 Flash 8B Google DeepMind | 33.0% | 5747–67 | 2 too little data |
| 186 | DBRXopen Databricks | 32.9% | 4640–52 | 2 too little data |
| 187 | GPT-4 (Jun 2023) OpenAI | 30.7% | 5751–63 | 4 too little data |
| 188 | Mixtral 8x7Bopen Mistral AI | 30.6% | 4337–49 | 2 too little data |
| 189 | Mistral NeMoopen Mistral AI | 29.9% | 4741–53 | 3 too little data |
| 190 | granite-4.0-micro — | 28.3% | 5441–67 | 2 too little data |
| 191 | GPT-3.5 Turbo (Nov 2023) OpenAI | 28.0% | 4943–55 | 2 too little data |
| 192 | phi-3-medium 14Bopen Microsoft | 27.6% | 5145–57 | 2 too little data |
| 193 | Gemma 2 9Bopen Google DeepMind | 27.5% | 5347–59 | 3 too little data |
| 194 | GPT-3.5 Turbo (Jan 2024) OpenAI | 27.2% | 4742–53 | 5 too little data |
| 195 | Ministral 8Bopen Mistral AI | 27.1% | 4741–53 | 3 too little data |
| 196 | Llama 3.1-8Bopen Meta AI | 27.0% | 5549–60 | 4 too little data |
| 197 | Llama 3-8Bopen Meta AI | 26.1% | 4034–46 | 3 too little data |
| 198 | Ministral 3B Mistral AI | 25.3% | 4943–55 | 2 too little data |
| 199 | granite-4.0-1b — | 24.0% | 5537–74 | 2 too little data |
| 200 | Llama 3.2 1Bopen Meta AI | 23.9% | 4227–58 | 3 too little data |
| 201 | Gemma 3 4Bopen Google DeepMind | 23.2% | 6353–73 | 4 too little data |
| 202 | Gemma 3 1Bopen Google DeepMind | 19.9% | 4025–55 | 3 too little data |
| 203 | Mistral 7B v0.3open Mistral AI | 15.2% | 5434–75 | 2 too little data |
| 204 | granite-4.0-350m — | 11.2% | 5337–68 | 3 too little data |
| 205 | deepseek-r1-0528-qwen3-8b — | 9.3% | 9083–98 | 2 too little data |
Data sources: Aider polyglot leaderboard data (Apache-2.0); Arena leaderboard dataset on Hugging Face (CC BY 4.0); Berkeley Function Calling Leaderboard, Gorilla project (Apache-2.0); Epoch AI, ‘Capabilities & benchmarking’ (CC BY 4.0); LiteLLM, model_prices_and_context_window.json (MIT); Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter (licences and changes).