Knowledge benchmark

MMMU-Pro leaderboard

9 models have a published score. Claude Opus 5.5 (Anthropic) leads with 87.7, and the spread to the lowest score, Gemma 4 26B A4B at 73.8, is 13.9 points. The best open-weight result, Kimi K3.1 Pro at 84.1, trails the best closed model, Claude Opus 5.5, by 3.6. The highest-scoring model you can deploy on OpenLLM Buddy is Kimi K3, ranked 6 of 9 with 81.6. Sources report different settings for this test, such as test year, version, or reasoning effort. Each setting is listed in the source table, so compare close scores with care.

Best openllm buddy catalog

81.6

Kimi K3

3 models scored

Best open weights

84.1

Kimi K3.1 Pro

2 models scored

Best closed api

87.7

Claude Opus 5.5

4 models scored

MMMU-Pro scores

Accuracy, usually the average of the standard and vision settings.

  • OpenLLM Buddy catalog
  • Open weights
  • Closed API
  1. 1Claude Opus 5.5MMMU-Pro (not original MMMU); AA independent run (Max, Default Fallback)87.7
  2. 2GPT-6 AstraMMMU-Pro (not original MMMU); AA independent run (Max)86.9
  3. 3GPT-6.1 SolMMMU-Pro (not original MMMU); AA independent run (Max)86.0
  4. 4Gemini 3.8 FlashMMMU-Pro (not original MMMU); AA independent run (High)85.6
  5. 5Kimi K3.1 ProMMMU-Pro (not original MMMU), no tools (vendor-reported)84.1
  6. 6Kimi K3MMMU-Pro, no tools81.6
  7. 7Gemma 4 31BMMMU-Pro76.9
  8. 8Step 5 PreviewMMMU-Pro (not original MMMU), High effort (vendor-reported)76.0
  9. 9Gemma 4 26B A4BMMMU-Pro73.8

What it measures

MMMU questions that text-only models could answer were removed, options were expanded to ten, and a vision setting embeds the question inside the image itself so the model must read it visually.

How it is scored

Accuracy, usually the average of the standard and vision settings.

Why it matters

It is a more robust check on visual reasoning now that plain MMMU scores have climbed.

Official benchmark page

More knowledge tests

Sources

ModelTestScoreSettingSource
Claude Opus 5.5MMMU-Pro87.7MMMU-Pro (not original MMMU); AA independent run (Max, Default Fallback)Artificial Analysis
GPT-6 AstraMMMU-Pro86.9MMMU-Pro (not original MMMU); AA independent run (Max)Artificial Analysis
GPT-6.1 SolMMMU-Pro86.0MMMU-Pro (not original MMMU); AA independent run (Max)Artificial Analysis
Gemini 3.8 FlashMMMU-Pro85.6MMMU-Pro (not original MMMU); AA independent run (High)Artificial Analysis
Kimi K3.1 ProMMMU-Pro84.1MMMU-Pro (not original MMMU), no tools (vendor-reported)DataCamp (relaying Moonshot's release table)
Kimi K3MMMU-Pro81.6MMMU-Pro, no toolsKimi K3 model card (Hugging Face)
Gemma 4 31BMMMU-Pro76.9MMMU-ProGemma 4 31B IT model card (Hugging Face)
Step 5 PreviewMMMU-Pro76.0MMMU-Pro (not original MMMU), High effort (vendor-reported)cellcog.ai (relaying StepFun's release table)
Gemma 4 26B A4BMMMU-Pro73.8MMMU-ProGemma 4 31B IT model card (Hugging Face), 26B A4B column

Scores are published results from model cards, lab reports, and public leaderboards, each linked to its source. Settings such as thinking mode or test version differ between sources and are shown next to each score. OpenLLM Buddy catalog models are uncensored fine-tunes; their scores are those of the official base model. Data collected 2026-10-30. How the index works