Agentic benchmark

Humanity's Last Exam (with tools) leaderboard

17 models have a published score. Kimi K3.1 Pro (Moonshot AI) leads with 70.0, and the spread to the lowest score, Gemma 4 26B A4B at 17.2, is 52.8 points. An open-weight model leads here: Kimi K3.1 Pro at 70.0 is 2.3 ahead of the best closed model, Claude Opus 5.5. The highest-scoring model you can deploy on OpenLLM Buddy is DeepSeek V4.1 Flash, ranked 5 of 17 with 63.9. Sources report different settings for this test, such as test year, version, or reasoning effort. Each setting is listed in the source table, so compare close scores with care.

Best openllm buddy catalog

63.9

DeepSeek V4.1 Flash

5 models scored

Best open weights

70.0

Kimi K3.1 Pro

7 models scored

Best closed api

67.7

Claude Opus 5.5

5 models scored

Humanity's Last Exam (with tools) scores

Exact-match accuracy, as with the no-tools version. Scores are not comparable with no-tools runs.

  • OpenLLM Buddy catalog
  • Open weights
  • Closed API
  1. 1Kimi K3.1 ProHLE-Full (2,500 q), with tools, max effort (vendor-reported)70.0
  2. 2Claude Opus 5.5Humanity's Last Exam, with tools (web search/fetch, code execution); max effort67.7
  3. 3Claude Fable 5.1Humanity's Last Exam, with tools; max effort (Opus 5.5 system card lists Fable 5.1 at 65.6%)65.0
  4. 4Claude Sonnet 5.5Humanity's Last Exam, with tools; max effort64.5
  5. 5DeepSeek V4.1 FlashPass@1, with tools, max effort63.9
  6. 6GLM 5.3with tools62.5
  7. 7Muse Spark Openwith tools, xhigh effort61.0
  8. 8DeepSeek V4 ProV4-Pro-0813, with tools60.0
  9. 9Claude Haiku 5.5Humanity's Last Exam, with tools; max effort57.4
  10. 10GPT-6 AstraHumanity's Last Exam, with tools; max score at any effort57.2
  11. 11Kimi K3HLE-Full, with tools, max effort56.0
  12. 12GLM 5.3 Flashwith tools (page attributes to HF official evaluation section)55.3
  13. 13Reflection Beam 1.0with tools, BF1651.1
  14. 14Nemotron 3 Ultrawith tools37.4
  15. 15Gemma 4 31Bwith search26.5
  16. 16gpt-oss-120bhigh effort, with tools19.0
  17. 17Gemma 4 26B A4Bwith search17.2

What it measures

Labs rerun Humanity's Last Exam with the model allowed to search the web, run code, or call other tools. The toolset differs by lab and is noted next to each score where the source names it.

How it is scored

Exact-match accuracy, as with the no-tools version. Scores are not comparable with no-tools runs.

Why it matters

The gap between the tool and no-tool runs shows how much a model gains from acting as an agent.

Official benchmark page

More agentic tests

Sources

ModelTestScoreSettingSource
Kimi K3.1 ProHumanity's Last Exam (with tools)70.0HLE-Full (2,500 q), with tools, max effort (vendor-reported)DataCamp (relaying Moonshot's release table)
Claude Opus 5.5Humanity's Last Exam (with tools)67.7Humanity's Last Exam, with tools (web search/fetch, code execution); max effortAnthropic Claude Opus 5.5 System Card (Table 8.1.A); also in launch post
Claude Fable 5.1Humanity's Last Exam (with tools)65.0Humanity's Last Exam, with tools; max effort (Opus 5.5 system card lists Fable 5.1 at 65.6%)Anthropic Claude Fable 5.1 / Mythos 5.1 System Card (Table 8.1.A)
Claude Sonnet 5.5Humanity's Last Exam (with tools)64.5Humanity's Last Exam, with tools; max effortAnthropic Claude Sonnet 5.5 System Card (Table 8.1.A); also in launch post
DeepSeek V4.1 FlashHumanity's Last Exam (with tools)63.9Pass@1, with tools, max effortDeepSeek-V4.1-Flash model card (Hugging Face)
GLM 5.3Humanity's Last Exam (with tools)62.5with toolsHugging Face model card (GLM-5.3)
Muse Spark OpenHumanity's Last Exam (with tools)61.0with tools, xhigh effortHugging Face model card (Muse-Spark-Open-816B-A42B)
DeepSeek V4 ProHumanity's Last Exam (with tools)60.0V4-Pro-0813, with toolsHugging Face model card (DeepSeek-V4-Pro-0813)
Claude Haiku 5.5Humanity's Last Exam (with tools)57.4Humanity's Last Exam, with tools; max effortAnthropic Claude Haiku 5.5 System Card (Table 8.1.A); also in launch post
GPT-6 AstraHumanity's Last Exam (with tools)57.2Humanity's Last Exam, with tools; max score at any effortOpenAI GPT-6 Astra launch post (Academic table)
Kimi K3Humanity's Last Exam (with tools)56.0HLE-Full, with tools, max effortKimi K3 model card (Hugging Face)
GLM 5.3 FlashHumanity's Last Exam (with tools)55.3with tools (page attributes to HF official evaluation section)CometAPI GLM-5.3-Flash spec page (secondary)
Reflection Beam 1.0Humanity's Last Exam (with tools)51.1with tools, BF16Hugging Face model card (Beam-1.0-501B-A23B)
Nemotron 3 UltraHumanity's Last Exam (with tools)37.4with toolsHugging Face model card (NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16)
Gemma 4 31BHumanity's Last Exam (with tools)26.5with searchGemma 4 31B IT model card (Hugging Face)
gpt-oss-120bHumanity's Last Exam (with tools)19.0high effort, with toolsgpt-oss model card paper (arXiv 2508.10925), Table 3
Gemma 4 26B A4BHumanity's Last Exam (with tools)17.2with searchGemma 4 31B IT model card (Hugging Face), 26B A4B column

Scores are published results from model cards, lab reports, and public leaderboards, each linked to its source. Settings such as thinking mode or test version differ between sources and are shown next to each score. OpenLLM Buddy catalog models are uncensored fine-tunes; their scores are those of the official base model. Data collected 2026-10-30. How the index works