Agentic benchmark

BrowseComp leaderboard

8 models have a published score. Kimi K3.1 Pro (Moonshot AI) leads with 92.1, and the spread to the lowest score, Nemotron 3 Ultra at 44.4, is 47.7 points. An open-weight model leads here: Kimi K3.1 Pro at 92.1 is 0.6 ahead of the best closed model, GPT-6 Astra. The highest-scoring model you can deploy on OpenLLM Buddy is Kimi K3, ranked 3 of 8 with 91.2. Sources report different settings for this test, such as test year, version, or reasoning effort. Each setting is listed in the source table, so compare close scores with care.

Best openllm buddy catalog

91.2

Kimi K3

1 models scored

Best open weights

92.1

Kimi K3.1 Pro

6 models scored

Best closed api

91.5

GPT-6 Astra

1 models scored

BrowseComp scores

Accuracy of the final short answer. Results depend on the browsing tools each lab provides.

  • OpenLLM Buddy catalog
  • Open weights
  • Closed API
  1. 1Kimi K3.1 Promax effort (vendor-reported)92.1
  2. 2GPT-6 AstraBrowseComp; max score at any effort91.5
  3. 3Kimi K3max effort91.2
  4. 4Step 5 PreviewHigh effort (vendor-reported)88.7
  5. 5Mistral Large 4ML4 OSS release, version not stated (SFT checkpoint 75.0)84.5
  6. 6DeepSeek V4 ProV4-Pro (Preview) Max83.4
  7. 7Reflection Beam 1.0with context management, BF1676.8
  8. 8Nemotron 3 UltraNVIDIA44.4

What it measures

1,266 questions whose answers are short and easy to verify but buried deep in the web, so the agent must plan searches, follow leads, and combine evidence.

How it is scored

Accuracy of the final short answer. Results depend on the browsing tools each lab provides.

Why it matters

It separates agents that give up after one search from agents that research.

Official benchmark page

More agentic tests

Sources

ModelTestScoreSettingSource
Kimi K3.1 ProBrowseComp92.1max effort (vendor-reported)DataCamp (relaying Moonshot's release table)
GPT-6 AstraBrowseComp91.5BrowseComp; max score at any effortOpenAI GPT-6 Astra launch post (Professional table)
Kimi K3BrowseComp91.2max effortKimi K3 model card (Hugging Face)
Step 5 PreviewBrowseComp88.7High effort (vendor-reported)cellcog.ai (relaying StepFun's release table)
Mistral Large 4BrowseComp84.5ML4 OSS release, version not stated (SFT checkpoint 75.0)Mistral AI blog: Mistral Large 4 open weights
DeepSeek V4 ProBrowseComp83.4V4-Pro (Preview) MaxHugging Face model card (DeepSeek-V4-Pro)
Reflection Beam 1.0BrowseComp76.8with context management, BF16Hugging Face model card (Beam-1.0-501B-A23B)
Nemotron 3 UltraBrowseComp44.4—Hugging Face model card (NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16)

Scores are published results from model cards, lab reports, and public leaderboards, each linked to its source. Settings such as thinking mode or test version differ between sources and are shown next to each score. OpenLLM Buddy catalog models are uncensored fine-tunes; their scores are those of the official base model. Data collected 2026-10-30. How the index works