benchmarks.site

Public leaderboard · General knowledge & reasoning

CritPt

Unpublished research-level physics problems written by working physicists, in areas like quantum physics and astrophysics. It tests whether AI can do the kind of reasoning needed in real scientific research.

Dataset

Top 3 right now

  1. 1stGemini 4 Carbon (High)36.0
  2. 2ndClaude Fable 5.2 (Xhigh, Default Fallback)35.7
  3. 3rdClaude Fable 5.2 (Max, Default Fallback)35.4

What it measures

Unpublished, research-level physics problems across fields such as condensed matter, quantum and astrophysics, scored by automatically checked final answers.

How to read the numbers

Share of problems with the correct final answer (%) — higher is better.

Results

25 rows · higher is better (Accuracy (%))CSV
1Gemini 4 Carbon (High)Google DeepMind36.02026-11-10
2Claude Fable 5.2 (Xhigh, Default Fallback)Anthropic35.72026-11-09
3Claude Fable 5.2 (Max, Default Fallback)Anthropic35.42026-11-09
4Claude Fable 5.2 (High, Default Fallback)Anthropic32.92026-11-09
5GPT-5.6 Sol (Max)OpenAI32.32026-07-09
6GPT-6 Astra (Max)OpenAI31.72026-09-03
7Claude Opus 5.5 (Max, Default Fallback)Anthropic31.72026-09-22
8Claude Opus 5.5 (Xhigh, Default Fallback)Anthropic31.72026-09-22
9GPT-6.1 Sol (Max)OpenAI31.72026-09-29
10GPT-6.1 Sol (Xhigh)OpenAI31.72026-09-29
11GPT-6 Astra (Xhigh)OpenAI31.42026-09-03
12Claude Sonnet 5.5 (Max, Default Fallback)Anthropic31.42026-09-28
13Claude Fable 5.1 (Xhigh, Default Fallback)Anthropic31.12026-09-01
14Claude Sonnet 5.5 (Xhigh, Default Fallback)Anthropic31.12026-09-28
15GPT-6 Sol (Max)OpenAI30.92026-09-22
16Claude Opus 5.5 (High, Default Fallback)Anthropic30.92026-09-22
17Claude Fable 5.2 (Medium, Default Fallback)Anthropic30.92026-11-09
18GPT-5.5 Pro (Xhigh)OpenAI30.62026-04-23
19Claude Fable 5.1 (High, Default Fallback)Anthropic30.32026-09-01
20GPT-5.4 Pro (Xhigh)OpenAI30.02026-03-05
21GPT-5.6 Terra (Max)OpenAI30.02026-07-09
22GPT-6.1 Sol (High)OpenAI30.02026-09-29
23Claude Fable 5.1 (Max, Default Fallback)Anthropic29.72026-09-01
24Claude Fable 5.2 (Low, Default Fallback)Anthropic29.42026-11-09
25Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic29.12026-07-24

Note: External results mirrored by Epoch AI (critpt_external.csv), converted to % (1 decimal). One row per model + reasoning-effort configuration.

Source: Epoch AI (Benchmarking Hub) · epoch.ai · results as of 2026-11-17 · retrieved 2026-11-17 · license: CC-BY-4.0

Markdown · More General knowledge & reasoning