benchmarks.site

Public leaderboard · Math

PutnamBench

Tests whether AI systems can write machine-checkable formal proofs for problems from the Putnam, a famously hard undergraduate math competition.

Leaderboard

Top 3 right now

  1. 1st1672
  2. 2nd2672
  3. 3rd3672

What it measures

Putnam competition problems formalized in Lean and other proof assistants, scored by the number of problems for which a machine-checked proof is produced.

How to read the numbers

Number of problems solved (out of 672) in Lean with answers — higher is better.

Results

25 rows · higher is better (Solved (of 672))CSV
1Aleph Prover (Logical Intelligence) 6722026-08-26Avg $74, Max $1468 per problem
2Humanfia6722026-08-27Avg $44.5 total cost/problem
3NEAR AI (w/ DeepSeek V4)6722026-09-04Mean $0.17, median $0.04, max $11.18 per problem
4Forall (Astrio)6722026-09-24Mean $3.52 actor-side cost/problem; median $2.49
5Aleph Prover (Logical Intelligence) 6682026-01-11Avg $68, Limit $1400 per problem
6Mistral Prover (DeepSeek V4 Pro + Leanstral)6582026-09-04Avg 5.38M tokens per solved problem
7Goedel-Architect 6422026-07-02Avg $5.8 total cost/problem
8Aleph Prover (Logical Intelligence) 6372025-12-26Avg $54, Limit $400 per problem
9Goedel-Architect5972026-06-06pass@4, Avg. $1.47 total cost / problem
10Leanstral 1.55882026-07-194m tokens per problem
11Seed-Prover 1.5 (ByteDance)5812025-12-2710 H20 days per problem
12Aleph Prover (Logical Intelligence)5002025-12-23Avg $23, Limit $100 per problem
13Hilbert4622025-10-06avg pass@1840
14AxProverBase (Axiomatic AI)3652026-04-14Avg $12.60, Opus 4.5 w/ 50 iterations
15Seed-Prover (ByteDance)3292025-12-23MEDIUM
16C2C-Goedel1102026-08-01pass@256
17Ax-Prover (Axiomatic AI)912025-12-23pass@1, avg. 100 tool calls
18Goedel-Prover-V2862025-07-22pass@184
19Ensemble Prover (Graviterra)652026-09-04Not reported
20DeepSeek-Prover-V2472025-06-03pass@1024
21GPT-5 (ReAct, 10 turns)282025-11-20pass@1, 10 tool calls
22C2C-Kimina252026-08-01pass@256
23DSP+232025-06-28pass@128
24Bourbaki142025-07-16pass@512
25Kimina-Prover-7B-Distill102025-04-14pass@192

Note: Lean (with solution/answer placeholders pre-filled) track only. Ties broken by earliest date added; the site ranks several tied 672/672 entries cheapest first. as_of is last repo commit. Isabelle/Rocq tracks not shown.

Source: PutnamBench (Trishul lab, UT Austin) · trishullab.github.io · results as of 2026-10-06 · retrieved 2026-11-17

Markdown · More Math