benchmarks.site

Public leaderboard · Multimodal & vision

MMMU / MMMU-Pro

College exam-style questions that come with pictures, such as charts, diagrams, maps and sheet music, across 30 subjects. It checks whether an AI can understand images and reason about them like a university student.

Leaderboard

Top 3 right now

  1. 1stChance Vision 1.586.9
  2. 2ndGPT-5.4 Thinking w/ tools82.1
  3. 3rdGPT-5.4 Thinking w/o tools81.2

What it measures

About 11,500 college-level questions with images across 30 subjects; MMMU-Pro (arXiv 2409.02813) adds ten-option answers and vision-only inputs. Scored by accuracy.

How to read the numbers

MMMU-Pro overall accuracy: share of questions answered correctly (%), on the harder version with ten answer options and questions embedded in images — higher is better.

Results

25 rows · higher is better (MMMU-Pro Overall)CSV
1Chance Vision 1.5proprietary2026-07-0186.9-
2GPT-5.4 Thinking w/ toolsproprietary2026-03-0582.1-
3GPT-5.4 Thinking w/o toolsproprietary2026-03-0581.2-
4Gemini 3.0 Proproprietary2025-11-1881.0-
5Gemini 3.1 Pro Thinking (High)proprietary2026-02-1980.5-
6Muse Spark Thinkingproprietary2026-04-0880.4-
7GPT-5.2 Thinking w/o Pythonproprietary2025-12-1180.4-
8GPT-5.2 Thinking w/o toolsproprietary2025-12-1179.5-
9GPT-5.1 Thinkingproprietary2025-11-1379.0-
10GPT-5 w/ thinkingproprietary2025-08-0778.484.2
11Claude Opus 4.6 w/ toolsproprietary2026-02-0577.3-
12Gemma 4 31Bopen source2026-04-0276.9-
13o3proprietary2025-04-1676.482.9
14GPT-5.1proprietary2025-11-1376.085.4
15Claude Sonnet 4.6 w/ toolsproprietary2026-02-1775.6-
16Claude Sonnet 4.6 w/o toolsproprietary2026-02-1774.5-
17Claude Opus 4.6 w/o toolsproprietary2026-02-0573.9-
18Claude Opus 4.5proprietary2025-11-2473.980.7
19Gemma 4 26B A4Bopen source2026-04-0273.8-
20dots.vlm1open source2025-08-0570.180.1
21Claude Sonnet 4.5proprietary2025-09-2968.977.8
22Qwen3-VL 235B-A22Bopen source2025-09-2268.178.7
23Gemini 2.5 Pro 05-06proprietary2025-05-0668.079.6
24Seed 1.5-VL Thinkingproprietary2025-05-1367.677.9
25Seed 1.6-Thinkingproprietary2025-06-2566.474.8

Note: Rows are models with an MMMU-Pro score, sorted by MMMU-Pro overall (the page itself lists entries unsorted by default); rank is computed from that order. Human-expert and random baselines excluded (human expert high: 85.4 on MMMU-Pro). Nearly all scores are reported by model authors ('source: author'), not independently verified. '-' means no MMMU validation score listed. as_of is the latest model date in the data file, which the page uses as its 'Last updated' value.

Source: MMMU team (mmmu-benchmark.github.io) · mmmu-benchmark.github.io · results as of 2026-11-10 · retrieved 2026-11-17

Markdown · More Multimodal & vision