benchmarks.site

Public leaderboard · Instruction following

MultiChallenge

Tests whether an AI can hold a realistic multi-turn conversation: remembering what you said earlier, following your instructions across turns, and reasoning over the whole chat.

Leaderboard

Top 3 right now

  1. 1stgpt 6.1 Sol81.95
  2. 2ndGemini 4 Argon78.70
  3. 3rdMuse Spark75.52

What it measures

Multi-turn conversations that test retaining instructions, using earlier context, handling edits and staying self-consistent, graded by an LLM judge with rubrics.

How to read the numbers

Share of multi-turn conversation test cases answered correctly (%), with a 95% confidence margin — higher is better.

Results

25 rows · higher is better (Score (%))CSV
1gpt 6.1 Sol81.954.16
2Gemini 4 Argon78.705.30
3Muse Spark75.524.05
4Muse Spark 1.175.300.60
5Deepseek V4p1 Flash 101774.905.30
6GPT 6.1 Terra74.205.40
6Muse Spark Open74.205.60
8gemini-3.1-pro-preview71.371.74
9gpt-5.4-pro-2026-03-0569.233.05
10Claude Haiku 5.567.675.34
11gemini-3-pro-preview65.672.20
12gpt-5.1-2025-11-13-thinking63.412.11
13gpt-5-thinking63.191.63
14o3-pro-2025-06-10-reasoning-high62.402.49
15kimi-k2.561.390.46
16gemini-3.1-flash-lite-preview60.610.38
17gpt-5-mini-thinking58.994.07
18claude-opus-4-5-20251101-thinking58.970.42
19claude-4-opus-thinking58.620.77
20claude-opus-4-1-20250805-thinking57.200.95
21claude-4-sonnet-thinking57.110.81
22o3-2025-04-16-reasoning-high56.623.32
23claude-opus-4-6 (Non-Thinking)56.021.08
24kimi-k2-thinking55.421.88
25claude-sonnet-4-5-20250929-thinking55.321.47

Note: Rank is rank-by-upper-bound (ties possible). Scored by an LLM judge (Gemini 2.5 Pro since Feb 2026). Model names as published; terms of reuse not stated on page.

Source: Scale AI (Scale Labs) · labs.scale.com · results as of 2026-11 (entries tagged NEW; judge update Feb 3, 2026) · retrieved 2026-11-17

Markdown · More Instruction following