# MultiChallenge

Tests whether an AI can hold a realistic multi-turn conversation: remembering what you said earlier, following your instructions across turns, and reasoning over the whole chat.

Category: instruction-following. Type: leaderboard. Status: active.
Metric: Share of multi-turn conversation test cases answered correctly (%), with a 95% confidence margin (higher is better).
Source: Scale AI (Scale Labs) — https://labs.scale.com/leaderboard/multichallenge, results as of 2026-11 (entries tagged NEW; judge update Feb 3, 2026), retrieved 2026-11-17, license: not stated.

| Rank | Model | Score (%) | Margin (±) |
|---|---|---|---|
| 1 | gpt 6.1 Sol | 81.95 | 4.16 |
| 2 | Gemini 4 Argon | 78.70 | 5.30 |
| 3 | Muse Spark | 75.52 | 4.05 |
| 4 | Muse Spark 1.1 | 75.30 | 0.60 |
| 5 | Deepseek V4p1 Flash 1017 | 74.90 | 5.30 |
| 6 | GPT 6.1 Terra | 74.20 | 5.40 |
| 6 | Muse Spark Open | 74.20 | 5.60 |
| 8 | gemini-3.1-pro-preview | 71.37 | 1.74 |
| 9 | gpt-5.4-pro-2026-03-05 | 69.23 | 3.05 |
| 10 | Claude Haiku 5.5 | 67.67 | 5.34 |
| 11 | gemini-3-pro-preview | 65.67 | 2.20 |
| 12 | gpt-5.1-2025-11-13-thinking | 63.41 | 2.11 |
| 13 | gpt-5-thinking | 63.19 | 1.63 |
| 14 | o3-pro-2025-06-10-reasoning-high | 62.40 | 2.49 |
| 15 | kimi-k2.5 | 61.39 | 0.46 |
| 16 | gemini-3.1-flash-lite-preview | 60.61 | 0.38 |
| 17 | gpt-5-mini-thinking | 58.99 | 4.07 |
| 18 | claude-opus-4-5-20251101-thinking | 58.97 | 0.42 |
| 19 | claude-4-opus-thinking | 58.62 | 0.77 |
| 20 | claude-opus-4-1-20250805-thinking | 57.20 | 0.95 |
| 21 | claude-4-sonnet-thinking | 57.11 | 0.81 |
| 22 | o3-2025-04-16-reasoning-high | 56.62 | 3.32 |
| 23 | claude-opus-4-6 (Non-Thinking) | 56.02 | 1.08 |
| 24 | kimi-k2-thinking | 55.42 | 1.88 |
| 25 | claude-sonnet-4-5-20250929-thinking | 55.32 | 1.47 |

Notes: Rank is rank-by-upper-bound (ties possible). Scored by an LLM judge (Gemini 2.5 Pro since Feb 2026). Model names as published; terms of reuse not stated on page.
