benchmarks.site

Public leaderboard · Coding & software engineering

SWE-Bench Pro

Tests whether AI agents can solve long, realistic software tasks in real open-source projects, harder than older bug-fixing tests.

Leaderboard

Top 3 right now

  1. 1stOpus 5 (Claude Code) xhigh98.00
  2. 2ndFable 5.1 (Claude Code) high92.20
  3. 3rdGemini 4 Argon (mini-swe-agent) high92.16

What it measures

Long-horizon software engineering tasks from open-source repositories in several languages, scored by the share of tasks resolved under hidden tests.

How to read the numbers

Share of tasks solved (%) — higher is better.

Results

18 rows · higher is better (Score (%))CSV
1Opus 5 (Claude Code) xhigh98.00
2Fable 5.1 (Claude Code) high92.20
3Gemini 4 Argon (mini-swe-agent) high92.16
4GPT-6 Astra (Codex) high90.20
4Muse Wonder 1.0 (mini-swe-agent) xhigh90.20
6Kimi-K3 (mini-swe-agent) max88.20
6Sonnet 5 (Claude Code) xhigh88.20
8GPT-5.6 Terra (Codex) xhigh86.30
8Inkling 2 Small86.27
9Minimax M3p1 Flash (mini-swe-agent) xhigh84.31
10GLM-5.3 (mini-swe-agent) max84.30
11GPT-5.6 Sol (Codex) xhigh82.40
12Muse Spark Open (mini-swe-agent) xhigh82.35
13Sonnet 5.5 (mini-swe-agent) xhigh70.59
14Gemini 3.8 Flash (mini-swe-agent) high58.80
15Inkling (mini-swe-agent) xhigh56.90
16Haiku 5.5 (mini-swe-agent) high52.94
17Haiku 4.5 (Claude Code) xhigh25.50

Note: SWE-Bench Pro V2 public split (642 tasks). Page has Full/HARD tabs; the table scraped is the default view and which tab it is was not identifiable from static HTML. Some entries gray-out/capped runs.

Source: Scale AI Labs · labs.scale.com · results as of 2026-09-22 · retrieved 2026-11-17

Markdown · More Coding & software engineering