benchmarks.site

Public leaderboard · Coding & software engineering

SWE Atlas

Tests whether AI coding assistants can do everyday professional software work beyond fixing bugs: answering questions about a codebase, writing tests and refactoring code.

Leaderboard

Top 3 right now

  1. 1stFable 5.280.55
  2. 2ndOpus 5.577.45
  3. 3rdFable 5.176.45

What it measures

Coding agents on codebase comprehension, test writing and refactoring tasks.

How to read the numbers

Overall SWE Atlas v1.1 index score (%) with ± confidence margin — higher is better.

Results

25 rows · higher is better (Score (%))CSV
1Fable 5.280.554.80
2Opus 5.577.455.05
3Fable 5.176.455.16
4Gemini 4 Carbon71.455.46
5Gemini 4 Argon70.085.48
6GPT-6 Astra69.425.55
7Sonnet 5.568.285.64
8GPT-6.1 Sol65.885.74
9Muse Wonder 1.064.835.77
10Qwen 3.8 Max64.405.71
11Grok 4.762.135.79
12DeepSeek V4.1 Flash62.125.85
13Inkling 2 Small61.895.83
14Kimi K3.1 Pro61.115.88
15Kimi K360.985.89
16Haiku 5.559.435.92
17GLM 5.358.105.90
18GPT-6.1 Terra57.695.94
19Gemini 3.8 Flash57.335.83
20Muse Ember 1.055.995.98
21GLM 5.3 Flash55.335.94
22GPT-6 Luna50.806.02
23Mistral Large 445.195.83
24Muse Spark Open42.285.90
25MiniMax M339.265.75

Note: SWE Atlas v1.1 (Nov 2026), 284 tasks; not comparable to v1.0. Judge: GPT-6.1 Sol majority of 3.

Source: Scale AI Labs · labs.scale.com · results as of 2026-11 · retrieved 2026-11-17

Markdown · More Coding & software engineering