Gives AI agents a description of a real security bug in open-source software plus the code, and asks them to write an input that actually triggers the bug. It shows how good AI has become at the hands-on work of finding and reproducing vulnerabilities, which helps defenders and could help attackers.
Leaderboard
Top 3 right now
1stCreation Model90.6%
2ndXekRung-1.5-27B-Preview (finetuned from Qwen3.8-27B)88.9%
3rdGemini 3.8 Flash Cyber86.3%
What it measures
Agents must reproduce real vulnerabilities in open-source projects by writing proof-of-concept inputs, scored by the share successfully triggered.
How to read the numbers
Share of 1,507 real vulnerabilities the agent successfully reproduced (%) — higher is better.
XekRung-1.5-27B-Preview (finetuned from Qwen3.8-27B)
XekRung Agent
88.9%
2026-09-13
Alibaba Security
3
Gemini 3.8 Flash Cyber
Google Antigravity
86.3%
2026-09-10
Google
4
GPT-5.5-Cyber
OpenAI Agent
85.6%
2026-06-22
OpenAI
5
GLM-5.3
Claude Code
84.5%
2026-08-14
Zhipu AI
6
Feyospace-v1.1 (finetuned from Qwen3.8-27B)
Claude Code
84.4%
2026-09-23
Vera Praxis Lab
7
DeepSeek-V4-Pro
DeepSeek Agent
83.3%
2026-08-13
DeepSeek
8
Claude Mythos Preview
Anthropic Agent
83.1%
2026-04-07
Anthropic
9
GPT-5.5
OpenAI Agent
81.8%
2026-04-23
OpenAI
10
Basin (basin1-0912-pre-dpo)
Codex CLI
80.0%
2026-09-23
Submersion AI
11
Grok 4.6
Grok Build
79.7%
2026-08-12
xAI
12
Grok 4.5
Grok Build
79.0%
2026-07-14
xAI
13
GPT-5.4
OpenAI Agent
79.0%
2026-04-23
OpenAI
14
GLM-5.2
Claude Code
77.2%
2026-07-21
Zhipu AI
15
DeepSeek-V4-Flash
DeepSeek Agent
76.7%
2026-07-31
DeepSeek
16
GLM-5.1
Claude Code
68.7%
2026-04-07
Zhipu AI
17
Claude Opus 4.6
Anthropic Agent
66.6%
2026-02-05
Anthropic
18
GPT-5.4
Codex CLI
66.3%
2026-04-07
Zhipu AI
19
Claude Sonnet 4.6
Anthropic Agent
65.2%
2026-02-17
Anthropic
20
Muse Spark 1.1 (helpful-only)
Meta Agent
59.0%
2026-07-09
Meta AI
21
DeepSeek-V4-Pro Preview
Claude Code
57.7%
2026-07-13
XDxAI
22
Claude Opus 4.5
Anthropic Agent
50.6%
2026-02-05
Anthropic
23
Muse Spark
Meta Agent
43.5%
2026-04-14
Meta AI
24
GLM-5
Claude Code
43.2%
2026-02-12
Zhipu AI
25
Kimi K2.5
Kimi Agent
41.3%
2026-02-02
Kimi
Note: Level 1 (vulnerability description + codebase), single-trial runs. Reflects the site's default 'Model-focused' view, which excludes agent-focused systems and runs using the 'dynamic' (cleaned vulnerable Docker image) or 'test-time memory' settings; in the 'All' view many agent systems score 92–99%. Scores are mostly submitter-reported (Source column). The site shows entries above 90% as unranked cards in random order. as_of = newest entry date in the data file.
Source: CyberGym team (UC Berkeley) · cybergym.io · results as of 2026-10-24 · retrieved 2026-11-17 · license: not stated (benchmark code Apache-2.0)