benchmarks.site

Public leaderboard · Safety & security

CyberGym

Gives AI agents a description of a real security bug in open-source software plus the code, and asks them to write an input that actually triggers the bug. It shows how good AI has become at the hands-on work of finding and reproducing vulnerabilities, which helps defenders and could help attackers.

Leaderboard

Top 3 right now

  1. 1stCreation Model90.6%
  2. 2ndXekRung-1.5-27B-Preview (finetuned from Qwen3.8-27B)88.9%
  3. 3rdGemini 3.8 Flash Cyber86.3%

What it measures

Agents must reproduce real vulnerabilities in open-source projects by writing proof-of-concept inputs, scored by the share successfully triggered.

How to read the numbers

Share of 1,507 real vulnerabilities the agent successfully reproduced (%) — higher is better.

Results

25 rows · higher is better (Success Rate)CSV
1Creation ModelClaude Code90.6%2026-10-21Creation Lab (Creation Lab, Ant Group)
2XekRung-1.5-27B-Preview (finetuned from Qwen3.8-27B)XekRung Agent88.9%2026-09-13Alibaba Security
3Gemini 3.8 Flash CyberGoogle Antigravity86.3%2026-09-10Google
4GPT-5.5-CyberOpenAI Agent85.6%2026-06-22OpenAI
5GLM-5.3Claude Code84.5%2026-08-14Zhipu AI
6Feyospace-v1.1 (finetuned from Qwen3.8-27B)Claude Code84.4%2026-09-23Vera Praxis Lab
7DeepSeek-V4-ProDeepSeek Agent83.3%2026-08-13DeepSeek
8Claude Mythos PreviewAnthropic Agent83.1%2026-04-07Anthropic
9GPT-5.5OpenAI Agent81.8%2026-04-23OpenAI
10Basin (basin1-0912-pre-dpo)Codex CLI80.0%2026-09-23Submersion AI
11Grok 4.6Grok Build79.7%2026-08-12xAI
12Grok 4.5Grok Build79.0%2026-07-14xAI
13GPT-5.4OpenAI Agent79.0%2026-04-23OpenAI
14GLM-5.2Claude Code77.2%2026-07-21Zhipu AI
15DeepSeek-V4-FlashDeepSeek Agent76.7%2026-07-31DeepSeek
16GLM-5.1Claude Code68.7%2026-04-07Zhipu AI
17Claude Opus 4.6Anthropic Agent66.6%2026-02-05Anthropic
18GPT-5.4Codex CLI66.3%2026-04-07Zhipu AI
19Claude Sonnet 4.6Anthropic Agent65.2%2026-02-17Anthropic
20Muse Spark 1.1 (helpful-only)Meta Agent59.0%2026-07-09Meta AI
21DeepSeek-V4-Pro PreviewClaude Code57.7%2026-07-13XDxAI
22Claude Opus 4.5Anthropic Agent50.6%2026-02-05Anthropic
23Muse SparkMeta Agent43.5%2026-04-14Meta AI
24GLM-5Claude Code43.2%2026-02-12Zhipu AI
25Kimi K2.5Kimi Agent41.3%2026-02-02Kimi

Note: Level 1 (vulnerability description + codebase), single-trial runs. Reflects the site's default 'Model-focused' view, which excludes agent-focused systems and runs using the 'dynamic' (cleaned vulnerable Docker image) or 'test-time memory' settings; in the 'All' view many agent systems score 92–99%. Scores are mostly submitter-reported (Source column). The site shows entries above 90% as unranked cards in random order. as_of = newest entry date in the data file.

Source: CyberGym team (UC Berkeley) · cybergym.io · results as of 2026-10-24 · retrieved 2026-11-17 · license: not stated (benchmark code Apache-2.0)

Markdown · More Safety & security