Find by task
Safety & security
Jailbreaks, cyber skills, malware detection. Below: our own tests first, then public leaderboards with their current leaders.
Our tests
Measured by us, with caveats stated.
- Malware detection: AMTSO, EICAR containers, Objective-See hashes, Mac malware familiesDoes a Mac malware scanner catch the standard harmless test files and known Mac malware families?7 tables
- Malware detection: test samples vs held-out clean filesA fair test: how much malware does the scanner catch, and how often does it wrongly flag normal files it was never tuned on?5 tables
- Malware scanning: static-heuristic false alarms before and after fixesHow often the scanner wrongly flagged normal Mac files, before and after fixes.9 tables
- ML classifiers on TLSH fingerprints of Mach-O filesCan a machine-learning model spot malicious Mac programs from a file 'fingerprint' alone?5 tables
- Scanner performance: database loading, re-scans, TLSH in SQL, Aho-CorasickSpeed measurements of a malware scanner: loading its databases, re-scanning, and two search libraries.2 tables
Public leaderboards
3 from labs and universities.
-
F5 Labs CASI & ARS Leaderboard
Tests how hard it is to trick AI models into harmful behavior using attack prompts. A higher score means the model is harder to break.
Leader: Claude Sonnet 5 98.00
-
Scale Fortress
Tests whether AI models can be pushed into giving dangerous help (e.g. weapons or crime) by adversarial prompts. Lower is safer.
Leader: claude-opus-5-5-max 1
-
CyberGym
Gives AI agents a description of a real security bug in open-source software plus the code, and asks them to write an input that actually triggers the bug. It shows how good AI has become at the hands-on work of finding and reproducing vulnerabilities, which helps defenders and could help attackers.
Leader: Creation Model 90.6%