Claude Opus 5 | 21 | SimpleBench — #12 · 80.6 CritPt — #25 · 29.1 FrontierMath — #15 · 85.6 OSWorld 2.0 — #1 · 44.33 OSWorld 2.0 — #3 · 36.89 OSWorld 2.0 — #4 · 33.33 OSWorld 2.0 — #5 · 33.01 OSWorld 2.0 — #6 · 31.43 OSWorld 2.0 — #7 · 30.21 OSWorld 2.0 — #8 · 28.98 OSWorld 2.0 — #11 · 25.33 OSWorld 2.0 — #12 · 22.32 OSWorld 2.0 — #14 · 18.81 Vending-Bench 2 — #8 · $11,181.87 ± $2,094 GDPval-AA — #15 · 1729 GDPval-AA — #20 · 1695 Arena (formerly LMArena / Chatbot Arena) — #16 · 1490 Arena (formerly LMArena / Chatbot Arena) — #19 · 1489 F5 Labs CASI & ARS Leaderboard — #5 · 93.05 Epoch Capabilities Index (ECI) — #10 · 164.68 LLM Stats — #11 · 52.5 |
Claude Fable 5.2 | 20 | ARC-AGI (ARC Prize) — #23 · 81.2 ARC-AGI (ARC Prize) — #24 · 79.3 ARC-AGI (ARC Prize) — #25 · 73.3 SimpleBench — #1 · 94.9 CritPt — #2 · 35.7 CritPt — #3 · 35.4 CritPt — #4 · 32.9 CritPt — #17 · 30.9 CritPt — #24 · 29.4 FrontierMath — #1 · 95.1 GDPval-AA — #6 · 1831 GDPval-AA — #10 · 1751 GDPval-AA — #16 · 1718 AA-Omniscience — #1 · 67 AA-Omniscience — #2 · 65 AA-Omniscience — #3 · 64 Design Arena — #3 · 1367 Epoch Capabilities Index (ECI) — #1 · 173.44 Vals AI — #1 · 71.31 LLM Stats — #1 · 60.8 |
Gemini 4 Carbon | 14 | SimpleBench — #2 · 92.6 CritPt — #1 · 36.0 FrontierMath — #6 · 90.5 MathArena — #1 · 94.28% ± 3.64% Terminal-Bench — #3 · 64.5% ± 3.2% SWE Atlas — #4 · 71.45 Vending-Bench 2 — #1 · $16,484.21 ± $1,096 GDPval-AA — #14 · 1733 APEX-Agents — #1 · 84.2% ±4.3% Arena (formerly LMArena / Chatbot Arena) — #1 · 1526 Design Arena — #8 · 1306 Epoch Capabilities Index (ECI) — #2 · 169.76 Vals AI — #3 · 70.45 LLM Stats — #4 · 57.4 |
Claude Opus 5.5 | 13 | SimpleBench — #4 · 88.4 CritPt — #7 · 31.7 CritPt — #8 · 31.7 CritPt — #16 · 30.9 FrontierMath — #4 · 91.2 GDPval-AA — #1 · 1866 GDPval-AA — #5 · 1836 GDPval-AA — #19 · 1709 Arena (formerly LMArena / Chatbot Arena) — #4 · 1507 Design Arena — #1 · 1445 Epoch Capabilities Index (ECI) — #4 · 168.54 Vals AI — #4 · 70.14 LLM Stats — #3 · 58.5 |
GPT-6.1 Sol | 13 | SimpleBench — #8 · 82.9 CritPt — #9 · 31.7 CritPt — #10 · 31.7 CritPt — #22 · 30.0 FrontierMath — #2 · 93.7 MathArena — #2 · 92.81% ± 3.10% Terminal-Bench — #5 · 60.9% ± 3.0% SWE Atlas — #8 · 65.88 Vending-Bench 2 — #6 · $12,860.61 ± $1,043 Design Arena — #4 · 1366 Epoch Capabilities Index (ECI) — #5 · 168.48 Vals AI — #10 · 63.94 LLM Stats — #12 · 52.2 |
Claude Sonnet 5.5 | 12 | SimpleBench — #19 · 75.9 CritPt — #12 · 31.4 CritPt — #14 · 31.1 FrontierMath — #10 · 88.8 Vending-Bench 2 — #9 · $11,165.28 ± $1,585 GDPval-AA — #4 · 1838 GDPval-AA — #12 · 1743 Design Arena — #10 · 1290 F5 Labs CASI & ARS Leaderboard — #9 · 91.78 Epoch Capabilities Index (ECI) — #8 · 166.56 Vals AI — #2 · 70.85 LLM Stats — #5 · 56.7 |
Gemini 4 Argon | 12 | SimpleBench — #3 · 91.7 FrontierMath — #13 · 86.0 MathArena — #3 · 84.39% ± 4.65% Terminal-Bench — #10 · 55.5% ± 2.8% SWE Atlas — #5 · 70.08 Vending-Bench 2 — #5 · $13,718.16 ± $3,100 APEX-Agents — #2 · 82.4% ±4.3% MultiChallenge — #2 · 78.70 Arena (formerly LMArena / Chatbot Arena) — #2 · 1522 Epoch Capabilities Index (ECI) — #9 · 166.48 Vals AI — #5 · 69.39 LLM Stats — #9 · 53.3 |
GPT-6 Astra | 12 | SimpleBench — #6 · 86.5 SimpleBench — #7 · 83.6 CritPt — #6 · 31.7 CritPt — #11 · 31.4 FrontierMath — #3 · 93.7 Terminal-Bench — #6 · 60.3% ± 2.8% SWE Atlas — #6 · 69.42 Vending-Bench 2 — #2 · $15,514.70 ± $1,074 Design Arena — #2 · 1399 Epoch Capabilities Index (ECI) — #3 · 169.0 Vals AI — #8 · 65.02 LLM Stats — #6 · 56.3 |
Claude Fable 5.1 | 11 | SimpleBench — #5 · 86.6 CritPt — #13 · 31.1 CritPt — #19 · 30.3 CritPt — #23 · 29.7 FrontierMath — #7 · 90.2 GDPval-AA — #9 · 1765 GDPval-AA — #13 · 1741 Arena (formerly LMArena / Chatbot Arena) — #7 · 1502 Design Arena — #6 · 1336 Epoch Capabilities Index (ECI) — #7 · 166.99 LLM Stats — #13 · 52.0 |
Kimi K3.1 Pro | 10 | FrontierMath — #23 · 78.2 MathArena — #5 · 58.27% ± 6.84% Terminal-Bench — #7 · 59.7% ± 4.2% SWE Atlas — #14 · 61.11 GDPval-AA — #2 · 1861 Arena (formerly LMArena / Chatbot Arena) — #24 · 1485 Design Arena — #9 · 1297 Epoch Capabilities Index (ECI) — #16 · 161.37 Vals AI — #12 · 62.52 LLM Stats — #7 · 54.4 |
deepseek-ai/DeepSeek-V4.1-Flash | 9 | OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours OCR: handwritten notes, boards and textbook pages — "Hard" test from 2026-09-29/30: 14 photos, handwriting a ours OCR: handwritten notes, boards and textbook pages — Three hard photos of Polish notes, 2026-10-04 ours OCR: handwritten notes, boards and textbook pages — Textbook pages (print, scans and photos), 2026-10-05 ours Language models: classification, summaries, chat with tools — Lesson subject classification, 2026-09-22 ours Language models: classification, summaries, chat with tools — Lesson summary (description), 2026-09-22 ours Language models: classification, summaries, chat with tools — "Lossless" summaries: how much lesson knowledge survived ours Language models: classification, summaries, chat with tools — DeepSeek-V4-Pro vs V4.1-Flash on lesson description, 202 ours Vectara Hallucination Leaderboard — #6 · 4.2 % |
gpt-5.5 | 8 | EQ-Bench — #6 · 1295.4 FrontierMath — #16 · 85.3 OSWorld 2.0 — #19 · 13.0 Toolathlon (Tool Decathlon) — #6 · 73.5 ± 1.2 CyberGym — #9 · 81.8% Polish Linguistic and Cultural Competency (PLCC) — #12 · 95.83 Polish Linguistic and Cultural Competency (PLCC) — #17 · 94.5 Epoch Capabilities Index (ECI) — #20 · 160.1 |
Claude Haiku 5.5 | 7 | Humanity's Last Exam (HLE) — #13 · 30.1 FrontierMath — #24 · 75.1 MultiChallenge — #10 · 67.67 F5 Labs CASI & ARS Leaderboard — #2 · 94.61 Scale Fortress — #4 · 4 Epoch Capabilities Index (ECI) — #18 · 160.43 Vals AI — #19 · 58.73 |
Gemini 3.8 Flash | 7 | Humanity's Last Exam (HLE) — #5 · 44.5 SimpleBench — #9 · 82.4 SWE Atlas — #19 · 57.33 Remote Labor Index (RLI) — #10 · 5.83 Arena (formerly LMArena / Chatbot Arena) — #9 · 1497 Epoch Capabilities Index (ECI) — #23 · 158.79 Vals AI — #24 · 55.95 |
google/gemma-4-31B-it | 7 | Omni models: video and audio input — Does the model hear the audio from a video file? ours Omni models: video and audio input — How many frames does the model see? ours OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours Language models: classification, summaries, chat with tools — Lesson summary (description), 2026-09-22 ours Language models: classification, summaries, chat with tools — Whiteboard photo in chat (1 photo, 1024×1024) ours Platforms: errors, reasoning billing and prices — Prices from /v1/openai/models (2026-10-16) ours EuroEval — #19 · 1.67 ± 0.10 |
GPT-5.6 Sol | 7 | SimpleBench — #25 · 71.7 CritPt — #5 · 32.3 FrontierMath — #9 · 89.1 Terminal-Bench — #19 · 38.5% ± 3.8% OSWorld 2.0 — #9 · 27.34 Arena (formerly LMArena / Chatbot Arena) — #23 · 1485 Epoch Capabilities Index (ECI) — #13 · 162.99 |
Qwen/Qwen3-VL-235B-A22B-Instruct | 7 | OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours OCR: handwritten notes, boards and textbook pages — "Hard" test from 2026-09-29/30: 14 photos, handwriting a ours OCR: handwritten notes, boards and textbook pages — Describing illustrations (charts, diagrams, maps), 2026- ours OCR: exam-sheet photos with handwritten answers — Blank exam-sheet page ours OCR: exam-sheet photos with handwritten answers — Cost and time of a single call ours Reasoning level (reasoning_effort): what models actually do — Reasoning and the output token limit ours Platforms: errors, reasoning billing and prices — Prices from /v1/openai/models (2026-10-16) ours |
Claude Fable 5 | 6 | SimpleBench — #10 · 81.9 FrontierMath — #12 · 87.0 Arena (formerly LMArena / Chatbot Arena) — #5 · 1504 Design Arena — #14 · 1272 Epoch Capabilities Index (ECI) — #11 · 164.13 LLM Stats — #15 · 51.9 |
deepseek-ai/DeepSeek-V4-Flash | 6 | Language models: classification, summaries, chat with tools — Lesson subject classification, 2026-09-22 ours Language models: classification, summaries, chat with tools — Lesson summary (description), 2026-09-22 ours Language models: classification, summaries, chat with tools — "Lossless" summaries: how much lesson knowledge survived ours Language models: short answers, grading text, time and cost — DeepSeek-V4-Flash: undated name vs dated name (2026-10-1 ours Reasoning level (reasoning_effort): what models actually do — The minimal value on different models (2026-09-03) ours Reasoning level (reasoning_effort): what models actually do — Cost of reasoning ours |
Gemini 4 Hydrogen | 6 | Terminal-Bench — #9 · 56.4% ± 3.0% Vending-Bench 2 — #3 · $14,549.49 ± $495 APEX-Agents — #4 · 78.0% ±4.8% Arena (formerly LMArena / Chatbot Arena) — #15 · 1490 Design Arena — #15 · 1268 Vals AI — #6 · 68.49 |
GPT-6 Astra - Provider Adapter | 6 | ARC-AGI (ARC Prize) — #1 · 99.9 ARC-AGI (ARC Prize) — #3 · 98.6 ARC-AGI (ARC Prize) — #4 · 98.4 ARC-AGI (ARC Prize) — #5 · 98.4 ARC-AGI (ARC Prize) — #6 · 98.0 ARC-AGI (ARC Prize) — #15 · 96.7 |
GPT-6 Sol | 6 | SimpleBench — #22 · 73.1 CritPt — #15 · 30.9 FrontierMath — #8 · 89.8 Terminal-Bench — #13 · 50.3% ± 3.2% Vending-Bench 2 — #4 · $14,427.85 ± $1,051 Epoch Capabilities Index (ECI) — #12 · 163.75 |
GPT-6.1 Terra | 6 | SimpleBench — #24 · 72.2 FrontierMath — #5 · 90.9 Terminal-Bench — #12 · 53.3% ± 3.1% SWE Atlas — #18 · 57.69 Epoch Capabilities Index (ECI) — #6 · 167.67 Vals AI — #14 · 60.54 |
Grok 4.7 | 6 | Terminal-Bench — #18 · 38.8% ± 3.6% SWE Atlas — #11 · 62.13 GDPval-AA — #17 · 1718 GDPval-AA — #18 · 1717 Design Arena — #13 · 1273 Vals AI — #20 · 58.62 |
inclusionAI/Ling-3.0-flash-VL | 6 | Omni models: video and audio input — Does the model hear the audio from a video file? ours Omni models: video and audio input — How many frames does the model see? ours OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours OCR: handwritten notes, boards and textbook pages — Three hard photos of Polish notes, 2026-10-04 ours OCR: handwritten notes, boards and textbook pages — Textbook pages (print, scans and photos), 2026-10-05 ours OCR: handwritten notes, boards and textbook pages — Describing illustrations (charts, diagrams, maps), 2026- ours |
Inkling 2 Small | 6 | MathArena — #7 · 31.88% ± 5.14% SWE-Bench Pro — #9 · 86.27 SWE Atlas — #13 · 61.89 Design Arena — #7 · 1311 Scale Fortress — #24 · 24 Vals AI — #23 · 56.08 |
Qwen/Qwen3.5-397B-A17B | 6 | OCR: handwritten notes, boards and textbook pages — "Hard" test from 2026-09-29/30: 14 photos, handwriting a ours OCR: handwritten notes, boards and textbook pages — Three hard photos of Polish notes, 2026-10-04 ours OCR: handwritten notes, boards and textbook pages — Textbook pages (print, scans and photos), 2026-10-05 ours OCR: handwritten notes, boards and textbook pages — Describing illustrations (charts, diagrams, maps), 2026- ours Language models: classification, summaries, chat with tools — Lesson summary (description), 2026-09-22 ours Language models: classification, summaries, chat with tools — "Lossless" summaries: how much lesson knowledge survived ours |
zai-org/GLM-5.3-Flash | 6 | Omni models: video and audio input — Does the model hear the audio from a video file? ours Omni models: video and audio input — How many frames does the model see? ours OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours Language models: classification, summaries, chat with tools — Lesson subject classification, 2026-09-22 ours EuroEval — #6 · 1.36 ± 0.04 EuroEval — #9 · 1.54 ± 0.08 |
Claude Fable 5.2 - Provider Adapter | 5 | ARC-AGI (ARC Prize) — #7 · 97.7 ARC-AGI (ARC Prize) — #12 · 97.3 ARC-AGI (ARC Prize) — #14 · 96.8 ARC-AGI (ARC Prize) — #19 · 94.9 ARC-AGI (ARC Prize) — #20 · 92.3 |
Claude Opus 4.7 | 5 | OSWorld 2.0 — #16 · 18.2 OSWorld 2.0 — #18 · 13.9 Vending-Bench 2 — #10 · $10,936.76 ± $1,181 Arena (formerly LMArena / Chatbot Arena) — #8 · 1501 Arena (formerly LMArena / Chatbot Arena) — #12 · 1494 |
Claude Opus 4.8 | 5 | FrontierMath — #20 · 80.0 OSWorld 2.0 — #13 · 20.6 OSWorld 2.0 — #15 · 18.52 Toolathlon (Tool Decathlon) — #3 · 76.2 ± 3.4 Epoch Capabilities Index (ECI) — #21 · 159.94 |
gpt-5.4 | 5 | EQ-Bench — #10 · 1257.7 FrontierMath — #22 · 78.6 F5 Labs CASI & ARS Leaderboard — #4 · 93.21 CyberGym — #13 · 79.0% CyberGym — #18 · 66.3% |
GPT-5.5 Pro | 5 | SimpleBench — #14 · 76.9 SimpleBench — #15 · 76.9 CritPt — #18 · 30.6 FrontierMath — #11 · 87.7 Epoch Capabilities Index (ECI) — #14 · 162.7 |
gpt-6-astra | 5 | LiveBench — #2 · 84.4 Scale Fortress — #9 · 9 EuroEval — #1 · 1.17 ± 0.05 Polish Linguistic and Cultural Competency (PLCC) — #10 · 96.33 Scale SEAL Leaderboards — #2 · 60.6 |
GPT-6.1 Sol - Provider Adapter | 5 | ARC-AGI (ARC Prize) — #16 · 96.4 ARC-AGI (ARC Prize) — #17 · 96.2 ARC-AGI (ARC Prize) — #18 · 95.0 ARC-AGI (ARC Prize) — #21 · 91.0 ARC-AGI (ARC Prize) — #22 · 82.8 |
Muse Spark 1.1 | 5 | OSWorld (OSWorld-Verified) — #7 · 80.7 MultiChallenge — #4 · 75.30 Arena (formerly LMArena / Chatbot Arena) — #14 · 1491 Scale Fortress — #10 · 10 CyberGym — #20 · 59.0% |
Muse Wonder | 5 | FrontierMath — #18 · 82.1 Terminal-Bench — #14 · 47.0% ± 3.8% GDPval-AA — #8 · 1770 Arena (formerly LMArena / Chatbot Arena) — #3 · 1511 Epoch Capabilities Index (ECI) — #15 · 161.68 |
Qwen/Qwen3.5-35B-A3B | 5 | Omni models: video and audio input — Does the model hear the audio from a video file? ours Omni models: video and audio input — How many frames does the model see? ours OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours OCR: handwritten notes, boards and textbook pages — "Hard" test from 2026-09-29/30: 14 photos, handwriting a ours OCR: handwritten notes, boards and textbook pages — Textbook pages (print, scans and photos), 2026-10-05 ours |
Qwen/Qwen3.8-27B | 5 | OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours OCR: handwritten notes, boards and textbook pages — "Hard" test from 2026-09-29/30: 14 photos, handwriting a ours OCR: handwritten notes, boards and textbook pages — Three hard photos of Polish notes, 2026-10-04 ours OCR: handwritten notes, boards and textbook pages — Textbook pages (print, scans and photos), 2026-10-05 ours EuroEval — #17 · 1.67 ± 0.09 |
claude-opus-4-5-20251101-thinking | 4 | Humanity's Last Exam (HLE) — #17 · 25.2 Remote Labor Index (RLI) — #13 · 3.75 MultiChallenge — #18 · 58.97 Scale Fortress — #5 · 5 |
claude-opus-4-6 | 4 | Humanity's Last Exam (HLE) — #24 · 19.0 EQ-Bench — #17 · 1210.0 Remote Labor Index (RLI) — #12 · 4.17 MultiChallenge — #23 · 56.02 |
DeepSeek V4.1 Flash 1017 | 4 | Terminal-Bench — #20 · 38.5% ± 3.8% GDPval-AA — #11 · 1746 Epoch Capabilities Index (ECI) — #22 · 159.41 Vals AI — #13 · 61.26 |
Fable 5.1 | 4 | Humanity's Last Exam (HLE) — #2 · 46.5 Terminal-Bench — #8 · 57.6% ± 3.3% SWE Atlas — #3 · 76.45 Remote Labor Index (RLI) — #4 · 17.92 |
Fable 5.2 | 4 | Terminal-Bench — #2 · 65.8% ± 2.8% SWE Atlas — #1 · 80.55 APEX-Agents — #3 · 80.1% ±4.7% Remote Labor Index (RLI) — #1 · 41.25 |
gemini-3.1-pro-preview | 4 | Humanity's Last Exam (HLE) — #3 · 46.4 EQ-Bench — #25 · 1132.3 MultiChallenge — #8 · 71.37 Polish Linguistic and Cultural Competency (PLCC) — #4 · 97 |
gemini-4-argon | 4 | LiveBench — #8 · 77.7 EQ-Bench — #12 · 1236.3 Polish Linguistic and Cultural Competency (PLCC) — #2 · 97.33 Scale SEAL Leaderboards — #4 · 53.2 |
google/gemma-4-26B-A4B-it | 4 | OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours Language models: classification, summaries, chat with tools — Lesson subject classification, 2026-09-22 ours Platforms: errors, reasoning billing and prices — Prices from /v1/openai/models (2026-10-16) ours Vectara Hallucination Leaderboard — #15 · 5.2 % |
GPT-5.4 Pro | 4 | SimpleBench — #21 · 74.1 CritPt — #20 · 30.0 FrontierMath — #17 · 82.5 Epoch Capabilities Index (ECI) — #19 · 160.24 |
gpt-6.1-sol | 4 | LiveBench — #6 · 81.4 EQ-Bench — #8 · 1272.4 EuroEval — #2 · 1.18 ± 0.04 Polish Linguistic and Cultural Competency (PLCC) — #6 · 96.83 |
Kimi K3 | 4 | SWE Atlas — #15 · 60.98 Toolathlon (Tool Decathlon) — #2 · 76.5 ± 1.9 Arena (formerly LMArena / Chatbot Arena) — #20 · 1487 Epoch Capabilities Index (ECI) — #24 · 158.58 |
moonshotai/Kimi-K2.6 | 4 | OCR: handwritten notes, boards and textbook pages — "Hard" test from 2026-09-29/30: 14 photos, handwriting a ours OCR: handwritten notes, boards and textbook pages — Textbook pages (print, scans and photos), 2026-10-05 ours Language models: classification, summaries, chat with tools — Lesson summary (description), 2026-09-22 ours Language models: classification, summaries, chat with tools — "Lossless" summaries: how much lesson knowledge survived ours |
Muse Spark | 4 | Humanity's Last Exam (HLE) — #7 · 40.6 MultiChallenge — #3 · 75.52 Arena (formerly LMArena / Chatbot Arena) — #18 · 1489 CyberGym — #23 · 43.5% |
Muse Wonder Full | 4 | Vending-Bench 2 — #7 · $12,291.56 ± $885 GDPval-AA — #3 · 1857 LLM Stats — #10 · 52.6 LLM Stats — #14 · 52.0 |
o3 | 4 | Humanity's Last Exam (HLE) — #21 · 20.3 Humanity's Last Exam (HLE) — #23 · 19.2 MMMU / MMMU-Pro — #13 · 76.4 Scale Fortress — #22 · 22 |
XiaomiMiMo/MiMo-V2.6-Pro | 4 | Omni models: video and audio input — Does the model hear the audio from a video file? ours Omni models: video and audio input — How many frames does the model see? ours OCR: handwritten notes, boards and textbook pages — "Hard" test from 2026-09-29/30: 14 photos, handwriting a ours OCR: handwritten notes, boards and textbook pages — Describing illustrations (charts, diagrams, maps), 2026- ours |
Claude Opus 4.6 | 3 | Arena (formerly LMArena / Chatbot Arena) — #6 · 1503 Arena (formerly LMArena / Chatbot Arena) — #11 · 1496 CyberGym — #17 · 66.6% |
Claude Sonnet 4.6 | 3 | OSWorld 2.0 — #20 · 9.3 OSWorld 2.0 — #21 · 8.3 CyberGym — #19 · 65.2% |
claude-opus-5 | 3 | EQ-Bench — #1 · 1364.9 Polish Linguistic and Cultural Competency (PLCC) — #24 · 92.17 Scale SEAL Leaderboards — #6 · 38.6 |
claude-opus-5-5 | 3 | EQ-Bench — #2 · 1339.5 Scale Fortress — #14 · 14 Scale SEAL Leaderboards — #3 · 55 |
claude-opus-5.5 | 3 | LiveBench — #5 · 83.4 MathArena — #4 · 82.61% ± 4.72% Polish Linguistic and Cultural Competency (PLCC) — #21 · 92.5 |
claude-sonnet-4-6 | 3 | EQ-Bench — #20 · 1196.8 OSWorld (OSWorld-Verified) — #16 · 72.1 EuroEval — #20 · 1.69 ± 0.08 |
deepseek-ai/DeepSeek-V4-Flash-Vision-Exp | 3 | OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours OCR: handwritten notes, boards and textbook pages — Describing illustrations (charts, diagrams, maps), 2026- ours Platforms: errors, reasoning billing and prices — Prices from /v1/openai/models (2026-10-16) ours |
deepseek-ai/DeepSeek-V4-Pro | 3 | Language models: classification, summaries, chat with tools — DeepSeek-V4-Pro vs V4.1-Flash on lesson description, 202 ours Reasoning level (reasoning_effort): what models actually do — The minimal value on different models (2026-09-03) ours Reasoning level (reasoning_effort): what models actually do — Cost of reasoning ours |
deepseek-ai/DeepSeek-V4-Pro-0813 | 3 | Language models: classification, summaries, chat with tools — "Lossless" summaries: how much lesson knowledge survived ours Language models: short answers, grading text, time and cost — DeepSeek-V4-Flash: undated name vs dated name (2026-10-1 ours Platforms: errors, reasoning billing and prices — Prices from /v1/openai/models (2026-10-16) ours |
DeepSeek-V4-Pro | 3 | Language models: classification, summaries, chat with tools — Chat with tools, 2026-09-22 ours EQ-Bench — #22 · 1156.3 CyberGym — #7 · 83.3% |
DeepSeek-V4.1-Flash | 3 | Language models: classification, summaries, chat with tools — Chat with tools, 2026-09-22 ours Language models: classification, summaries, chat with tools — Whiteboard photo in chat (1 photo, 1024×1024) ours Design Arena — #11 · 1288 |
GBDT 300× d4 | 3 | ML classifiers on TLSH fingerprints of Mach-O files — Main results (out-of-fold, threshold at 0.1% FP) ours ML classifiers on TLSH fingerprints of Mach-O files — Per-fold thresholds (summed over the 5 test folds) ours ML classifiers on TLSH fingerprints of Mach-O files — Additional recall figures recorded by the script ours |
GBDT 600× d6 | 3 | ML classifiers on TLSH fingerprints of Mach-O files — Main results (out-of-fold, threshold at 0.1% FP) ours ML classifiers on TLSH fingerprints of Mach-O files — Per-fold thresholds (summed over the 5 test folds) ours ML classifiers on TLSH fingerprints of Mach-O files — Additional recall figures recorded by the script ours |
Gemini 3.7 Flash | 3 | Remote Labor Index (RLI) — #11 · 5.00 Arena (formerly LMArena / Chatbot Arena) — #17 · 1489 Epoch Capabilities Index (ECI) — #25 · 158.36 |
Gemini 4 Argon - Provider Adapter | 3 | ARC-AGI (ARC Prize) — #9 · 97.5 ARC-AGI (ARC Prize) — #10 · 97.3 ARC-AGI (ARC Prize) — #11 · 97.3 |
Gemini 4 Carbon - Provider Adapter | 3 | ARC-AGI (ARC Prize) — #2 · 99.1 ARC-AGI (ARC Prize) — #8 · 97.7 ARC-AGI (ARC Prize) — #13 · 96.9 |
gemini-2.5-pro-preview-06-05 | 3 | Humanity's Last Exam (HLE) — #20 · 21.6 Remote Labor Index (RLI) — #23 · 0.83 Polish Linguistic and Cultural Competency (PLCC) — #22 · 92.17 |
gemini-3-pro-preview | 3 | Humanity's Last Exam (HLE) — #8 · 37.5 Remote Labor Index (RLI) — #22 · 1.25 MultiChallenge — #11 · 65.67 |
GLM-5.3 | 3 | Language models: classification, summaries, chat with tools — Chat with tools, 2026-09-22 ours Terminal-Bench — #16 · 41.2% ± 3.3% CyberGym — #5 · 84.5% |
GLM-5.3-Flash | 3 | Language models: classification, summaries, chat with tools — Chat with tools, 2026-09-22 ours LiveBench — #24 · 60.7 Terminal-Bench — #22 · 37.0% ± 3.6% |
google/gemini-3.1-pro | 3 | OCR: exam-sheet photos with handwritten answers — Blank exam-sheet page ours OCR: exam-sheet photos with handwritten answers — Cost and time of a single call ours Platforms: errors, reasoning billing and prices — Prices from /v1/openai/models (2026-10-16) ours |
google/gemini-3.5-flash | 3 | ASR: Polish speech recognition — Test of 2026-09-22: clean speech synthesis, Polish + Eng ours OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours Platforms: errors, reasoning billing and prices — Prices from /v1/openai/models (2026-10-16) ours |
gpt-5-2025-08-07 | 3 | Humanity's Last Exam (HLE) — #15 · 25.3 Humanity's Last Exam (HLE) — #16 · 25.3 Remote Labor Index (RLI) — #20 · 1.67 |
gpt-5.2-2025-12-11 | 3 | Humanity's Last Exam (HLE) — #14 · 27.8 Remote Labor Index (RLI) — #15 · 2.50 Remote Labor Index (RLI) — #19 · 2.08 |
gpt-5.4-pro-2026-03-05 | 3 | Humanity's Last Exam (HLE) — #6 · 44.3 MultiChallenge — #9 · 69.23 Scale Fortress — #19 · 19 |
GPT-5.6 Terra | 3 | CritPt — #21 · 30.0 FrontierMath — #14 · 86.0 Epoch Capabilities Index (ECI) — #17 · 160.73 |
gpt-5.6-sol | 3 | EQ-Bench — #13 · 1236.1 EuroEval — #7 · 1.40 ± 0.04 Polish Linguistic and Cultural Competency (PLCC) — #16 · 94.67 |
gpt-5.6-terra | 3 | LiveBench — #18 · 67.2 EQ-Bench — #15 · 1220.1 EuroEval — #13 · 1.58 ± 0.07 |
GPT-6 Luna | 3 | FrontierMath — #21 · 78.9 SWE Atlas — #22 · 50.80 F5 Labs CASI & ARS Leaderboard — #10 · 89.15 |
gpt-6-sol | 3 | Scale Fortress — #15 · 15 EuroEval — #8 · 1.45 ± 0.05 Polish Linguistic and Cultural Competency (PLCC) — #19 · 93.83 |
Grok 4.6 | 3 | SimpleBench — #18 · 75.9 GDPval-AA — #24 · 1658 CyberGym — #11 · 79.7% |
Hy4 | 3 | Terminal-Bench — #21 · 37.9% ± 3.5% APEX-Agents — #5 · 77.6% ±4.5% Vals AI — #17 · 59.35 |
Kimi-K2.6 | 3 | Language models: classification, summaries, chat with tools — Chat with tools, 2026-09-22 ours Language models: classification, summaries, chat with tools — Whiteboard photo in chat (1 photo, 1024×1024) ours EQ-Bench — #21 · 1191.4 |
Kimi-K3 | 3 | Language models: classification, summaries, chat with tools — Chat with tools, 2026-09-22 ours LiveBench — #20 · 65.0 EQ-Bench — #4 · 1319.2 |
Logistic regression | 3 | ML classifiers on TLSH fingerprints of Mach-O files — Main results (out-of-fold, threshold at 0.1% FP) ours ML classifiers on TLSH fingerprints of Mach-O files — Per-fold thresholds (summed over the 5 test folds) ours ML classifiers on TLSH fingerprints of Mach-O files — Additional recall figures recorded by the script ours |
MiniMax M3 | 3 | SWE Atlas — #25 · 39.26 OSWorld (OSWorld-Verified) — #11 · 75.2 OSWorld 2.0 — #22 · 4.6 |
Mistral Large 4 | 3 | Terminal-Bench — #24 · 27.6% ± 3.9% SWE Atlas — #23 · 45.19 Vals AI — #22 · 56.53 |
MLP 0.2M | 3 | ML classifiers on TLSH fingerprints of Mach-O files — Main results (out-of-fold, threshold at 0.1% FP) ours ML classifiers on TLSH fingerprints of Mach-O files — Per-fold thresholds (summed over the 5 test folds) ours ML classifiers on TLSH fingerprints of Mach-O files — Additional recall figures recorded by the script ours |
MLP 1M | 3 | ML classifiers on TLSH fingerprints of Mach-O files — Main results (out-of-fold, threshold at 0.1% FP) ours ML classifiers on TLSH fingerprints of Mach-O files — Per-fold thresholds (summed over the 5 test folds) ours ML classifiers on TLSH fingerprints of Mach-O files — Additional recall figures recorded by the script ours |
Muse Ember | 3 | SimpleBench — #23 · 72.7 Terminal-Bench — #23 · 30.6% ± 3.8% Vals AI — #15 · 60.48 |
Muse Spark 1.2 | 3 | SimpleBench — #20 · 74.5 Toolathlon (Tool Decathlon) — #4 · 75.9 ± 1.3 Arena (formerly LMArena / Chatbot Arena) — #13 · 1493 |
Muse Spark 1.3 | 3 | SimpleBench — #11 · 81.8 GDPval-AA — #22 · 1681 Arena (formerly LMArena / Chatbot Arena) — #10 · 1497 |
Muse Spark Open | 3 | MathArena — #8 · 27.02% ± 5.61% SWE Atlas — #24 · 42.28 MultiChallenge — #7 · 74.20 |
openai/whisper-large-v3 | 3 | ASR: Polish speech recognition — Test of 2026-09-22: clean speech synthesis, Polish + Eng ours ASR: Polish speech recognition — Degraded recordings, 2026-09-29 (single models, no text ours ASR: Polish speech recognition — Earlier measurement, 2026-09-23 (one deliberately degrad ours |
openai/whisper-large-v3-turbo | 3 | ASR: Polish speech recognition — Test of 2026-09-22: clean speech synthesis, Polish + Eng ours ASR: Polish speech recognition — Degraded recordings, 2026-09-29 (single models, no text ours ASR: Polish speech recognition — Earlier measurement, 2026-09-23 (one deliberately degrad ours |
Opus 5.5 | 3 | Terminal-Bench — #1 · 65.8% ± 3.1% SWE Atlas — #2 · 77.45 Remote Labor Index (RLI) — #2 · 21.25 |
Sonnet 5.5 | 3 | Humanity's Last Exam (HLE) — #4 · 46.1 Terminal-Bench — #4 · 63.9% ± 3.0% SWE Atlas — #7 · 68.28 |
XiaomiMiMo/MiMo-V2.6-Flash | 3 | Omni models: video and audio input — Does the model hear the audio from a video file? ours Omni models: video and audio input — How many frames does the model see? ours OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours |
AdaZoom-GUI-4B | 2 | ScreenSpot-Pro — #9 · 76.79 ScreenSpot-Pro — #22 · 70.59 |
anthropic/claude-opus-5-5 | 2 | Reasoning level (reasoning_effort): what models actually do — The minimal value on different models (2026-09-03) ours Platforms: errors, reasoning billing and prices — Prices from /v1/openai/models (2026-10-16) ours |
Claude Mythos Preview | 2 | CyberGym — #8 · 83.1% LLM Stats — #8 · 53.6 |
Claude Opus 4.5 | 2 | MMMU / MMMU-Pro — #18 · 73.9 CyberGym — #22 · 50.6% |
claude-4-opus-thinking | 2 | MultiChallenge — #19 · 58.62 Kagi LLM Benchmarking Project — #15 · 74.3 |
claude-4-sonnet-thinking | 2 | MultiChallenge — #21 · 57.11 Kagi LLM Benchmarking Project — #20 · 73.0 |
claude-fable-5 | 2 | EQ-Bench — #5 · 1312.5 Polish Linguistic and Cultural Competency (PLCC) — #15 · 95 |
claude-fable-5-2 | 2 | Scale Fortress — #21 · 21 Scale SEAL Leaderboards — #1 · 72.4 |
claude-fable-5.1 | 2 | LiveBench — #3 · 84.0 Polish Linguistic and Cultural Competency (PLCC) — #14 · 95.17 |
claude-fable-5.2 | 2 | LiveBench — #1 · 84.9 Polish Linguistic and Cultural Competency (PLCC) — #3 · 97.33 |
claude-opus-4-1-20250805-thinking | 2 | MultiChallenge — #20 · 57.20 Scale Fortress — #18 · 18 |
claude-opus-4-6-thinking-max | 2 | Humanity's Last Exam (HLE) — #11 · 34.4 Scale Fortress — #13 · 13 |
claude-opus-4-7 | 2 | Humanity's Last Exam (HLE) — #10 · 36.2 EQ-Bench — #7 · 1293.2 |
claude-sonnet-4-5-20250929-thinking | 2 | MultiChallenge — #25 · 55.32 Scale Fortress — #11 · 11 |
DeepSeek V4.1 Flash | 2 | SWE Atlas — #12 · 62.12 Remote Labor Index (RLI) — #7 · 9.58 |
deepseek-ai/DeepSeek-V3.1 | 2 | Vectara Hallucination Leaderboard — #20 · 5.5 % CPTU-Bench — #19 · 4.03 |
deepseek-ai/DeepSeek-V3.2 | 2 | Language models: classification, summaries, chat with tools — Lesson summary (description), 2026-09-22 ours CPTU-Bench — #15 · 4.14 |
deepseek-ai/DeepSeek-V4-Flash-0731 | 2 | Language models: short answers, grading text, time and cost — DeepSeek-V4-Flash: undated name vs dated name (2026-10-1 ours Platforms: errors, reasoning billing and prices — Prices from /v1/openai/models (2026-10-16) ours |
DeepSeek-V4-Flash | 2 | Language models: classification, summaries, chat with tools — Chat with tools, 2026-09-22 ours CyberGym — #15 · 76.7% |
deepseek-v4.1-flash-1017 | 2 | LiveBench — #13 · 70.3 MathArena — #6 · 42.48% ± 6.32% |
Fable 5 | 2 | Terminal-Bench — #15 · 44.8% ± 3.8% Remote Labor Index (RLI) — #6 · 15.83 |
Gemini 3 Pro | 2 | Arena (formerly LMArena / Chatbot Arena) — #22 · 1486 OmniDocBench — #18 · 92.91 |
Gemini 3.1 Pro Preview | 2 | SimpleBench — #13 · 79.6 Arena (formerly LMArena / Chatbot Arena) — #21 · 1487 |
Gemini 3.5 Flash | 2 | SimpleBench — #16 · 76.7 Toolathlon (Tool Decathlon) — #7 · 67.3 ± 1.2 |
gemini-4-carbon | 2 | LiveBench — #4 · 83.5 Polish Linguistic and Cultural Competency (PLCC) — #1 · 98.33 |
gemini-4-hydrogen | 2 | LiveBench — #9 · 77.0 Polish Linguistic and Cultural Competency (PLCC) — #7 · 96.83 |
GLM 5.3 | 2 | SWE Atlas — #17 · 58.10 Vals AI — #16 · 59.85 |
GLM 5.3 Flash | 2 | SWE Atlas — #21 · 55.33 Toolathlon (Tool Decathlon) — #1 · 78.4 ± 1.9 |
glm-5.2 | 2 | EQ-Bench — #18 · 1208.1 CyberGym — #14 · 77.2% |
google/gemini-3.1-flash-lite | 2 | ASR: Polish speech recognition — Test of 2026-09-22: clean speech synthesis, Polish + Eng ours OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours |
google/gemma-3-12b-it | 2 | OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours Vectara Hallucination Leaderboard — #8 · 4.4 % |
google/gemma-3-27b-it | 2 | OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours Language models: classification, summaries, chat with tools — Lesson subject classification, 2026-09-22 ours |
GPT 6 Astra | 2 | Humanity's Last Exam (HLE) — #1 · 54.8 Remote Labor Index (RLI) — #3 · 20.83 |
gpt-5 | 2 | Reasoning level (reasoning_effort): what models actually do — The minimal value on different models (2026-09-03) ours Kagi LLM Benchmarking Project — #21 · 72.7 |
gpt-5-mini-2025-08-07 | 2 | Humanity's Last Exam (HLE) — #22 · 19.4 EuroEval — #23 · 1.71 ± 0.07 |
gpt-5-pro-2025-10-06 | 2 | Humanity's Last Exam (HLE) — #12 · 31.6 Scale Fortress — #20 · 20 |
gpt-5.4-2026-03-05 | 2 | Humanity's Last Exam (HLE) — #9 · 36.2 Polish Linguistic and Cultural Competency (PLCC) — #23 · 92.17 |
gpt-6-luna | 2 | LiveBench — #25 · 60.4 Scale Fortress — #6 · 6 |
gpt-6.1-terra | 2 | LiveBench — #7 · 78.8 Polish Linguistic and Cultural Competency (PLCC) — #18 · 94 |
gpt-oss-120b | 2 | Language models: classification, summaries, chat with tools — Chat with tools, 2026-09-22 ours Scale Fortress — #3 · 3 |
GroundCascade | 2 | ScreenSpot-Pro — #1 · 85.14 ScreenSpot-Pro — #2 · 85.01 |
Haiku 5.5 | 2 | Terminal-Bench — #17 · 40.0% ± 3.6% SWE Atlas — #16 · 59.43 |
Holo2-235B-A22B | 2 | ScreenSpot-Pro — #7 · 78.49 ScreenSpot-Pro — #23 · 70.59 |
Kimi K2.5 | 2 | OSWorld (OSWorld-Verified) — #24 · 63.3 CyberGym — #25 · 41.3% |
kimi-k2.5 | 2 | Humanity's Last Exam (HLE) — #18 · 24.4 MultiChallenge — #15 · 61.39 |
MAI-UI-32B | 2 | ScreenSpot-Pro — #8 · 77.46 ScreenSpot-Pro — #13 · 73.50 |
meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8 | 2 | OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours CPTU-Bench — #25 · 3.93 |
meta-llama/Llama-4-Scout-17B-16E-Instruct | 2 | OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours Language models: classification, summaries, chat with tools — Lesson subject classification, 2026-09-22 ours |
mimo-v2.6-pro | 2 | LiveBench — #19 · 66.1 GDPval-AA — #21 · 1692 |
MiniMax-M3 | 2 | Language models: classification, summaries, chat with tools — "Lossless" summaries: how much lesson knowledge survived ours EQ-Bench — #24 · 1140.1 |
MiniMaxAI/MiniMax-M3 | 2 | OCR: handwritten notes, boards and textbook pages — "Hard" test from 2026-09-29/30: 14 photos, handwriting a ours OCR: handwritten notes, boards and textbook pages — Describing illustrations (charts, diagrams, maps), 2026- ours |
mistralai/Mistral-Small-3.2-24B-Instruct-2506 | 2 | OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours OCR: handwritten notes, boards and textbook pages — Cheaper models, 2026-10-06 (the same 21 readings) ours |
moonshotai/Kimi-K3 | 2 | OCR: handwritten notes, boards and textbook pages — Three hard photos of Polish notes, 2026-10-04 ours Platforms: errors, reasoning billing and prices — Prices from /v1/openai/models (2026-10-16) ours |
muse-spark-1.3 | 2 | LiveBench — #16 · 68.6 Polish Linguistic and Cultural Competency (PLCC) — #25 · 91.83 |
muse-wonder | 2 | EQ-Bench — #3 · 1324.9 Polish Linguistic and Cultural Competency (PLCC) — #20 · 93.83 |
Qwen/Qwen3-32B | 2 | Language models: classification, summaries, chat with tools — Lesson subject classification, 2026-09-22 ours Vectara Hallucination Leaderboard — #25 · 5.9 % |
Qwen/Qwen3-ASR-0.6B | 2 | ASR: Polish speech recognition — Test of 2026-09-22: clean speech synthesis, Polish + Eng ours ASR: Polish speech recognition — Degraded recordings, 2026-09-29 (single models, no text ours |
Qwen/Qwen3-ASR-1.7B | 2 | ASR: Polish speech recognition — Test of 2026-09-22: clean speech synthesis, Polish + Eng ours ASR: Polish speech recognition — Degraded recordings, 2026-09-29 (single models, no text ours |
Qwen/Qwen3-Embedding-0.6B | 2 | Embeddings: retrieval accuracy in Polish — Data ours PL-MTEB (Polish MTEB) — #19 · 65.50 |
Qwen/Qwen3-Embedding-4B | 2 | Embeddings: retrieval accuracy in Polish — Data ours PL-MTEB (Polish MTEB) — #4 · 72.34 |
Qwen/Qwen3-Embedding-8B | 2 | Embeddings: retrieval accuracy in Polish — Data ours PL-MTEB (Polish MTEB) — #3 · 73.54 |
Qwen/Qwen3.5-9B | 2 | OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours OCR: handwritten notes, boards and textbook pages — Cheaper models, 2026-10-06 (the same 21 readings) ours |
Qwen/Qwen3.6-35B-A3B | 2 | OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours Platforms: errors, reasoning billing and prices — Prices from /v1/openai/models (2026-10-16) ours |
Qwen3.5-397B | 2 | Language models: classification, summaries, chat with tools — Chat with tools, 2026-09-22 ours Language models: classification, summaries, chat with tools — Whiteboard photo in chat (1 photo, 1024×1024) ours |
Qwen3.8 Max | 2 | FrontierMath — #25 · 74.7 GDPval-AA — #23 · 1674 |
qwen3.8-max | 2 | LiveBench — #23 · 60.9 Arena (formerly LMArena / Chatbot Arena) — #25 · 1484 |
qwen4-max | 2 | LiveBench — #11 · 73.3 Design Arena — #5 · 1360 |
Soniox stt-async-v6 | 2 | ASR: Polish speech recognition — External providers on the same recordings, 2026-11-06 ours Diarization: who is speaking — Comparison with external providers, 2026-11-06 ours |
UiPath Screen Agent w/ Opus 4.5 | 2 | OSWorld (OSWorld-Verified) — #18 · 67.1 OSWorld (OSWorld-Verified) — #21 · 64.4 |
ABot-OCR | 1 | OmniDocBench — #15 · 93.30 |
agent s3 w/ GPT-5 bBoN | 1 | OSWorld (OSWorld-Verified) — #17 · 69.9 |
agent s3 w/ Opus 4.5 + GPT-5 bBoN | 1 | OSWorld (OSWorld-Verified) — #15 · 72.6 |
agent s3 w/ Opus 4.5 bBoN | 1 | OSWorld (OSWorld-Verified) — #19 · 66.0±1.1 |
ahocorasick_rs | 1 | Scanner performance: database loading, re-scans, TLSH in SQL, Aho-Corasick — External libraries ours |
ai21labs/jamba-mini-2 | 1 | Vectara Hallucination Leaderboard — #16 · 5.3 % |
Alibaba-NLP/gte-Qwen2-1.5B-instruct | 1 | PL-MTEB (Polish MTEB) — #12 · 70.00 |
allenai/olmOCR-2-7B-1025 | 1 | OCR: handwritten notes, boards and textbook pages — Cheaper models, 2026-10-06 (the same 21 readings) ours |
amazon/nova-2-lite-v1:0 | 1 | Vectara Hallucination Leaderboard — #12 · 5.1 % |
amazon/nova-micro-v1:0 | 1 | Vectara Hallucination Leaderboard — #19 · 5.5 % |
amazon/nova-pro-v1:0 | 1 | Vectara Hallucination Leaderboard — #11 · 5.1 % |
antgroup/finix_s1_32b | 1 | Vectara Hallucination Leaderboard — #1 · 1.8 % |
Anthropic Claude Code | 1 | SWE-rebench — #7 · 60.4 |
Anthropic Fable 5 [high] | 1 | SWE-rebench — #1 · 64.5 |
Anthropic Opus 5 [high] | 1 | SWE-rebench — #3 · 63.4 |
Anthropic Sonnet 5 [high] | 1 | SWE-rebench — #9 · 56.8 |
anthropic/claude-fable-5 | 1 | Platforms: errors, reasoning billing and prices — Prices from /v1/openai/models (2026-10-16) ours |
anthropic/claude-haiku-5.5-20261007 | 1 | OpenRouter Rankings — #18 · 2779.1 |
anthropic/claude-opus-4.6 [low] | 1 | Context Arena (MRCR) — #25 · 68.8 |
anthropic/claude-opus-4.6 [medium] | 1 | Context Arena (MRCR) — #20 · 72.5 |
anthropic/claude-opus-4.8 [max] | 1 | Context Arena (MRCR) — #16 · 75.2 |
anthropic/claude-opus-5 [max] | 1 | Context Arena (MRCR) — #3 · 91.3 |
anthropic/claude-opus-5.5-20260921 | 1 | OpenRouter Rankings — #6 · 7365.4 |
anthropic/claude-sonnet-4.6 [high] | 1 | Context Arena (MRCR) — #21 · 72.4 |
anthropic/claude-sonnet-4.6 [low] | 1 | Context Arena (MRCR) — #19 · 72.6 |
anthropic/claude-sonnet-4.6 [medium] | 1 | Context Arena (MRCR) — #22 · 71.7 |
anthropic/claude-sonnet-5-5 | 1 | Platforms: errors, reasoning billing and prices — Prices from /v1/openai/models (2026-10-16) ours |
anthropic/claude-sonnet-5.5-20260928 | 1 | OpenRouter Rankings — #19 · 2666.3 |
assemblyai/universal-3-5-pro | 1 | Open ASR Leaderboard — #15 · 6.21 |
Aurora | 1 | TTS Arena V2 — #1 · 1687 |
BAAI/bge-m3 | 1 | Embeddings: retrieval accuracy in Polish — Data ours |
baidu/ernie-5.5-flash-preview-20261110 | 1 | OpenRouter Rankings — #20 · 2470.8 |
Basin | 1 | CyberGym — #10 · 80.0% |
beam-1p0-501b-a23b | 1 | Scale Fortress — #7 · 7 |
Building the automaton in threads | 1 | Scanner performance: database loading, re-scans, TLSH in SQL, Aho-Corasick — External libraries ours |
ByteDance/Seed-2.0-mini | 1 | OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours |
ByteDance/Seed-2.0-pro | 1 | OCR: handwritten notes, boards and textbook pages — Describing illustrations (charts, diagrams, maps), 2026- ours |
CastleFlow v1.0 | 1 | TTS Arena V2 — #4 · 1558 |
Chance Vision 1.5 | 1 | MMMU / MMMU-Pro — #1 · 86.9 |
ChatGPT agent | 1 | Remote Labor Index (RLI) — #21 · 1.25 |
chatgpt-4o | 1 | Kagi LLM Benchmarking Project — #14 · 75.0 |
Claude 3.5 Sonnet | 1 | Scale Fortress — #12 · 12 |
Claude Fable 5.1 Max Effort | 1 | F5 Labs CASI & ARS Leaderboard — #7 · 92.30 |
Claude Mythos 5.2 | 1 | LLM Stats — #2 · 59.3 |
Claude Opus 4.6 w/ tools | 1 | MMMU / MMMU-Pro — #11 · 77.3 |
Claude Opus 4.6 w/o tools | 1 | MMMU / MMMU-Pro — #17 · 73.9 |
Claude Opus 5.5 Reasoning | 1 | F5 Labs CASI & ARS Leaderboard — #8 · 92.00 |
Claude Sonnet 4.5 | 1 | MMMU / MMMU-Pro — #21 · 68.9 |
Claude Sonnet 4.6 w/ tools | 1 | MMMU / MMMU-Pro — #15 · 75.6 |
Claude Sonnet 4.6 w/o tools | 1 | MMMU / MMMU-Pro — #16 · 74.5 |
Claude Sonnet 5 | 1 | F5 Labs CASI & ARS Leaderboard — #1 · 98.00 |
claude_3_5_sonnet_20240620_inspect | 1 | METR Time Horizons — #18 · 11.4 min (5.5-22.4) |
claude_3_5_sonnet_20241022_inspect | 1 | METR Time Horizons — #16 · 20.5 min (10.1-40.8) |
claude_3_7_sonnet_inspect | 1 | METR Time Horizons — #14 · 60.4 min (33.0-104.2) |
claude_3_opus_inspect | 1 | METR Time Horizons — #22 · 4.0 min (1.7-8.8) |
claude_4_1_opus_inspect | 1 | METR Time Horizons — #12 · 100.5 min (59.3-159.5) |
claude_4_opus_inspect | 1 | METR Time Horizons — #13 · 100.4 min (60.0-163.5) |
claude_mythos_preview_early_inspect | 1 | METR Time Horizons — #1 · 1044.8 min (508.9-3304.3) |
claude_opus_4_5_inspect | 1 | METR Time Horizons — #7 · 293.0 min (161.7-623.7) |
claude_opus_4_6_inspect | 1 | METR Time Horizons — #2 · 718.8 min (316.7-3633.8) |
claude-4-5-opus | 1 | Kagi LLM Benchmarking Project — #25 · 70.7 |
claude-4-5-opus-thinking | 1 | Kagi LLM Benchmarking Project — #8 · 80.2 |
claude-4-5-Sonnet | 1 | Remote Labor Index (RLI) — #18 · 2.08 |
claude-4-6-opus | 1 | Kagi LLM Benchmarking Project — #23 · 72.4 |
claude-4-6-opus-thinking | 1 | Kagi LLM Benchmarking Project — #5 · 83.6 |
claude-4-7-opus | 1 | Kagi LLM Benchmarking Project — #19 · 73.3 |
claude-4-7-opus-thinking | 1 | Kagi LLM Benchmarking Project — #7 · 80.7 |
claude-4-8-opus-thinking | 1 | Kagi LLM Benchmarking Project — #3 · 88.8 |
claude-5-fable | 1 | Kagi LLM Benchmarking Project — #2 · 88.8 |
claude-5-fable-thinking | 1 | Kagi LLM Benchmarking Project — #1 · 91.4 |
claude-fable-5_max | 1 | Epoch AI Benchmarking Hub — #12 · 87.0 |
claude-fable-5-1 | 1 | Scale SEAL Leaderboards — #5 · 51.3 |
claude-fable-5-1_max | 1 | Epoch AI Benchmarking Hub — #7 · 90.2 |
claude-fable-5-2_max | 1 | Epoch AI Benchmarking Hub — #1 · 95.1 |
claude-fable-5-max | 1 | MCPMark — #4 · 86.61 |
claude-fable-5[1m] | 1 | OSWorld (OSWorld-Verified) — #2 · 86.0 |
claude-haiku-5-5_max | 1 | Epoch AI Benchmarking Hub — #24 · 75.1 |
claude-haiku-5.5 | 1 | LiveBench — #14 · 68.9 |
claude-opus-4-1-20250805 | 1 | Scale Fortress — #23 · 23 |
claude-opus-4-5-20251101 | 1 | Scale Fortress — #16 · 16 |
claude-opus-4-8 | 1 | EQ-Bench — #9 · 1267.8 |
claude-opus-4-8_max | 1 | Epoch AI Benchmarking Hub — #20 · 80.0 |
claude-opus-4-8-max | 1 | MCPMark — #6 · 76.38 |
claude-opus-5_max | 1 | Epoch AI Benchmarking Hub — #15 · 85.6 |
claude-opus-5-5_max | 1 | Epoch AI Benchmarking Hub — #4 · 91.2 |
claude-opus-5-5-max | 1 | Scale Fortress — #1 · 1 |
claude-opus-5[1m] | 1 | OSWorld (OSWorld-Verified) — #4 · 83.4 |
claude-sonnet-4-5-20250929 | 1 | OSWorld (OSWorld-Verified) — #25 · 62.9 |
claude-sonnet-4-5-20250929#thinking | 1 | EuroEval — #15 · 1.56 ± 0.08 |
claude-sonnet-5 | 1 | EQ-Bench — #14 · 1221.2 |
claude-sonnet-5-5_max | 1 | Epoch AI Benchmarking Hub — #10 · 88.8 |
claude-sonnet-5.5 | 1 | LiveBench — #10 · 77.0 |
Coasty CUA v1 | 1 | OSWorld (OSWorld-Verified) — #5 · 82.8 |
codefuse-ai/F2LLM-v2-0.6B | 1 | PL-MTEB (Polish MTEB) — #14 · 69.63 |
codefuse-ai/F2LLM-v2-1.7B | 1 | PL-MTEB (Polish MTEB) — #8 · 72.03 |
codefuse-ai/F2LLM-v2-14B | 1 | PL-MTEB (Polish MTEB) — #1 · 75.13 |
codefuse-ai/F2LLM-v2-330M | 1 | PL-MTEB (Polish MTEB) — #18 · 66.53 |
codefuse-ai/F2LLM-v2-4B | 1 | PL-MTEB (Polish MTEB) — #5 · 73.42 |
codefuse-ai/F2LLM-v2-8B | 1 | PL-MTEB (Polish MTEB) — #2 · 74.61 |
Codex GPT 5.5 | 1 | Remote Labor Index (RLI) — #9 · 6.25 |
CohereLabs/cohere-transcribe-03-2026 | 1 | Open ASR Leaderboard — #18 · 6.72 |
Creation Model | 1 | CyberGym — #1 · 90.6% |
Cursor Cursor | 1 | SWE-rebench — #10 · 51.7 |
davinci_002 | 1 | METR Time Horizons — #25 · 0.1 min (0.1-0.2) |
Deepdub eTTS 3.2 | 1 | TTS Arena V2 — #14 · 1532 |
DeepSeek DeepSeek-V4 Pro [high] | 1 | SWE-rebench — #14 · 40.2 |
DeepSeek V4 Pro 0813 | 1 | Toolathlon (Tool Decathlon) — #5 · 74.4 ± 1.2 |
Deepseek V4p1 Flash 1017 | 1 | MultiChallenge — #5 · 74.90 |
deepseek-ai/DeepSeek-OCR | 1 | OCR: handwritten notes, boards and textbook pages — Cheaper models, 2026-10-06 (the same 21 readings) ours |
deepseek-ai/DeepSeek-R1 | 1 | CPTU-Bench — #16 · 4.14 |
deepseek-ai/DeepSeek-V3 | 1 | CPTU-Bench — #20 · 4.02 |
deepseek-ai/DeepSeek-V3-0324 | 1 | CPTU-Bench — #18 · 4.03 |
deepseek-ai/DeepSeek-V3.2-Exp | 1 | Vectara Hallucination Leaderboard — #17 · 5.3 % |
DeepSeek-OCR 2 | 1 | OmniDocBench — #23 · 90.25 |
DeepSeek-V3.2 | 1 | Language models: classification, summaries, chat with tools — Chat with tools, 2026-09-22 ours |
DeepSeek-V4-Flash, Qwen3.8-Flash | 1 | Language models: classification, summaries, chat with tools — Whiteboard photo in chat (1 photo, 1024×1024) ours |
DeepSeek-V4-Pro Preview | 1 | CyberGym — #21 · 57.7% |
deepseek-v4-pro-max | 1 | MCPMark — #8 · 71.65 |
deepseek/deepseek-flash#low | 1 | EuroEval — #10 · 1.50 ± 0.05 |
deepseek/deepseek-flash#no-thinking | 1 | EuroEval — #16 · 1.57 ± 0.06 |
deepseek/deepseek-v4-flash-20260731 | 1 | OpenRouter Rankings — #14 · 3196.4 |
deepseek/deepseek-v4.1-flash-20260910 | 1 | OpenRouter Rankings — #3 · 15455.0 |
deepseek/deepseek-v4.1-flash-20261017 | 1 | OpenRouter Rankings — #1 · 61170.1 |
deepvk/USER-bge-m3 | 1 | PL-MTEB (Polish MTEB) — #25 · 62.46 |
dots.ocr | 1 | OmniDocBench — #21 · 90.77 |
dots.vlm1 | 1 | MMMU / MMMU-Pro — #20 · 70.1 |
Duvo Eye-1 | 1 | ScreenSpot-Pro — #17 · 72.93 |
Edge0/ARK-ASR-0.6B | 1 | Open ASR Leaderboard — #21 · 6.75 |
Eleven Flash v2.5 | 1 | TTS Arena V2 — #18 · 1503 |
Eleven Multilingual v2 | 1 | TTS Arena V2 — #15 · 1507 |
Eleven Turbo v2.5 | 1 | TTS Arena V2 — #16 · 1505 |
Eleven v3 | 1 | TTS Arena V2 — #20 · 1521 |
elevenlabs/scribe_v2 | 1 | Open ASR Leaderboard — #7 · 5.93 |
embeddinggemma-300m | 1 | Embeddings: retrieval accuracy in Polish — Data ours |
Fable 5.1 high | 1 | SWE-Bench Pro — #2 · 92.20 |
Fable-5.1 | 1 | Scale Fortress — #17 · 17 |
Feyospace-v1.1 | 1 | CyberGym — #6 · 84.4% |
FireRed-OCR | 1 | OmniDocBench — #16 · 93.26 |
FluidVoice-Labs/fluid-speech-pro | 1 | Open ASR Leaderboard — #11 · 6.06 |
GBOX Agent | 1 | OSWorld (OSWorld-Verified) — #22 · 64.2 |
Gemini 2.5 Pro 05-06 | 1 | MMMU / MMMU-Pro — #23 · 68.0 |
Gemini 2.5 Pro Experimental | 1 | Humanity's Last Exam (HLE) — #25 · 18.2 |
Gemini 3 Flash | 1 | OmniDocBench — #19 · 92.62 |
Gemini 3 Pro Preview | 1 | SimpleBench — #17 · 76.4 |
Gemini 3.0 Pro | 1 | MMMU / MMMU-Pro — #4 · 81.0 |
Gemini 3.1 Pro Thinking | 1 | MMMU / MMMU-Pro — #5 · 80.5 |
Gemini 3.8 Flash Cyber | 1 | CyberGym — #3 · 86.3% |
Gemini 3.8 Flash high | 1 | SWE-Bench Pro — #15 · 58.80 |
Gemini 4 Argon high | 1 | SWE-Bench Pro — #3 · 92.16 |
Gemini Argon 4 | 1 | Remote Labor Index (RLI) — #5 · 17.08 |
gemini_3_1_pro | 1 | METR Time Horizons — #3 · 384.1 min (233.5-694.8) |
gemini_3_pro | 1 | METR Time Horizons — #8 · 224.3 min (139.6-379.2) |
gemini-2.0-flash-001 | 1 | CPTU-Bench — #8 · 4.29 |
gemini-2.0-flash-lite-001 | 1 | CPTU-Bench — #17 · 4.09 |
gemini-3-pro | 1 | Kagi LLM Benchmarking Project — #9 · 80.1 |
Gemini-3.0-Pro-Preview | 1 | Polish Linguistic and Cultural Competency (PLCC) — #11 · 95.83 |
gemini-3.1-flash-lite-preview | 1 | MultiChallenge — #16 · 60.61 |
Gemini-3.5-Flash | 1 | Polish Linguistic and Cultural Competency (PLCC) — #9 · 96.33 |
Gemini-3.6-Flash | 1 | Polish Linguistic and Cultural Competency (PLCC) — #13 · 95.5 |
Gemini-3.7-Flash | 1 | Polish Linguistic and Cultural Competency (PLCC) — #8 · 96.67 |
Gemini-3.8-Flash | 1 | Polish Linguistic and Cultural Competency (PLCC) — #5 · 97 |
gemini-4-argon_high | 1 | Epoch AI Benchmarking Hub — #13 · 86.0 |
gemini-4-carbon_high | 1 | Epoch AI Benchmarking Hub — #6 · 90.5 |
gemini/gemini-3-flash-preview#thinking | 1 | EuroEval — #4 · 1.32 ± 0.04 |
gemini/gemini-3.1-flash-lite-preview | 1 | EuroEval — #12 · 1.59 ± 0.08 |
gemini/gemini-3.6-flash | 1 | EuroEval — #5 · 1.31 ± 0.04 |
gemini/gemini-3.7-flash | 1 | EuroEval — #3 · 1.24 ± 0.03 |
Gemma 4 26B A4B | 1 | MMMU / MMMU-Pro — #19 · 73.8 |
Gemma 4 31B | 1 | MMMU / MMMU-Pro — #12 · 76.9 |
gladia/solaria-3 | 1 | Open ASR Leaderboard — #24 · 6.86 |
GLM-5 | 1 | CyberGym — #24 · 43.2% |
glm-5-thinking | 1 | Kagi LLM Benchmarking Project — #12 · 75.0 |
GLM-5.1 | 1 | CyberGym — #16 · 68.7% |
GLM-5.3 max | 1 | SWE-Bench Pro — #11 · 84.30 |
GLM-OCR | 1 | OmniDocBench — #7 · 95.22 |
google/gemini-2.5-flash-lite | 1 | Vectara Hallucination Leaderboard — #3 · 3.3 % |
google/gemini-3.6-flash [high] | 1 | Context Arena (MRCR) — #15 · 77.3 |
google/gemini-3.6-flash [medium] | 1 | Context Arena (MRCR) — #17 · 74.5 |
google/gemini-3.7-flash | 1 | Platforms: errors, reasoning billing and prices — Prices from /v1/openai/models (2026-10-16) ours |
google/gemini-3.7-flash [high] | 1 | Context Arena (MRCR) — #4 · 88.6 |
google/gemini-3.7-flash [low] | 1 | Context Arena (MRCR) — #14 · 78.3 |
google/gemini-3.7-flash [medium] | 1 | Context Arena (MRCR) — #8 · 82.1 |
google/gemma-4-26B-A4B-it non-reasoning | 1 | CPTU-Bench — #11 · 4.23 |
google/gemma-4-26B-A4B-it reasoning | 1 | CPTU-Bench — #13 · 4.21 |
google/gemma-4-31B-it non-reasoning | 1 | CPTU-Bench — #5 · 4.32 |
google/gemma-4-31B-it reasoning | 1 | CPTU-Bench — #6 · 4.31 |
google/gemma-4-31B-it-turbo | 1 | OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours |
gpt 6.1 Sol | 1 | MultiChallenge — #1 · 81.95 |
GPT 6.1 Terra | 1 | MultiChallenge — #6 · 74.20 |
gpt_3_5_turbo_instruct | 1 | METR Time Horizons — #24 · 0.6 min (0.3-1.1) |
gpt_4 | 1 | METR Time Horizons — #21 · 4.0 min (1.9-8.0) |
gpt_4_1106_inspect | 1 | METR Time Horizons — #20 · 4.0 min (1.9-8.4) |
gpt_4_turbo_inspect | 1 | METR Time Horizons — #23 · 3.7 min (2.0-6.7) |
gpt_4o_inspect | 1 | METR Time Horizons — #19 · 7.0 min (4.0-12.9) |
gpt_5_1_codex_max_inspect | 1 | METR Time Horizons — #9 · 223.7 min (134.3-396.2) |
gpt_5_2 | 1 | METR Time Horizons — #4 · 352.2 min (198.1-815.2) |
gpt_5_2025_08_07_inspect | 1 | METR Time Horizons — #10 · 203.0 min (112.6-405.6) |
gpt_5_3_codex | 1 | METR Time Horizons — #5 · 349.5 min (194.9-816.4) |
gpt_5_4 | 1 | METR Time Horizons — #6 · 341.7 min (186.6-768.8) |
gpt-4o-mini-transcribe | 1 | ASR: Polish speech recognition — External providers on the same recordings, 2026-11-06 ours |
gpt-4o-transcribe | 1 | ASR: Polish speech recognition — External providers on the same recordings, 2026-11-06 ours |
gpt-4o-transcribe-diarize | 1 | ASR: Polish speech recognition — External providers on the same recordings, 2026-11-06 ours |
GPT-5 w/ thinking | 1 | MMMU / MMMU-Pro — #10 · 78.4 |
gpt-5-2 | 1 | Kagi LLM Benchmarking Project — #18 · 73.3 |
gpt-5-5 | 1 | Kagi LLM Benchmarking Project — #4 · 88.8 |
gpt-5-5-xhigh | 1 | MCPMark — #2 · 92.91 |
gpt-5-6-sol-max | 1 | MCPMark — #3 · 92.91 |
gpt-5-mini-thinking | 1 | MultiChallenge — #17 · 58.99 |
gpt-5-pro | 1 | Kagi LLM Benchmarking Project — #11 · 76.8 |
gpt-5-thinking | 1 | MultiChallenge — #13 · 63.19 |
GPT-5.1 | 1 | MMMU / MMMU-Pro — #14 · 76.0 |
GPT-5.1 Thinking | 1 | MMMU / MMMU-Pro — #9 · 79.0 |
gpt-5.1-2025-11-13-thinking | 1 | MultiChallenge — #12 · 63.41 |
gpt-5.1-thinking | 1 | Humanity's Last Exam (HLE) — #19 · 23.7 |
GPT-5.2 Thinking w/o Python | 1 | MMMU / MMMU-Pro — #7 · 80.4 |
GPT-5.2 Thinking w/o tools | 1 | MMMU / MMMU-Pro — #8 · 79.5 |
GPT-5.4 Thinking w/ tools | 1 | MMMU / MMMU-Pro — #2 · 82.1 |
GPT-5.4 Thinking w/o tools | 1 | MMMU / MMMU-Pro — #3 · 81.2 |
gpt-5.4-2026-03-05_xhigh | 1 | Epoch AI Benchmarking Hub — #22 · 78.6 |
gpt-5.4-pro-2026-03-05_xhigh | 1 | Epoch AI Benchmarking Hub — #17 · 82.5 |
gpt-5.5_xhigh | 1 | Epoch AI Benchmarking Hub — #16 · 85.3 |
GPT-5.5-Cyber | 1 | CyberGym — #4 · 85.6% |
gpt-5.5-pro_xhigh | 1 | Epoch AI Benchmarking Hub — #11 · 87.7 |
gpt-5.5-xhigh | 1 | Scale Fortress — #25 · 25 |
gpt-5.6 | 1 | Reasoning level (reasoning_effort): what models actually do — The minimal value on different models (2026-09-03) ours |
GPT-5.6 Luna | 1 | FrontierMath — #19 · 82.1 |
GPT-5.6 Sol xhigh | 1 | SWE-Bench Pro — #12 · 82.40 |
GPT-5.6 Terra xhigh | 1 | SWE-Bench Pro — #8 · 86.30 |
gpt-5.6-luna | 1 | EQ-Bench — #23 · 1145.9 |
gpt-5.6-luna_max | 1 | Epoch AI Benchmarking Hub — #19 · 82.1 |
gpt-5.6-sol_max | 1 | Epoch AI Benchmarking Hub — #9 · 89.1 |
gpt-5.6-terra / gpt-5.6-sol | 1 | Reasoning level (reasoning_effort): what models actually do — Reasoning and the output token limit ours |
gpt-5.6-terra_max | 1 | Epoch AI Benchmarking Hub — #14 · 86.0 |
GPT-6 Astra high | 1 | SWE-Bench Pro — #4 · 90.20 |
gpt-6-astra_max | 1 | Epoch AI Benchmarking Hub — #3 · 93.7 |
gpt-6-luna_max | 1 | Epoch AI Benchmarking Hub — #21 · 78.9 |
gpt-6-sol_max | 1 | Epoch AI Benchmarking Hub — #8 · 89.8 |
gpt-6.1-sol_max | 1 | Epoch AI Benchmarking Hub — #2 · 93.7 |
gpt-6.1-terra_max | 1 | Epoch AI Benchmarking Hub — #5 · 90.9 |
gpt-transcribe | 1 | ASR: Polish speech recognition — External providers on the same recordings, 2026-11-06 ours |
Gradium TTS | 1 | TTS Arena V2 — #10 · 1554 |
Gradium TTS 2026.10 | 1 | TTS Arena V2 — #9 · 1588 |
Grok 4.5 | 1 | CyberGym — #12 · 79.0% |
Grok 4.7 High | 1 | F5 Labs CASI & ARS Leaderboard — #3 · 93.31 |
Grok Grok 4.5 [high] | 1 | SWE-rebench — #2 · 63.8 |
grok-4 | 1 | Kagi LLM Benchmarking Project — #17 · 73.6 |
grok-4-20-thinking | 1 | Kagi LLM Benchmarking Project — #13 · 75.0 |
grok-4-5 | 1 | Kagi LLM Benchmarking Project — #6 · 83.5 |
grok-4.6 | 1 | LiveBench — #22 · 63.5 |
grok-4.7 | 1 | LiveBench — #17 · 67.9 |
GTA1 w/ GPT-5 | 1 | OSWorld (OSWorld-Verified) — #23 · 63.4 |
Haiku 4.5 xhigh | 1 | SWE-Bench Pro — #18 · 25.50 |
Haiku 5.5 high | 1 | SWE-Bench Pro — #17 · 52.94 |
HIPPO Agent w/ Opus 4.5 | 1 | OSWorld (OSWorld-Verified) — #12 · 74.5 |
HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1 | 1 | PL-MTEB (Polish MTEB) — #22 · 64.57 |
HojoAI/Hojo-ASR-V1 | 1 | Open ASR Leaderboard — #19 · 6.74 |
Holo2-30B-A3B | 1 | ScreenSpot-Pro — #11 · 75.21 |
Holo2-8B | 1 | ScreenSpot-Pro — #20 · 71.41 |
Holo3-35B-A3B | 1 | OSWorld (OSWorld-Verified) — #8 · 80.4±2.2 |
hotchpotch/bekko-embedding-v1-a25m | 1 | PL-MTEB (Polish MTEB) — #23 · 61.50 |
Hume Octave | 1 | TTS Arena V2 — #6 · 1551 |
HunyuanOCR | 1 | OmniDocBench — #24 · 89.95 |
HuzzleWorld-2B | 1 | ScreenSpot-Pro — #21 · 70.84 |
HuzzleWorld-3B | 1 | ScreenSpot-Pro — #18 · 72.42 |
ibm-granite/granite-4.0-h-small | 1 | Vectara Hallucination Leaderboard — #14 · 5.2 % |
ibm-granite/granite-speech-5.0-2b | 1 | Open ASR Leaderboard — #17 · 6.68 |
Indeed-UI-32B | 1 | ScreenSpot-Pro — #15 · 73.31 |
Indeed-UI-32B-zoomin | 1 | ScreenSpot-Pro — #3 · 82.73 |
Indeed-UI-8B | 1 | ScreenSpot-Pro — #14 · 73.37 |
Indeed-UI-8B-zoom-in | 1 | ScreenSpot-Pro — #4 · 80.96 |
Inkling | 1 | EQ-Bench — #16 · 1212.1 |
Inkling xhigh | 1 | SWE-Bench Pro — #16 · 56.90 |
Inkling, Inkling-Small, gemma-4-31B-it | 1 | ASR: Polish speech recognition — Degraded recordings, 2026-09-29 (single models, no text ours |
Intelligence-Indeed Agent | 1 | OSWorld (OSWorld-Verified) — #1 · 90.2 |
intfloat/e5-mistral-7b-instruct | 1 | PL-MTEB (Polish MTEB) — #11 · 69.94 |
intfloat/multilingual-e5-large-instruct | 1 | PL-MTEB (Polish MTEB) — #10 · 69.13 |
Inworld TTS | 1 | TTS Arena V2 — #7 · 1543 |
Inworld TTS 1.5 MAX | 1 | TTS Arena V2 — #5 · 1565 |
Inworld TTS MAX | 1 | TTS Arena V2 — #3 · 1567 |
Junie Junie | 1 | SWE-rebench — #6 · 61.8 |
Kimi 2.6 | 1 | OSWorld 2.0 — #23 · 4.6 |
Kimi K2.6 | 1 | OSWorld (OSWorld-Verified) — #14 · 73.1 |
kimi-k2-5-thinking | 1 | Kagi LLM Benchmarking Project — #10 · 78.5 |
kimi-k2-6 | 1 | MCPMark — #7 · 72.83 |
kimi-k2-7-code | 1 | MCPMark — #5 · 81.89 |
kimi-k2-thinking | 1 | MultiChallenge — #24 · 55.42 |
Kimi-K3 max | 1 | SWE-Bench Pro — #6 · 88.20 |
kimi-k3-max | 1 | MCPMark — #1 · 96.06 |
kimi-k3.1-pro | 1 | LiveBench — #15 · 68.7 |
kimi-k3.1-pro-together_max | 1 | Epoch AI Benchmarking Hub — #23 · 78.2 |
Kokoro v1.0 | 1 | TTS Arena V2 — #25 · 1470 |
KV-Ground-8B + Qwen3.5-27B Consistency Router | 1 | ScreenSpot-Pro — #5 · 80.90 |
KV-Ground-GuiOwl1.5-0315-8B | 1 | ScreenSpot-Pro — #16 · 73.18 |
KV-Ground-GuiOwl1.5-0315-8B-ZoomIn | 1 | ScreenSpot-Pro — #6 · 80.52 |
KV-Ground-GuiOwl1.5-4B-0228-ZoomIn | 1 | ScreenSpot-Pro — #10 · 76.41 |
KV-Ground-Qwen3VL-4B-ZoomIn | 1 | ScreenSpot-Pro — #24 · 70.27 |
Lightning v3.1 Pro | 1 | TTS Arena V2 — #13 · 1544 |
Ling 3.1 Flash 1019 | 1 | GDPval-AA — #25 · 1655 |
Linq-AI-Research/Linq-Embed-Mistral | 1 | PL-MTEB (Polish MTEB) — #6 · 71.10 |
Logics-Parsing-v2 | 1 | OmniDocBench — #14 · 93.33 |
MAI-UI-8B | 1 | ScreenSpot-Pro — #19 · 71.92 |
manu/bge-m3-custom-fr | 1 | PL-MTEB (Polish MTEB) — #15 · 64.74 |
Manus 1.0 | 1 | Remote Labor Index (RLI) — #16 · 2.50 |
Manus 1.5 | 1 | Remote Labor Index (RLI) — #17 · 2.50 |
Manus_1.6 | 1 | Remote Labor Index (RLI) — #14 · 2.92 |
meta-llama/Llama-3.3-70B-Instruct-Turbo | 1 | Vectara Hallucination Leaderboard — #5 · 4.1 % |
meta-models/Muse-Glimmer-30B | 1 | OCR: handwritten notes, boards and textbook pages — Describing illustrations (charts, diagrams, maps), 2026- ours |
meta-models/Muse-Glimmer-30B reasoning | 1 | CPTU-Bench — #1 · 4.42 |
meta/muse-spark-1.3-contributor-20260902 | 1 | OpenRouter Rankings — #16 · 2964.9 |
microsoft/azure-speech-09-2026 | 1 | Open ASR Leaderboard — #4 · 5.5 |
microsoft/harrier-oss-v1-0.6b | 1 | PL-MTEB (Polish MTEB) — #7 · 70.47 |
microsoft/harrier-oss-v1-270m | 1 | PL-MTEB (Polish MTEB) — #16 · 66.93 |
microsoft/Phi-4 | 1 | Vectara Hallucination Leaderboard — #4 · 3.7 % |
microsoft/Phi-4-multimodal-instruct | 1 | OCR: handwritten notes, boards and textbook pages — Cheaper models, 2026-10-06 (the same 21 readings) ours |
MiMo V2.6 Flash | 1 | Vals AI — #25 · 54.83 |
MiMo V2.6 Pro | 1 | Vals AI — #21 · 57.81 |
mimo-v2.5-pro | 1 | EQ-Bench — #19 · 1197.7 |
MiMo-V2.6-Pro and MiMo-V2.6-Flash | 1 | ASR: Polish speech recognition — Degraded recordings, 2026-09-29 (single models, no text ours |
MinerU-2.5 | 1 | OmniDocBench — #17 · 93.04 |
MinerU2.5-Pro | 1 | OmniDocBench — #6 · 95.75 |
MiniCPM-OCR 1.0 | 1 | OmniDocBench — #4 · 96.44 |
MiniMax M3.1 Flash | 1 | OSWorld 2.0 — #10 · 26.85 |
Minimax M3p1 Flash xhigh | 1 | SWE-Bench Pro — #10 · 84.31 |
Minimax MiniMax M3 | 1 | SWE-rebench — #11 · 47.2 |
MiniMax Speech 02 HD | 1 | TTS Arena V2 — #19 · 1503 |
MiniMax Speech 02 Turbo | 1 | TTS Arena V2 — #23 · 1483 |
MiniMax Speech 2.6 HD | 1 | TTS Arena V2 — #21 · 1504 |
MiniMax Speech 2.6 Turbo | 1 | TTS Arena V2 — #17 · 1525 |
MiniMax Speech 2.8 HD | 1 | TTS Arena V2 — #11 · 1545 |
MiniMax Speech 2.8 Turbo | 1 | TTS Arena V2 — #22 · 1502 |
minimax-m3.1-flash | 1 | LiveBench — #21 · 63.7 |
minimax/minimax-m3.1-flash-20261019 | 1 | OpenRouter Rankings — #11 · 3350.9 |
mistralai/mistral-large-2411 | 1 | Vectara Hallucination Leaderboard — #9 · 4.5 % |
mistralai/Mistral-Large-Instruct-2407 | 1 | CPTU-Bench — #24 · 3.93 |
mistralai/Mistral-Large-Instruct-2411 | 1 | CPTU-Bench — #21 · 4.00 |
mistralai/mistral-small-2501 | 1 | Vectara Hallucination Leaderboard — #13 · 5.1 % |
mistralai/Mistral-Small-3.1-24B-Instruct-2503 | 1 | OCR: handwritten notes, boards and textbook pages — Cheaper models, 2026-10-06 (the same 21 readings) ours |
mistralai/Voxtral-Mini-3B-2507 | 1 | ASR: Polish speech recognition — Test of 2026-09-22: clean speech synthesis, Polish + Eng ours |
mistralai/Voxtral-Small-24B-2507 | 1 | ASR: Polish speech recognition — Test of 2026-09-22: clean speech synthesis, Polish + Eng ours |
modulate/multilingual | 1 | Open ASR Leaderboard — #9 · 5.96 |
MonkeyOCR V2 | 1 | OmniDocBench — #20 · 91.80 |
moonshotai/Kimi-K2-Instruct-0905 | 1 | CPTU-Bench — #22 · 3.98 |
moonshotai/kimi-k3.1-pro-20261023 | 1 | OpenRouter Rankings — #12 · 3241.9 |
multilingual-e5-large | 1 | Embeddings: retrieval accuracy in Polish — Data ours |
Musci-research/Musci-ASR-qwen-1019-hf | 1 | Open ASR Leaderboard — #5 · 5.7 |
Muse Ember 1.0 | 1 | SWE Atlas — #20 · 55.99 |
Muse Spark 1.3 Max | 1 | Design Arena — #12 · 1279 |
Muse Spark Open xhigh | 1 | SWE-Bench Pro — #13 · 82.35 |
Muse Spark Thinking | 1 | MMMU / MMMU-Pro — #6 · 80.4 |
Muse Wonder 1.0 | 1 | SWE Atlas — #9 · 64.83 |
Muse Wonder 1.0 xhigh | 1 | SWE-Bench Pro — #5 · 90.20 |
muse-ember-1.0 | 1 | Scale Fortress — #8 · 8 |
muse-spark-1.1 | 1 | EQ-Bench — #11 · 1244.4 |
muse-spark-open | 1 | Scale Fortress — #2 · 2 |
muse-wonder-1.0 | 1 | LiveBench — #12 · 70.7 |
muse-wonder-1.0_max | 1 | Epoch AI Benchmarking Hub — #18 · 82.1 |
MuseSpark 1.2 | 1 | OSWorld 2.0 — #17 · 15.84 |
MuseSpark 1.3 | 1 | OSWorld 2.0 — #2 · 39.25 |
MVP_Qwen3VL-32B | 1 | ScreenSpot-Pro — #12 · 74.15 |
Nemotron 3.5 ASR Multilingual, language specified | 1 | ASR: Polish speech recognition — Live speech (streaming), 2026-11-05/06 ours |
Nemotron-Diarization | 1 | ASR: Polish speech recognition — Earlier measurement, 2026-09-23 (one deliberately degrad ours |
nvidia/Nemotron-3-Diarization | 1 | Diarization: who is speaking — Comparison with external providers, 2026-11-06 ours |
nvidia/nemotron-3-ultra-550b-a55b-20260604 | 1 | OpenRouter Rankings — #13 · 3198.3 |
nvidia/Nemotron-3.5-ASR-Streaming-Multilingual-0.6b | 1 | ASR: Polish speech recognition — Test of 2026-09-22: clean speech synthesis, Polish + Eng ours |
nvidia/nemotron-3.5-super-vl-20261015 | 1 | OpenRouter Rankings — #15 · 2969.1 |
nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B | 1 | Open ASR Leaderboard — #13 · 6.13 |
nvidia/parakeet-tdt-0.6b-v2 | 1 | Open ASR Leaderboard — #25 · 6.91 |
o1_inspect | 1 | METR Time Horizons — #15 · 38.8 min (21.2-65.0) |
o1_preview | 1 | METR Time Horizons — #17 · 20.3 min (11.7-33.4) |
o3_inspect | 1 | METR Time Horizons — #11 · 119.7 min (74.6-190.9) |
o3-2025-04-16-reasoning-high | 1 | MultiChallenge — #22 · 56.62 |
o3-pro | 1 | Kagi LLM Benchmarking Project — #24 · 72.1 |
o3-pro-2025-06-10-reasoning-high | 1 | MultiChallenge — #14 · 62.40 |
omarelshehy/arabic-english-sts-matryoshka | 1 | PL-MTEB (Polish MTEB) — #20 · 65.07 |
OpenAI Codex | 1 | SWE-rebench — #8 · 58.0 |
OpenAI gpt-4o-transcribe-diarize | 1 | Diarization: who is speaking — Comparison with external providers, 2026-11-06 ours |
OpenAI GPT-5.6 Luna [medium] | 1 | SWE-rebench — #13 · 43.6 |
OpenAI GPT-5.6 Sol [medium] | 1 | SWE-rebench — #5 · 62.3 |
OpenAI gpt-live-transcribe | 1 | ASR: Polish speech recognition — Live speech (streaming), 2026-11-05/06 ours |
OpenAI gpt-realtime-whisper | 1 | ASR: Polish speech recognition — Live speech (streaming), 2026-11-05/06 ours |
openai/gpt-4.1-2025-04-14 | 1 | Vectara Hallucination Leaderboard — #22 · 5.6 % |
openai/gpt-5.4-2026-03-05#none | 1 | EuroEval — #25 · 1.76 ± 0.09 |
openai/gpt-5.4-mini-2026-03-17 | 1 | Vectara Hallucination Leaderboard — #21 · 5.5 % |
openai/gpt-5.4-mini-2026-03-17#high | 1 | EuroEval — #14 · 1.61 ± 0.05 |
openai/gpt-5.4-mini-2026-03-17#low | 1 | EuroEval — #24 · 1.70 ± 0.06 |
openai/gpt-5.4-mini-2026-03-17#medium | 1 | EuroEval — #22 · 1.64 ± 0.06 |
openai/gpt-5.4-nano-2026-03-17 | 1 | Vectara Hallucination Leaderboard — #2 · 3.1 % |
openai/gpt-5.5 [high] | 1 | Context Arena (MRCR) — #12 · 79.3 |
openai/gpt-5.5 [medium] | 1 | Context Arena (MRCR) — #7 · 83.0 |
openai/gpt-5.5 [xhigh] | 1 | Context Arena (MRCR) — #6 · 85.8 |
openai/gpt-5.6-sol [max] | 1 | Context Arena (MRCR) — #1 · 92.4 |
openai/gpt-5.6-terra [max] | 1 | Context Arena (MRCR) — #11 · 80.2 |
openai/gpt-6-luna-20260922 | 1 | OpenRouter Rankings — #5 · 8081.3 |
openai/gpt-6.1-sol-20260929 | 1 | OpenRouter Rankings — #10 · 3711.0 |
openai/gpt-oss-120b | 1 | Language models: classification, summaries, chat with tools — Lesson subject classification, 2026-09-22 ours |
openai/gpt-oss-20b | 1 | Language models: classification, summaries, chat with tools — Lesson subject classification, 2026-09-22 ours |
openai/gpt-transcribe | 1 | Open ASR Leaderboard — #22 · 6.8 |
OpenAPA w/ gemini-3.1-pro | 1 | OSWorld (OSWorld-Verified) — #9 · 78.3 |
OpenAudio S2 | 1 | TTS Arena V2 — #12 · 1539 |
OpenDoc-0.1B | 1 | OmniDocBench — #22 · 90.67 |
Opus 4.8 | 1 | Remote Labor Index (RLI) — #8 · 8.33 |
Opus 5 | 1 | Terminal-Bench — #11 · 53.3% ± 3.1% |
Opus 5 xhigh | 1 | SWE-Bench Pro — #1 · 98.00 |
OrdalieTech/Solon-embeddings-large-0.1 | 1 | PL-MTEB (Polish MTEB) — #17 · 63.49 |
OS-Symphony w/ GPT-5 | 1 | OSWorld (OSWorld-Verified) — #20 · 65.8 |
OtterAI/speech-v3.1 | 1 | Open ASR Leaderboard — #2 · 5.21 |
Ovis2.6-30B-A3B | 1 | OmniDocBench — #13 · 93.70 |
OvisOCR2 | 1 | OmniDocBench — #3 · 96.47 |
OXR-1.0 | 1 | OmniDocBench — #1 · 97.03 |
p99lab/audrey-0.7 | 1 | Open ASR Leaderboard — #6 · 5.7 |
PaddleOCR-VL | 1 | OmniDocBench — #9 · 94.18 |
PaddleOCR-VL-1.5 | 1 | OmniDocBench — #8 · 94.93 |
PaddleOCR-VL-1.6 | 1 | OmniDocBench — #5 · 96.34 |
Papla P1 | 1 | TTS Arena V2 — #8 · 1552 |
Parmesan | 1 | TTS Arena V2 — #24 · 1481 |
Pointer Agent w/ Opus 4.7 | 1 | OSWorld (OSWorld-Verified) — #3 · 83.6 |
Pointer Agent w/ Sonnet 4.6 | 1 | OSWorld (OSWorld-Verified) — #6 · 81.5 |
pyahocorasick | 1 | Scanner performance: database loading, re-scans, TLSH in SQL, Aho-Corasick — External libraries ours |
Qianfan-OCR | 1 | OmniDocBench — #12 · 93.90 |
Qwen 3.7 Plus | 1 | OSWorld (OSWorld-Verified) — #13 · 73.3 |
Qwen 3.7-Plus | 1 | OSWorld 2.0 — #24 · 2.8 |
Qwen 3.8 Max | 1 | SWE Atlas — #10 · 64.40 |
Qwen 3.8 Max 0902 | 1 | Vals AI — #18 · 59.05 |
Qwen 3.8-Max | 1 | F5 Labs CASI & ARS Leaderboard — #6 · 92.31 |
Qwen 4 Max | 1 | Vals AI — #11 · 63.20 |
Qwen Qwen3.5-35B-A3B | 1 | SWE-rebench — #17 · 17.1 |
Qwen Qwen3.6-27B | 1 | SWE-rebench — #15 · 31.2 |
Qwen Qwen3.6-35B-A3B | 1 | SWE-rebench — #16 · 24.7 |
qwen-3-5-397b | 1 | Kagi LLM Benchmarking Project — #16 · 73.7 |
Qwen/Qwen2.5-72B-Instruct | 1 | CPTU-Bench — #23 · 3.95 |
qwen/qwen3-14b | 1 | Vectara Hallucination Leaderboard — #18 · 5.4 % |
Qwen/Qwen3-235B-A22B-2507 | 1 | Language models: classification, summaries, chat with tools — Lesson summary (description), 2026-09-22 ours |
qwen/qwen3-4b | 1 | Vectara Hallucination Leaderboard — #23 · 5.7 % |
qwen/qwen3-8b | 1 | Vectara Hallucination Leaderboard — #10 · 4.8 % |
Qwen/Qwen3-ASR-1.7B-hf | 1 | Open ASR Leaderboard — #14 · 6.15 |
Qwen/Qwen3-ASR-1.7B, raw audio | 1 | ASR: Polish speech recognition — Earlier measurement, 2026-09-23 (one deliberately degrad ours |
Qwen/Qwen3-Next-80B-A3B | 1 | Language models: classification, summaries, chat with tools — Lesson subject classification, 2026-09-22 ours |
Qwen/Qwen3-VL-30B-A3B-Instruct | 1 | OCR: handwritten notes, boards and textbook pages — Test from 2026-09-22: 4 photos, 19 models ours |
Qwen/Qwen3.5-0.8B | 1 | OCR: handwritten notes, boards and textbook pages — Cheaper models, 2026-10-06 (the same 21 readings) ours |
Qwen/Qwen3.5-122B-A10B non-reasoning | 1 | CPTU-Bench — #10 · 4.26 |
Qwen/Qwen3.5-122B-A10B reasoning | 1 | CPTU-Bench — #3 · 4.36 |
Qwen/Qwen3.5-27B non-thinking | 1 | CPTU-Bench — #9 · 4.27 |
Qwen/Qwen3.5-27B thinking | 1 | CPTU-Bench — #4 · 4.34 |
Qwen/Qwen3.5-2B | 1 | OCR: handwritten notes, boards and textbook pages — Cheaper models, 2026-10-06 (the same 21 readings) ours |
Qwen/Qwen3.5-35B-A3B non-thinking | 1 | CPTU-Bench — #14 · 4.18 |
Qwen/Qwen3.5-35B-A3B thinking | 1 | CPTU-Bench — #12 · 4.22 |
Qwen/Qwen3.5-35B-A3B-FP8 | 1 | EuroEval — #18 · 1.66 ± 0.09 |
Qwen/Qwen3.5-4B | 1 | OCR: handwritten notes, boards and textbook pages — Cheaper models, 2026-10-06 (the same 21 readings) ours |
Qwen/Qwen3.6-27B | 1 | OCR: handwritten notes, boards and textbook pages — Describing illustrations (charts, diagrams, maps), 2026- ours |
Qwen/Qwen3.6-27B non-reasoning | 1 | CPTU-Bench — #7 · 4.29 |
Qwen/Qwen3.6-27B reasoning | 1 | CPTU-Bench — #2 · 4.41 |
Qwen/Qwen3.6-27B-FP8 | 1 | EuroEval — #11 · 1.60 ± 0.07 |
qwen/qwen3.6-max-preview [enabled] | 1 | Context Arena (MRCR) — #23 · 71.6 |
qwen/qwen3.8-27b [low] | 1 | Context Arena (MRCR) — #9 · 81.8 |
qwen/qwen3.8-27b [medium] | 1 | Context Arena (MRCR) — #13 · 78.9 |
qwen/qwen3.8-27b [xhigh] | 1 | Context Arena (MRCR) — #18 · 74.3 |
qwen/qwen3.8-max [low] | 1 | Context Arena (MRCR) — #10 · 81.5 |
qwen/qwen3.8-max [medium] | 1 | Context Arena (MRCR) — #5 · 87.3 |
qwen/qwen3.8-max [xhigh] | 1 | Context Arena (MRCR) — #2 · 92.3 |
Qwen3-ASR | 1 | ASR: Polish speech recognition — Live speech (streaming), 2026-11-05/06 ours |
Qwen3-ASR-1.7B | 1 | ASR: Polish speech recognition — Language lessons: Polish and a foreign language in one s ours |
qwen3-max | 1 | Kagi LLM Benchmarking Project — #22 · 72.5 |
Qwen3-VL 235B-A22B | 1 | MMMU / MMMU-Pro — #22 · 68.1 |
Qwen3-VL-235B | 1 | OmniDocBench — #25 · 89.78 |
Qwen3.8-Flash | 1 | Language models: classification, summaries, chat with tools — Chat with tools, 2026-09-22 ours |
qwen3.8-max_xhigh | 1 | Epoch AI Benchmarking Hub — #25 · 74.7 |
Qwen3.8-Max-0902 | 1 | Terminal-Bench — #25 · 27.0% ± 3.9% |
Qwen4 Max | 1 | GDPval-AA — #7 · 1809 |
reson8/rumble-2 | 1 | Open ASR Leaderboard — #8 · 5.95 |
salesforce-sovereign-intelligence/SOV-Transcribe-1 | 1 | Open ASR Leaderboard — #10 · 5.96 |
Salesforce/SFR-Embedding-2_R | 1 | PL-MTEB (Polish MTEB) — #13 · 68.88 |
Salesforce/SFR-Embedding-Mistral | 1 | PL-MTEB (Polish MTEB) — #9 · 69.31 |
sdadas/mmlw-e5-small | 1 | PL-MTEB (Polish MTEB) — #21 · 63.41 |
Seed 1.5-VL Thinking | 1 | MMMU / MMMU-Pro — #24 · 67.6 |
Seed 1.6-Thinking | 1 | MMMU / MMMU-Pro — #25 · 66.4 |
smallestai/pulse | 1 | Open ASR Leaderboard — #16 · 6.57 |
Snowflake/snowflake-arctic-embed-l-v2.0 | 1 | PL-MTEB (Polish MTEB) — #24 · 61.31 |
snowflake/snowflake-arctic-instruct | 1 | Vectara Hallucination Leaderboard — #7 · 4.3 % |
Soniox stt-rt-v3 | 1 | ASR: Polish speech recognition — Live speech (streaming), 2026-11-05/06 ours |
Soniox stt-rt-v3, PCM | 1 | ASR: Polish speech recognition — Live speech (streaming), 2026-11-05/06 ours |
Soniox stt-rt-v6 | 1 | ASR: Polish speech recognition — Live speech (streaming), 2026-11-05/06 ours |
Soniox stt-rt-v6, PCM | 1 | ASR: Polish speech recognition — Live speech (streaming), 2026-11-05/06 ours |
Sonnet 5 xhigh | 1 | SWE-Bench Pro — #7 · 88.20 |
Sonnet 5.5 xhigh | 1 | SWE-Bench Pro — #14 · 70.59 |
sophea/asr-k1 | 1 | Open ASR Leaderboard — #12 · 6.11 |
sprag/symphony | 1 | Open ASR Leaderboard — #23 · 6.85 |
star-june-2026 | 1 | TTS Arena V2 — #2 · 1633 |
stepfun/step-5-preview-20261008 | 1 | OpenRouter Rankings — #7 · 6360.0 |
TeleOCR | 1 | OmniDocBench — #2 · 96.91 |
tencent/hy4-20261104 | 1 | OpenRouter Rankings — #2 · 16042.2 |
TheStageAI/thewhisper-large-v3-turbo | 1 | Open ASR Leaderboard — #20 · 6.74 |
thinkingmachines/Inkling | 1 | OCR: handwritten notes, boards and textbook pages — Describing illustrations (charts, diagrams, maps), 2026- ours |
typesafe/jev-1.13-20260917 | 1 | OpenRouter Rankings — #17 · 2868.8 |
UI-Venus-1-5-30B-A3B | 1 | ScreenSpot-Pro — #25 · 69.64 |
Unlimited-OCR | 1 | OmniDocBench — #10 · 94.00 |
VLAA-GUI w/ Opus 4.5 | 1 | OSWorld (OSWorld-Verified) — #10 · 76.3 |
Voxtral Mini and Small | 1 | ASR: Polish speech recognition — Earlier measurement, 2026-09-23 (one deliberately degrad ours |
whisper-large-v3 | 1 | ASR: Polish speech recognition — Language lessons: Polish and a foreign language in one s ours |
Wonder | 1 | Vals AI — #9 · 64.88 |
Wonder Full | 1 | Vals AI — #7 · 66.79 |
xai-org/grok-3 | 1 | Vectara Hallucination Leaderboard — #24 · 5.8 % |
xai/grok-4-1-fast-reasoning | 1 | EuroEval — #21 · 1.69 ± 0.07 |
XekRung-1.5-27B-Preview | 1 | CyberGym — #2 · 88.9% |
xiaomi/mimo-v2.6-flash-20260921 | 1 | OpenRouter Rankings — #8 · 4164.6 |
XiaomiMiMo MiMo V2.5 Pro | 1 | SWE-rebench — #12 · 46.5 |
yoob/yoob-asr-pro-v1 | 1 | Open ASR Leaderboard — #3 · 5.33 |
Youtu-Parsing | 1 | OmniDocBench — #11 · 93.74 |
z-ai/glm-5.3 [high] | 1 | Context Arena (MRCR) — #24 · 69.3 |
z-ai/glm-5.3-20260816 | 1 | OpenRouter Rankings — #9 · 3738.6 |
z-ai/glm-5.3-flash-20260826 | 1 | OpenRouter Rankings — #4 · 11806.2 |
Z.ai GLM-5.2 [high] | 1 | SWE-rebench — #4 · 62.9 |
zoom/scribe_v2.1_pro | 1 | Open ASR Leaderboard — #1 · 4.87 |