benchmarks.site

Public leaderboard · Polish language

CPTU-Bench

Checks how well AI models really understand Polish: reading between the lines (sarcasm, hidden meaning, idioms) and answering tricky questions with logic traps, ambiguity or absurd premises. It shows whether a model grasps Polish beyond the literal words.

Leaderboard

Top 3 right now

  1. 1stmeta-models/Muse-Glimmer-30B reasoning (API, FP8)4.42
  2. 2ndQwen/Qwen3.6-27B reasoning (API, FP8)4.41
  3. 3rdQwen/Qwen3.5-122B-A10B reasoning (API, FP8)4.36

What it measures

Understanding of Polish text: implicatures (sentiment, implied meaning, phraseology) and tricky questions (logic puzzles, ambiguity, absurdity), scored on a 0–5 scale.

How to read the numbers

Average score on a 0–5 scale across four parts (sentiment, language understanding, phraseology, tricky questions) — higher is better.

Results

25 rows · higher is better (Average)CSV
1meta-models/Muse-Glimmer-30B reasoning (API, FP8)30B4.424.524.394.234.52
2Qwen/Qwen3.6-27B reasoning (API, FP8)27B4.414.584.214.214.64
3Qwen/Qwen3.5-122B-A10B reasoning (API, FP8)122B4.364.364.184.194.72
4Qwen/Qwen3.5-27B thinking (API)27B4.344.424.214.114.61
5google/gemma-4-31B-it non-reasoning (API, FP8)31B4.324.584.464.074.19
6google/gemma-4-31B-it reasoning (API, FP8)31B4.314.584.383.774.52
7Qwen/Qwen3.6-27B non-reasoning (API, FP8)27B4.294.334.084.404.36
8gemini-2.0-flash-0014.294.524.324.343.99
9Qwen/Qwen3.5-27B non-thinking (API)27B4.274.294.174.204.43
10Qwen/Qwen3.5-122B-A10B non-reasoning (API, FP8)122B4.264.334.243.944.53
11google/gemma-4-26B-A4B-it non-reasoning (API, FP8)26B4.234.394.004.214.30
12Qwen/Qwen3.5-35B-A3B thinking (API)35B4.224.103.944.154.70
13google/gemma-4-26B-A4B-it reasoning (API, FP8)26B4.214.394.083.914.47
14Qwen/Qwen3.5-35B-A3B non-thinking (API)35B4.184.234.054.234.19
15deepseek-ai/DeepSeek-V3.2 (API)685B4.144.464.363.544.20
16deepseek-ai/DeepSeek-R1 (API)685B4.144.494.343.604.12
17gemini-2.0-flash-lite-0014.094.234.054.243.85
18deepseek-ai/DeepSeek-V3-0324 (API)685B4.034.364.203.544.02
19deepseek-ai/DeepSeek-V3.1 (API)685B4.034.424.333.483.87
20deepseek-ai/DeepSeek-V3 (API)685B4.024.364.223.523.99
21mistralai/Mistral-Large-Instruct-2411123B4.004.333.983.993.72
22moonshotai/Kimi-K2-Instruct-0905 (API)1000B3.984.394.183.433.93
23Qwen/Qwen2.5-72B-Instruct72.7B3.954.083.973.933.81
24mistralai/Mistral-Large-Instruct-2407123B3.934.234.003.863.65
25meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8 (API)402B3.934.394.113.483.76

Note: Per-category scores copied from the Space's data.json; 'Average' is the plain mean of the four categories, computed exactly as the leaderboard app does (app.py) and shown to 2 decimals like the leaderboard. as_of is the Space's last commit date. License is the Space repository's declared license (MIT). 102 models in total; top 25 shown.

Source: SpeakLeash · huggingface.co · results as of 2026-08-11 · retrieved 2026-11-17 · license: MIT

Markdown · More Polish language