ASR: Polish speech recognition
asr-polish-01.csv— 1. Test of 2026-09-22: clean speech synthesis, Polish + English in one recording (9 rows)asr-polish-02.csv— 2. Degraded recordings, 2026-09-29 (single models, no text correction) (6 rows)asr-polish-03.csv— Earlier measurement, 2026-09-23 (one deliberately degraded recording) (5 rows)asr-polish-04.csv— 3. Language lessons: Polish and a foreign language in one sentence, 2026-09-29/30 (6 rows)asr-polish-05.csv— 3. Language lessons: Polish and a foreign language in one sentence, 2026-09-29/30 (3 rows)asr-polish-06.csv— 4. External providers on the same recordings, 2026-11-06 (5 rows)asr-polish-07.csv— 5. Live speech (streaming), 2026-11-05/06 (6 rows)asr-polish-08.csv— 5. Live speech (streaming), 2026-11-05/06 (4 rows)
Diarization: who is speaking
diarization-01.csv— 1. nvidia/Nemotron-3-Diarization (DeepInfra), 2026-09-30 (8 rows)diarization-02.csv— 2. Comparison with external providers, 2026-11-06 (3 rows)
Omni models: video and audio input
omni-video-audio-01.csv— 1. Does the model hear the audio from a video file? (6 rows)omni-video-audio-02.csv— 2. How many frames does the model see? (6 rows)
OCR: handwritten notes, boards and textbook pages
ocr-handwriting-boards-textbooks-01.csv— 1. Test from 2026-09-22: 4 photos, 19 models (22 rows)ocr-handwriting-boards-textbooks-02.csv— 2. "Hard" test from 2026-09-29/30: 14 photos, handwriting and small numbers (9 rows)ocr-handwriting-boards-textbooks-03.csv— Three hard photos of Polish notes, 2026-10-04 (5 rows)ocr-handwriting-boards-textbooks-04.csv— 3. Textbook pages (print, scans and photos), 2026-10-05 (6 rows)ocr-handwriting-boards-textbooks-05.csv— Cheaper models, 2026-10-06 (the same 21 readings) (9 rows)ocr-handwriting-boards-textbooks-06.csv— 305-page scan and loops on leader dots, 2026-10-08 (3 rows)ocr-handwriting-boards-textbooks-07.csv— 4. Describing illustrations (charts, diagrams, maps), 2026-10-06 (10 rows)
OCR: exam-sheet photos with handwritten answers
ocr-exam-sheets-01.csv— 1. Reading a single handwritten word (Gemini 3.1 Pro) (4 rows)ocr-exam-sheets-02.csv— 3. Blank exam-sheet page (4 rows)ocr-exam-sheets-03.csv— 4. Identifying the exam sheet from photos (2 rows)ocr-exam-sheets-04.csv— 5. Cost and time of a single call (8 rows)
Language models: classification, summaries, chat with tools
llm-classification-summaries-chat-01.csv— 1. Lesson subject classification, 2026-09-22 (10 rows)llm-classification-summaries-chat-02.csv— 2. Lesson summary (description), 2026-09-22 (8 rows)llm-classification-summaries-chat-03.csv— 3. "Lossless" summaries: how much lesson knowledge survived, 2026-09-29 (6 rows)llm-classification-summaries-chat-04.csv— DeepSeek-V4-Pro vs V4.1-Flash on lesson description, 2026-11-05 (2 rows)llm-classification-summaries-chat-05.csv— 4. Chat with tools, 2026-09-22 (13 rows)llm-classification-summaries-chat-06.csv— Whiteboard photo in chat (1 photo, 1024×1024) (5 rows)
Language models: short answers, grading text, time and cost
llm-short-answers-grading-cost-01.csv— 1. DeepSeek-V4-Flash: undated name vs dated name (2026-10-13) (5 rows)
Reasoning level (reasoning_effort): what models actually do
reasoning-effort-01.csv— 1. The minimal value on different models (2026-09-03) (5 rows)reasoning-effort-02.csv— 2. Cost of reasoning (3 rows)reasoning-effort-03.csv— 3. Reasoning and the output token limit (2 rows)
Embeddings: retrieval accuracy in Polish
embeddings-polish-01.csv— Data (6 rows)
Platforms: availability, latency and limits
platforms-availability-latency-limits-01.csv— DeepInfra (12 rows)
Platforms: errors, reasoning billing and prices
platforms-errors-billing-prices-01.csv— DeepInfra (10 rows)platforms-errors-billing-prices-02.csv— Prices from /v1/openai/models (2026-10-16) (14 rows)
Malware detection: AMTSO, EICAR containers, Objective-See hashes, Mac malware families
malware-detection-safe-01.csv(5 rows)malware-detection-safe-02.csv— 1. Official test files: AMTSO Security Features Check + eicar.org (11 rows)malware-detection-safe-03.csv— 2. EICAR in 57 container formats (57 rows)malware-detection-safe-04.csv— Clean twins (near-miss files) (57 rows)malware-detection-safe-05.csv— 3. SHA-256 hashes from Objective-See reports 2019–2024 (33 rows)malware-detection-safe-06.csv— 4. Mac malware families 2019–2025 with a family-named rule (8 rows)malware-detection-safe-07.csv— 4. Mac malware families 2019–2025 with a family-named rule (83 rows)
Malware detection: test samples vs held-out clean files
malware-detection-held-out-01.csv— Runs (3 rows)malware-detection-held-out-02.csv— Run 20261103 (3 rows)malware-detection-held-out-03.csv— Run 20261103-fix (3 rows)malware-detection-held-out-04.csv— Run 20261109-speed (3 rows)malware-detection-held-out-05.csv— 2. Held-out clean files by category (INFO+ findings) (9 rows)
Malware scanning: static-heuristic false alarms before and after fixes
malware-static-false-positives-01.csv(4 rows)malware-static-false-positives-02.csv— 1. Clean files: summary (4 rows)malware-static-false-positives-03.csv— 2. Clean files by category (13 rows)malware-static-false-positives-04.csv— before (6 false alarms, all SUSPICIOUS) (2 rows)malware-static-false-positives-05.csv— after (2 false alarms) (2 rows)malware-static-false-positives-06.csv— 4. Fixtures: detection on test samples (6 rows)malware-static-false-positives-07.csv— By sample group, heur mode (INFO+ / SUSPICIOUS+ / MALICIOUS) (7 rows)malware-static-false-positives-08.csv— By sample group, yara mode (INFO+ / SUSPICIOUS+ / MALICIOUS) (7 rows)malware-static-false-positives-09.csv— Neutral archives (48 files, no expected verdict) (6 rows)
ML classifiers on TLSH fingerprints of Mach-O files
ml-tlsh-macho-classifiers-01.csv— Dataset (4 rows)ml-tlsh-macho-classifiers-02.csv— 1. Main results (out-of-fold, threshold at 0.1% FP) (5 rows)ml-tlsh-macho-classifiers-03.csv— 2. Per-fold thresholds (summed over the 5 test folds) (5 rows)ml-tlsh-macho-classifiers-04.csv— 3. Baseline: TLSH nearest-neighbour rule (no model) (2 rows)ml-tlsh-macho-classifiers-05.csv— 4. Additional recall figures recorded by the script (5 rows)
Scanner performance: database loading, re-scans, TLSH in SQL, Aho-Corasick
scanner-performance-01.csv— Ours (own engine) (3 rows)scanner-performance-02.csv— External libraries (3 rows)