Platform: DeepInfra. Measurement: 2026-09-22. Description of metrics and caveats: project overview.
Data
- 12 Polish note documents, including deliberately misleading, off-topic ones: „równania reakcji” ("reaction equations"), „równania Maxwella” ("Maxwell's equations"), „układy równań liniowych” ("systems of linear equations"), „kwadratura koła” ("squaring the circle").
- Graded test for the query „równania kwadratowe” ("quadratic equations"): a point score (0–1 scale; the exact definition is not given in the source) and a margin = lowest score of a relevant note minus highest score of an irrelevant note (a negative value = an irrelevant note outranked a relevant one).
- 10-query test: point score 0–1 (1.00 = full accuracy); for bge-m3 the source gives 0.95 and 9 of 10 queries with a relevant note in first place.
- Small sample: 12 documents, 10 queries, one call per configuration.
- "With instruction" = the query is prefixed with a short task instruction (recommended for the Qwen3-Embedding family; documents without a prefix).
| BAAI/bge-m3 | 1024 | 1.5–1.6 s | 0.632 / −0.063 | 0.95 (9/10 in first place) |
|---|---|---|---|---|
| Qwen/Qwen3-Embedding-0.6B | 1024 | 1.5–3 s (one cold 26 s) | 0.953 / +0.179 without instruction; 0.935 / +0.187 with instruction | 0.925 without instruction; 1.00 with instruction |
| Qwen/Qwen3-Embedding-4B | 2560 | 1.5–5.8 s | 0.953 / +0.078; with instruction margin +0.127; at 1024 dimensions 0.888 / +0.104 | 1.00 |
| Qwen/Qwen3-Embedding-8B | 4096 | 10–31 s | 0.953 / +0.048; with instruction margin +0.097 | 1.00 |
| multilingual-e5-large | 1024 (512-token limit) | 1.6–29 s | 0.751 / −0.030 (similarities clustered in the 0.73–0.88 range) | 1.00 |
| embeddinggemma-300m | 768 | 0.9–1.6 s | 0.817 / −0.036 | 1.00 |
Observations:
- bge-m3 ranked the note with Viète's formulas below three unrelated notes that only share the word „równania” ("equations") or „kwadrat” ("square").
- Qwen3-Embedding-0.6B had the best margin (+0.19) on the graded test and 10/10 with the instruction.
- Qwen3-Embedding-4B honors the dimension-reduction parameter (down to 1024).
- Vectors from different models are not interchangeable.