Metric: DER (diarization error rate; lower = better), computed on 10 ms frames, including speech by several people at once, consistent with pyannote.metrics to within 0.1–0.2 points. Reference point: the trivial answer "all speech belongs to one speaker" gives a DER of 13–18% on this set (the teacher speaks 82–87% of the time), so a DER of 6–14% looks good mainly because one speaker dominates. Metric definitions and caveats: overview.
1. nvidia/Nemotron-3-Diarization (DeepInfra), 2026-09-30
- The model returns only speaker turns (anonymous labels
speaker_N, up to 8 speakers, consistent within a file, 10 ms resolution, no confidence and no text); turns of different speakers overlap when several people speak at once. The output is deterministic (the same on every call on the same file). - Data: 4 synthetic lessons, 7.1 min in total: history, biology, mathematics, English; 3–4 students in each, 4–5 places with two people speaking at once. In three lessons the students are a pitch-shifted (higher) synthetic Zosia, so the voices differ mainly in pitch (a hard case); the English lesson has clearly different voices. The
classroomandbacklevels are as in ASR (section 2). Synthetic speech, no real children. - "Groups / speakers" = number of detected voice groups / number of true speakers. "Overlap: recall / precision" = frames in which several people speak at once. In parentheses at the
backlevel: DER according to the source with a 0.25 s collar; forclassroomthe source gives a single value.
| history | classroom | 6.8% | 3 / 4 | 94% / 94% | 5/5 |
|---|---|---|---|---|---|
| biology | classroom | 6.2% | 3 / 5 | 98% / 90% | 4/4 |
| mathematics | classroom | 11.5% | 3 / 5 | 82% / 88% | 3/4 |
| English | classroom | 11.7% | 4 / 5 | 75% / 88% | 4/5 |
| history | back | 8.3% (5.5%) | 3 / 4 | 84% / 87% | 4/5 |
| biology | back | 6.5% (4.4%) | 3 / 5 | 98% / 88% | 4/4 |
| mathematics | back | 13.5% (9.6%) | 2 / 5 | 85% / 92% | 3/4 |
| English | back | 11.1% (7.0%) | 4 / 5 | 91% / 87% | 4/5 |
Conclusions:
- Speech by several people at once is a strong point: 75–98% of frames, 31 of 36 events detected.
- Students whose voices differed only in pitch (pitch-shifted TTS) were sometimes merged into one group or attached to the teacher: the model distinguished 0–1 of 3–4 students, and short answers („Czterdzieści dziewięć.” ("Forty-nine.")) went to the teacher's group. With clearly different voices (English) it found 4 groups for 5 people. Results on real children may differ.
- Time and scale: a 45-minute lesson (21.9 MB mp3) in one request: 16.7 s, 4 groups, 189 turns.
- The model is a streaming model (with speaker memory); the file was not split.
2. Comparison with external providers, 2026-11-06
The same 4 lessons, classroom and back levels, the same DER measure. Results are for single models; WER is not given because it does not apply to diarization itself.
| nvidia/Nemotron-3-Diarization | DeepInfra | 9.4% (classroom 9.1%) | 31 of 36 | |
|---|---|---|---|---|
Soniox stt-async-v6 (diarization in file mode) | Soniox | 13.6% (classroom 12.7%) | 0 of 36 | at the level of the "one speaker" answer; does not capture speech by several people |
OpenAI gpt-4o-transcribe-diarize (classroom only) | OpenAI | 18.1% | 0 of 18 | provides labeled segments; processing time 29.9 s per minute of recording |
Limitations: one run per model, small sample (4 short lessons), synthetic speech.