# Diarization: who is speaking

> On four short synthetic lessons, nvidia/Nemotron-3-Diarization (DeepInfra) averaged 9.4% DER and detected 31 of 36 overlapping-speech events, while Soniox stt-async-v6 and OpenAI gpt-4o-transcribe-diarize detected none; note that a trivial "one speaker" answer already scores 13–18% DER on this set.

Area: Speech. Page: https://benchmarks.site/diarization/. Methodology and caveats: https://benchmarks.site/methodology/#speech-vision-text

# Diarization: recognizing who is speaking

Metric: **DER** (diarization error rate; lower = better), computed on 10 ms frames, including speech by several people at once, consistent with pyannote.metrics to within 0.1–0.2 points. Reference point: the trivial answer "all speech belongs to one speaker" gives a DER of 13–18% on this set (the teacher speaks 82–87% of the time), so a DER of 6–14% looks good mainly because one speaker dominates. Metric definitions and caveats: [overview](/methodology/#speech-vision-text).

## 1. nvidia/Nemotron-3-Diarization (DeepInfra), 2026-09-30

- The model returns only speaker turns (anonymous labels `speaker_N`, up to 8 speakers, consistent within a file, 10 ms resolution, no confidence and no text); turns of different speakers overlap when several people speak at once. The output is deterministic (the same on every call on the same file).
- Data: 4 synthetic lessons, 7.1 min in total: history, biology, mathematics, English; 3–4 students in each, 4–5 places with two people speaking at once. In three lessons the students are a pitch-shifted (higher) synthetic Zosia, so the voices differ mainly in pitch (a hard case); the English lesson has clearly different voices. The `classroom` and `back` levels are as in [ASR](/asr-polish/) (section 2). **Synthetic speech, no real children.**
- "Groups / speakers" = number of detected voice groups / number of true speakers. "Overlap: recall / precision" = frames in which several people speak at once. In parentheses at the `back` level: DER according to the source with a 0.25 s collar; for `classroom` the source gives a single value.

| recording | level | DER | groups / speakers | several people at once: recall / precision (frames) | "overlap" events detected |
|---|---|---|---|---|---|
| history | classroom | 6.8% | 3 / 4 | 94% / 94% | 5/5 |
| biology | classroom | 6.2% | 3 / 5 | 98% / 90% | 4/4 |
| mathematics | classroom | 11.5% | 3 / 5 | 82% / 88% | 3/4 |
| English | classroom | 11.7% | 4 / 5 | 75% / 88% | 4/5 |
| history | back | 8.3% (5.5%) | 3 / 4 | 84% / 87% | 4/5 |
| biology | back | 6.5% (4.4%) | 3 / 5 | 98% / 88% | 4/4 |
| mathematics | back | 13.5% (9.6%) | 2 / 5 | 85% / 92% | 3/4 |
| English | back | 11.1% (7.0%) | 4 / 5 | 91% / 87% | 4/5 |

Conclusions:
- Speech by several people at once is a strong point: 75–98% of frames, 31 of 36 events detected.
- Students whose voices differed only in pitch (pitch-shifted TTS) were sometimes merged into one group or attached to the teacher: the model distinguished 0–1 of 3–4 students, and short answers („Czterdzieści dziewięć.” ("Forty-nine.")) went to the teacher's group. With clearly different voices (English) it found 4 groups for 5 people. Results on real children may differ.
- Time and scale: a 45-minute lesson (21.9 MB mp3) in one request: 16.7 s, 4 groups, 189 turns.
- The model is a streaming model (with speaker memory); the file was not split.

## 2. Comparison with external providers, 2026-11-06

The same 4 lessons, `classroom` and `back` levels, the same DER measure. Results are for single models; WER is not given because it does not apply to diarization itself.

| model | platform | DER average | several people at once detected | notes |
|---|---|---|---|---|
| nvidia/Nemotron-3-Diarization | DeepInfra | 9.4% (classroom 9.1%) | 31 of 36 | |
| Soniox `stt-async-v6` (diarization in file mode) | Soniox | 13.6% (classroom 12.7%) | 0 of 36 | at the level of the "one speaker" answer; does not capture speech by several people |
| OpenAI `gpt-4o-transcribe-diarize` (classroom only) | OpenAI | 18.1% | 0 of 18 | provides labeled segments; processing time 29.9 s per minute of recording |

Limitations: one run per model, small sample (4 short lessons), synthetic speech.
