Blog
Ask any team running diarization in production what surprised them, and speaker counting comes up fast. The transcript looks right, the labels are stable, and then a five-person meeting comes back with six speakers, or a panel discussion with nine voices comes back with five. Nobody advertises this problem. We'd rather explain it.
Here's the short answer to how counting works: in clustering-based diarization, nobody counts. The system converts stretches of speech into voice fingerprints called embeddings, groups similar fingerprints into clusters, and the number of clusters that survive is the speaker count. The estimate is a byproduct of grouping, which is why it fails in the specific, predictable ways this article walks through, why every honest vendor's accuracy has a ceiling here, and why the parameters that let you constrain the estimate are some of the most useful lines in the API docs.
A note on terminology before we start. Speaker diarization answers "who spoke when" without knowing who those people actually are, which makes it a question about structure rather than identity. Speaker identification is the separate capability that matches a voice to a known person. Counting belongs to structure, and it sits at the diarization layer of the voice AI stack.
Clustering is the counter
A clustering-based pipeline works in stages. Segmentation finds who-is-speaking-when structure locally. An embedding model turns each speech stretch into a vector that captures how the voice sounds rather than what it says. Then clustering groups those vectors, merging the ones that are close, and the count falls out of where the merging stops.
That stopping decision rides on a similarity threshold, and the threshold is the whole game. Too permissive and two real people collapse into one cluster, which is under-counting. Too strict and one person's varied speech splits into several, which is over-counting.
The Community-1 model card names each stage in the current generation: powerset segmentation (Plaquet and Bredin, 2023), WeSpeaker embeddings (Wang et al., 2023), and Bayesian HMM clustering of x-vector sequences, the VBx method (Landini et al., 2022). Hervé Bredin, pyannote's creator, walks through the mechanics in his technical talks on the architecture and on the research direction, and the overview of how diarization actually works is the fastest way to see the stage-by-stage picture visually.
The reason a single perfect threshold cannot exist is the fundamental ambiguity of voice. One speaker varies enormously across a recording: they get animated, they lean away from the mic, they move rooms, they get tired. Two different speakers can meanwhile sound remarkably alike. When within-speaker variation is bigger than between-speaker difference, and on hard audio it often is, no threshold cleanly separates "same person, different moment" from "different person, similar voice." Counting errors are what that ambiguity looks like at the output rather than a defect layered on top of it.
The four ways the count goes wrong
Speaker count is the load-bearing structure underneath every downstream metric that treats "per speaker" as a unit. Talk time per participant, sentiment per side, compliance flags per person, cost per attendee, engagement per host: every one of those numbers is only as trustworthy as the count they roll up to. A pipeline that quietly turns five speakers into six turns every per-speaker chart into a slightly wrong story. That is why the failure modes below are worth understanding rather than papering over.
Short turns and backchannels. A "yeah," an "mm-hmm," a half-second interjection: these give the embedding model almost nothing to work with, and a fingerprint built from a sliver of speech is unreliable. Brief participants either get absorbed into whoever they sound most like, so the meeting's quiet attendee vanishes, or spawn a spurious extra cluster, so the enthusiastic back-channeler becomes a phantom sixth speaker. Meetings are full of exactly this speech.
Similar voices. Same-gender colleagues, family members, speakers of similar age recorded on the same channel: their embeddings sit close together, and close embeddings merge. This is the classic under-count, and it explains why counting accuracy degrades as the real speaker count rises, since ten voices in one room have less acoustic room between them than three.
Acoustic drift. The same voice through a headset, then a speakerphone, then from across the room produces embeddings that drift apart, and drift splits clusters. This is the over-count mechanism, and it compounds with recording length, which is one reason long recordings are where counting fails slowly rather than obviously.
Overlap. When people talk simultaneously, the mixed regions produce contaminated embeddings that belong to no one cleanly. Our deep dive on overlapping speech covers the mechanism. For counting, the consequence is noise injected exactly where conversations are most dense.
The other approach: don't count at all
End-to-end diarization models sidestep estimation by building a fixed number of speaker slots into the architecture. That buys elegant handling of overlap and tight integration, at the price of a hard ceiling. NVIDIA's Sortformer supports a maximum of four speakers by construction, per its own model card, with degraded performance beyond that.
A cap functions as a promise that your audio stays small, and real-world audio does not promise back. Our pyannote vs. NeMo comparison covers this trade in full. Clustering-based counting, for all the failure modes above, has no ceiling, which is why it remains the approach for unconstrained audio.
How Precision-2 approaches it, with the numbers
pyannoteAI's public writeup on the Precision-2 launch is the source for what changed and what the numbers look like. In headline terms, Precision-2 is "14% more accurate than Precision-1 and 28% more accurate than pyannote.audio OSS 3.1 model," measured as diarization error rate.
On counting specifically, the writeup states the result plainly: "Precision-2 predicts the correct number of speakers on 70% of our most difficult internal benchmark (250+ files with 2 to 10 speakers), while Precision-1 barely reached 50%." The post separately reports a relative reduction of 37% on the speaker confusion rate. Those are two different measurements rather than one derived from the other, and quoting them as such is the honest version.
The delta from 50% to 70% matters as much as the absolute figure, because the comparison is against the vendor's own prior model on the same set rather than against a competitor chosen for the purpose.
The other transparent proof point is the public benchmark, which evaluates diarization across ten DIHARD domains, 259 recordings, roughly 67 hours of audio, and 9.3% overlapping speech, against nine other commercial and open systems. Precision-2 records the lowest diarization error rate in all ten domains.
DER is not a pure counting metric, since it combines speaker confusion, missed detection, and false alarms. Speaker confusion is one of its three components, and dominating DER across ten domains against nine systems is the strongest available evidence that the counting-adjacent behavior generalizes beyond one internal set. It is worth saying plainly that a direct per-domain counting-accuracy table would be better evidence still, and none of the systems in this market publishes one.
The practical fix: tell the system what you know
Here's the guidance that saves real products, and it's almost embarrassingly simple: the count is only estimated when you don't provide it. In most applications, somebody knows something about the speaker count before the audio is processed. A two-party call is a two-party call. A recorded deposition has a participant list. A podcast has named hosts and guests.
The API takes that knowledge three ways. Pass numSpeakers when you have the exact count, and estimation leaves the loop entirely. The docs note that "setting numSpeakers typically results in better overall diarization performance since the model can optimize for a specific speaker count." When the exact number is unknown but a range is not, pass minSpeakers, maxSpeakers, or both. The two range parameters cannot be combined with numSpeakers, and minSpeakers must be less than or equal to maxSpeakers.
A minimal diarization request with a known count:
json
For a bounded range instead of an exact count:
json
When you genuinely don't know the count, two other tools help. Confidence scores return a 0 to 100 value per diarization turn, so low-confidence turns become a review queue rather than a silent error, and they are the closest thing to a warning light that the count may be wrong on a given file. And voiceprints hold a known participant's identity steady across breaks, sessions, and separate recordings, which removes acoustic drift as a source of over-counting for the people you have enrolled.
The product-design corollary is worth a sentence in your planning doc: capture speaker count metadata at the point where your users have it, whether that is the meeting invite, the case file, or the booking form, and pass it through. The cheapest accuracy improvement available in diarization comes from keeping information you already had rather than from a better model.
Frequently asked questions
How do diarization models know how many speakers there are? Clustering-based systems don't know; they estimate. Speech is converted into voice embeddings, similar embeddings are grouped, and the surviving cluster count is the answer. End-to-end models avoid estimating by fixing a maximum speaker count in the architecture, which trades the problem for a ceiling. The stage-by-stage explainer video walks through the whole pipeline visually.
Why does my diarization detect the wrong number of speakers? The usual suspects: brief speakers whose embeddings are unreliable, similar voices merging, one voice drifting apart across channels or hours, and overlap contaminating the fingerprints. Under-counts usually mean merged voices, and over-counts usually mean one voice split by acoustic drift. Our explainer video shows where each of these hits the pipeline.
How accurate is automatic speaker counting? On pyannoteAI's most difficult internal benchmark of 250+ files spanning 2 to 10 speakers, Precision-2 predicts the exact count on 70% of files, against 50% for Precision-1. On the public ten-domain benchmark it records the lowest DER in every domain tested.
Can I just tell the system how many speakers there are? Yes, and you should whenever you know. Pass numSpeakers, minSpeakers, or maxSpeakers on the request and the estimate is constrained or bypassed. This is the single cheapest accuracy improvement available in a diarization pipeline.
What should I do when the count is genuinely unknown? Enable confidence scores and route low-confidence turns to review, and enrol voiceprints for the participants who recur, so drift stops producing phantom speakers for the people you can identify.
Ready to test the counting behavior on your own hard files? Start with the API quickstart, and pass the speaker count on any file where you already know it.
