Mean DER, 7 evaluation sets
Speaker count withheld from both systems. Lower is better.
Precision-3
15.5%
Universal-3.5 Pro
40%
61%
relative reduction in diarization error rate
The workload moved from transcription to diarization
When meeting intelligence, call QA or agent analytics carry the product, speaker attribution becomes the component that needs benchmarking on its own.
Speaker labels are needed during the conversation
Streaming diarization is a separate capability from batch, measured on different numbers: time to first speaker label, and whether an early decision can be revised once more audio arrives.
Known speakers need names, not letters
Anonymous labels reset at every file. Recurring participants and enrolled agents need an identity that survives across files, sessions and streams.
Deployment and pricing shape changed
On-premise, on-devices and EU residency requirements narrow the shortlist before any accuracy test runs.
Diarization via diarization: "speaker", with a speaker property on every word. Speaker identification by enrolment is available in both batch and real-time, and on-premise containers are offered alongside cloud.
enable_speaker_diarization: true returns a speaker per token over the WebSocket or async API, across all supported languages. The documentation notes that async gives significantly higher accuracy than real-time, where temporary speaker switches stabilise as more context arrives.
The diarize_model parameter both enables diarization and selects a version. The v2 batch diarizer runs on Nova-1, Nova-2, Nova-3, enhanced and base; streaming accepts v1 or latest only, and v2 returns a validation error.
Both models were run on the same 14 evaluation sets with no speaker count supplied to either system. Precision-3 averages 15.32 DER against 32.17, a 52% relative reduction. Lower is better.
Splitting DER into its three components shows where the difference comes from:
Precision-3 records the lowest DER on all 14 datasets. Full methodology and per-domain charts: pyannote.ai/benchmark.
DIHARD CTS
Where STT tells you what was said, pyannote tells you who said it, when, and how. It does not compete with STT or LLMs, it completes them.
01
02
03
04
05
06
Speaker intelligence is step two: audio input, speaker intelligence, transcription, business logic, text to speech, audio output. Inside that step the model runs voice activity detection, speaker segmentation, speaker embedding and clustering with attribution, and returns speaker turns, speech detection and a speaker count the rest of the pipeline can rely on.
The order matters. Diarization derived from a completed word timeline inherits every transcription error with no way to recover downstream. Running it first improves tcpWER, which penalises both transcription and speaker attribution errors, most visibly in cross talk, four or more speakers, far-field microphones and background noise.
Run every provider on one test set that looks like your production audio: real accents, background noise, overlapping speech, real channel conditions. Twenty to fifty files is usually enough. Withhold the speaker count from every system, since providing it flatters models that guess badly, and use the same scoring tooling throughout. Report the three DER components separately rather than the aggregate, because each points at a different stage of the pipeline.
The criteria that actually separate providers: DER on your own audio, whether overlap is detected by the model or inferred from a word timeline, speaker count behaviour with the count withheld, timestamp granularity, streaming latency and whether an early label can be revised, speaker identity across sessions, confidence scores for human review, deployment options, and the pricing unit rather than the headline rate.
Diarization runs inside a transcription job. Setting speaker_labels to true returns a list of utterances, each an uninterrupted segment of speech from a single speaker. It does not support multichannel transcription, and enabling both returns an error. There is no endpoint that returns speaker turns without also producing a transcript.
In streaming, speaker_labels: true is added to the connection, and max_speakers between 1 and 10 hints at the expected count. AssemblyAI's documentation notes that streaming diarization is most reliable with two speakers, that accuracy decreases as the count rises because less audio per person means weaker voice profiles, and that a streaming system cannot revise a decision once made. The feature is documented as public beta.
What is the best alternative to AssemblyAI?
Yes, absolutely! You can sign up for a free trial and enjoy 150 hours of identification and 10 voiceprints for one month. You will have access to our latest premium models to ensure accuracy and precision for your speech processing needs.
How does pyannoteAI compare to AssemblyAI for speaker diarization?
Yes, absolutely! You can sign up for a free trial and enjoy 150 hours of identification and 10 voiceprints for one month. You will have access to our latest premium models to ensure accuracy and precision for your speech processing needs.
Can diarization run without transcription?
Yes, absolutely! You can sign up for a free trial and enjoy 150 hours of identification and 10 voiceprints for one month. You will have access to our latest premium models to ensure accuracy and precision for your speech processing needs.
How is diarization accuracy measured?
Yes, absolutely! You can sign up for a free trial and enjoy 150 hours of identification and 10 voiceprints for one month. You will have access to our latest premium models to ensure accuracy and precision for your speech processing needs.