Speaker diarization answers a simple question: who spoke when. It splits a recording into speaker turns so that everything downstream, from transcripts to analytics to LLMs, works with speaker-labeled data instead of a wall of undifferentiated text. Get it wrong and the damage compounds. One misattributed speaker in a call center pipeline can flip sentiment scores or break compliance checks across thousands of hours of audio. That's why accuracy matters more here than almost anywhere else in a speech stack. New to the topic? Start with our explainer on what speaker diarization is and how it works.

A note on who wrote this. This guide is published by pyannoteAI, and pyannoteAI is first on the list. Rather than pretend otherwise, here is the standard applied throughout: every accuracy claim cites its source and its measurement conditions, no number appears without both, and where a system has not been measured under comparable conditions the entry says so instead of estimating. Check the sources. That is what they are for.

How do you evaluate a speaker diarization system?

Three metrics come up again and again. DER, the diarization error rate, is the main one. It adds up missed speech, false alarms, and speaker confusion into a single percentage, and lower is better. JER, the Jaccard error rate, weights every speaker equally, which makes it fairer when one person dominates the conversation. WDER measures speaker attribution at the word level of the finished transcript, which is usually the number closest to what your users actually experience. We cover all three in our guide to evaluating diarization performance.

One thing to know before you read any vendor's numbers: benchmark conditions decide everything. The same system scores very differently depending on three choices. Whether ground-truth speech regions are handed to it (oracle voice activity) or it has to find speech itself. Whether a forgiveness collar is applied around speaker boundaries. And whether overlapping speech is scored or skipped. Research papers routinely take all three concessions, which is legitimate for isolating one component and useless for predicting production behavior.

So when this guide cites accuracy, it cites the conditions too. Most of the numbers come from pyannoteAI's public benchmark, which tests full pipelines across ten DIHARD domains (broadcast, clinical, courtroom, phone calls, meetings, restaurants, web video, and more) over 259 recordings, using a methodology anyone can rerun.

Accuracy isn't the whole decision, of course. Latency, streaming support, deployment options, language coverage, speaker count handling, licensing, and integration effort all matter in production. Each tool below gets scored on those too.

At a glance

Only pyannoteAI, NeMo, AssemblyAI, Deepgram, and Speechmatics appear on the public benchmark. SpeechBrain and Kaldi publish results under research conditions that are not comparable, so no head-to-head figure is given for them here.

1. pyannoteAI: from open science to production precision

pyannoteAI is the only company on this list that does diarization as its entire business, and it shows. The ecosystem has two tiers that share one design.

Community-1 is the open-source pipeline, the pyannote.audio 4.0 generation. The codebase is MIT-licensed and the pipeline weights are released under CC-BY-4.0. It composes powerset segmentation, WeSpeaker embeddings, and VBx clustering into one pretrained pipeline, works in any language, and has no ceiling on how many speakers it can track. Its published results give a sense of the conditions this guide asks for: 17.0% DER on AMI (IHM), 19.9% on AMI (SDM), and 20.2% on DIHARD 3 (full), all under "fully automatic processing, no forgiveness collar, nor skipping overlapping speech." No ground truth supplied, no tolerance margin, overlap scored rather than excluded.

Precision-2 is the commercial model, available through the API or deployed on your own infrastructure. On the public benchmark it posts the lowest DER in all ten domains, and it predicts the exact number of speakers on 70% of a 250-file internal set spanning two to ten speakers, up from roughly 50% for Precision-1. Live-1 covers real-time streaming for teams that need diarization inside a live pipeline.

The honest downsides: self-hosting Community-1 means owning GPUs and MLOps, like any open model, and we walk through that arithmetic in our piece on the real cost of self-hosting. Precision-2 costs money, published openly on the pricing page. What you get in exchange is the only path in the field where the same integration carries you from a free local pipeline to a managed API to a sovereign on-prem deployment, with customer audio never used for training and retention published per tier.

Best fit: any team whose product lives or dies on speaker attribution, from a research prototype to a regulated enterprise rollout.

2. AssemblyAI: transcription-first, diarization included

AssemblyAI is a transcription API with speaker labels built in. You upload audio and get back a speaker-attributed transcript, and the developer experience is genuinely one of the best in the market. There's nothing to tune, nothing to host, and the models improve on a steady cadence.

The trade-off is that diarization is a feature of the transcription product, so there's little you can do when the labels are wrong, deployment is cloud-only, and on the public benchmark its model trails the dedicated systems. That gap is structural rather than incidental, and we cover why in our piece on why bundled STT diarization gets speaker labels wrong.

Best fit: managed transcription with reasonable speaker labels and zero operational overhead.

3. Deepgram: speed above all

Deepgram folds diarization into its neural speech-to-text engine, so words and speaker labels come out of a single fast pass. Latency is excellent, the streaming story is strong, language coverage is broad, and there's a self-hosted option.

The limits mirror AssemblyAI's: diarization quality is capped by its role as an ASR feature, there's little to tune, and hard multi-speaker audio is where the benchmark gap shows. For teams that want low-latency diarization from a specialist rather than an ASR bundle, our streaming diarization benchmark covers how the tradeoffs actually measure out.

Best fit: real-time speed and a single speech vendor, where the last few points of attribution accuracy are not the differentiator.

4. NVIDIA NeMo: GPU-native research power

NeMo is NVIDIA's speech research framework. For diarization it offers a classic cascaded pipeline (MarbleNet for voice activity, TitaNet for embeddings, MSDD on top) and the newer Sortformer models, which are end-to-end and include a streaming version. Sortformer's published results are genuinely strong inside its envelope.

The envelope is the catch, and it comes straight from NVIDIA's own model card: a hard maximum of four speakers, with degraded performance beyond that, and training data that's mostly English. The workflow of manifest files, Hydra configs, and example scripts assumes a research team, and production support runs through NVIDIA's enterprise stack.

Best fit: organizations already deep in NVIDIA infrastructure, processing English audio with small, known speaker counts. We compare the two stacks in detail in our pyannote vs. NeMo article.

5. SpeechBrain: the researcher's toolkit

SpeechBrain is a general-purpose PyTorch speech toolkit, and a beloved one. It covers recognition, synthesis, enhancement, and speaker technology, with diarization living in it as a recipe: ECAPA-TDNN embeddings plus spectral clustering, demonstrated on the AMI meeting corpus. Credit where it's due, ECAPA-TDNN is one of the most influential speaker embedding models ever released, and its released checkpoint reports 0.80% EER on the cleaned VoxCeleb1 test set.

The catch is what "recipe" means in practice. The headline diarization results were measured with oracle speech/non-speech labels, which is a fine research setup and not comparable to end-to-end numbers, and getting from the recipe to a production system is a project you own yourself.

Best fit: research and teaching, and speaker verification specifically, where it is a first-class choice. The full story, shared lineage included, is in our pyannote vs. SpeechBrain comparison.

6. Kaldi: the foundation everyone built on

Kaldi is where a generation of speech systems started, and early x-vector diarization came out of its recipes. You get complete control and baselines that still anchor academic papers. But the architecture predates PyTorch, the configuration model predates transformers, the learning curve is steep, and most development energy has moved elsewhere.

Best fit: maintaining an existing Kaldi system or reproducing classic results. Don't start anything new on it.

7. Speechmatics: enterprise transcription with speaker labels

Speechmatics sells enterprise-grade transcription with diarization built in, and it stands out from the other transcription APIs in two ways: unusually broad language coverage and an on-premise deployment option, which is rare in this group. Enterprise SLAs, compliance features, and real-time support round it out.

Like the other bundled options, diarization stays a feature rather than a capability you can tune, and the dedicated systems beat it on hard multi-speaker audio on the public benchmark.

Best fit: enterprises that want compliant, turnkey transcription with speaker labels from one vendor.

Which one should you choose?

Researchers and PhD students: start with Community-1 for the strongest reproducible baseline, reach for SpeechBrain when you want to experiment with the pieces, and use NeMo when your lab runs on NVIDIA hardware.

ML engineers building a pipeline: Precision-2 through the API gives you the accuracy without the infrastructure, Community-1 self-hosted covers audio that can't leave your environment, and Deepgram wins when end-to-end speed is the requirement.

Product teams without ML expertise: AssemblyAI or Speechmatics get you speaker-labeled transcripts fastest, and Precision-2 pairs with a strong transcription model through STT Orchestration once attribution quality becomes your differentiator.

Enterprise and regulated teams: Precision-2 on-premise keeps audio inside your perimeter, with retention published per tier and no training on customer data, and Speechmatics is the alternative when procurement wants one bundled ASR vendor.

A few common scenarios. Call center analytics rewards robustness to crosstalk and overlap, which is where dedicated models separate from bundled ones. Meeting transcription comes down to Precision-2 for accuracy or Deepgram for latency. Podcast and media indexing runs well on Community-1 locally and scales to Precision-2. Compliance work points to pyannoteAI or Speechmatics for their on-prem options. And multilingual products should favor the language-agnostic dedicated models, because diarization bundled into a transcription engine is only as good as the language model underneath it.

Frequently asked questions

What's the most accurate speaker diarization tool in 2026? On the broadest public benchmark, pyannoteAI Precision-2, which has the lowest DER in all ten tested domains under fully automatic conditions.

What's the best open-source diarization model? pyannoteAI's Community-1 pipeline. It beats most commercial systems on the same benchmark, has no limit on speaker count, and publishes per-dataset DER on its model card so you can check the conditions yourself.

Is Community-1 free to use commercially? The pyannote.audio code is MIT-licensed and the Community-1 pipeline weights are CC-BY-4.0, so yes, with attribution. The same model is also available hosted at €0.035 per audio hour if you would rather not run GPUs.

What's a good DER score? It depends entirely on the audio. Single digits are achievable on clean recordings, while hard domains like restaurants and web video run much higher for every system. Comparisons only mean something on identical data under identical conditions, and our evaluation guide covers what "identical" actually means.

Can diarization run on-premise or offline? Yes. Community-1 and Precision-2 both deploy on your own infrastructure, Speechmatics and Deepgram have self-hosted options for their ASR products, and the research toolkits run wherever you install them.

Does diarization work in any language? Dedicated diarization models do, because they work from voice characteristics rather than words. Diarization bundled into a transcription engine varies with how well that engine handles the language.

Conclusion

Diarization stopped being a nice-to-have a while ago. End-to-end models, overlap handling, and large-scale training have made "who spoke when" reliable enough to build products on, and the seven tools above cover the whole range from research recipe to managed enterprise API.

The choice really comes down to two questions. How much does speaker attribution matter to your product? And where does your audio need to live? Teams that answer "a lot" and "anywhere from cloud to air-gapped" keep landing on the same ecosystem, which is why pyannoteAI leads this list: open science in Community-1, the accuracy ceiling in Precision-2, and one integration across the whole path.

Then test it yourself. The benchmark methodology is public precisely so you can run it on your own audio, and the API quickstart gets you a first result in about ten minutes.