SpeechBrain and pyannote appear in the same shortlists because both are respected open-source projects with academic roots and speaker technology at their core. The two projects are shaped by different ambitions. SpeechBrain aims to cover all of speech processing as a research toolkit, and diarization is one recipe among dozens. pyannote exists for speaker diarization first, and everything in it serves that problem.

The short answer, up front: for speech research across many tasks, for training your own models, and for speaker verification work, SpeechBrain is an excellent toolkit and one of the field's genuine contributions. For diarization as a capability you deploy, pyannote provides what SpeechBrain does not attempt: pretrained end-to-end pipelines, overlap-aware segmentation, published full-pipeline benchmarks, and a supported path from open source to hosted API.

At a glance

A toolkit for all of speech vs. a framework for one problem

SpeechBrain, by its own description, is a general-purpose speech toolkit: speech recognition, synthesis, enhancement, translation, emotion recognition, language identification, multi-microphone beamforming, speaker verification, and more, built for researchers assembling and training systems. Speaker diarization lives in it as a recipe, most prominently the AMI recipe, which composes an ECAPA-TDNN embedding extractor with spectral clustering (with PLDA available for scoring) into an experiment a researcher runs against the AMI meeting corpus.

pyannote's open-source framework (pyannote.audio, now the 4.0 generation with the Community-1 pipeline) inverts the shape: one problem, treated end to end. The model card names every stage. Segmentation uses the powerset multi-class formulation of Plaquet and Bredin (2023), embeddings come from the WeSpeaker toolkit of Wang and colleagues (2023), and clustering uses Bayesian HMM clustering of x-vector sequences, the VBx method of Landini and colleagues (2022). Those stages ship as one pretrained pipeline that runs on arbitrary audio, with real-world speaker counts, long-form audio, and independence from ground-truth annotations handled inside the pipeline rather than left to the user. The difference is not quality of engineering on either side. It is what each project is for.

The shared lineage: ECAPA-TDNN

These projects are relatives rather than strangers. ECAPA-TDNN, introduced by Desplanques and colleagues in 2020, became one of the most influential speaker embedding architectures in the field, and SpeechBrain's implementation and pretrained release (spkrec-ecapa-voxceleb) made it the reference version, with the SpeechBrain team's 2021 paper extending it to diarization and reporting strong results on AMI. That embedding model has been used far beyond SpeechBrain itself, including in earlier generations of pyannote's own pipelines.

The current generation has moved on. Community-1 uses WeSpeaker embeddings, so the lineage is a debt the field owes rather than a dependency pyannote still carries. SpeechBrain's embedding research is part of why the whole field, pyannote included, got better, and the distinction being drawn in this article is not embeddings against embeddings. It is a recipe a researcher assembles against a pipeline a developer deploys.

Accuracy, and why the published numbers are hard to compare

Most quick comparisons miss a subtlety that decides the whole question: SpeechBrain's flagship diarization results are not measuring the same thing as full-pipeline benchmarks.

The ECAPA diarization paper underlying the AMI recipe evaluates with oracle speech/non-speech labels, meaning ground-truth voice activity is handed to the system and the measurement isolates clustering quality. That is a legitimate and standard research choice, and the paper says so plainly. It also means those DER figures cannot sit next to end-to-end pipeline results, where voice activity detection errors (missed and false speech) are part of the score.

The contrast is easiest to see on AMI, since that is the corpus both projects report on. pyannote publishes Community-1's AMI results as 17.0% DER on AMI (IHM) and 19.9% on AMI (SDM), under conditions the model card states explicitly: "fully automatic processing, no forgiveness collar, nor skipping overlapping speech." No ground truth is supplied, no tolerance margin is applied around speaker boundaries, and overlapping speech counts against the score rather than being excluded from it. Those are the three concessions that make research DER figures look lower, and pyannote takes none of them.

So a team that benchmarks the AMI recipe on its own audio, without oracle labels, is measuring a different and harder task than the SpeechBrain headline numbers describe. Read conditions before you read percentages, for either project.

pyannote's broader results are full-pipeline by the same construction. The public benchmark reports DER across ten DIHARD domains over 259 recordings for Community-1 and Precision-2 against commercial and open systems, with the methodology documented. SpeechBrain's AMI recipe is not among the systems in that comparison, so this article carries no head-to-head number for it.

Two structural limits of the recipe approach are worth naming. Its segmentation stage assigns each segment to a single speaker, so overlapping speech is not attributed to two people the way a powerset formulation attributes it. And the experiment is configured around a specific corpus, with researchers routinely describing their setups as modified versions of the AMI recipe rather than applications of a general pipeline.

Recipes vs. pipelines: what running each actually involves

Running SpeechBrain diarization means running an experiment: install the toolkit, take the AMI recipe's experiment script and hyperpyyaml configuration, prepare the dataset layout the recipe expects, and execute, with modification expected for any audio that is not AMI-shaped. This is by design. Recipes are reproducible research artifacts, and SpeechBrain's documentation and community treat them that way.

Running pyannote means calling a pipeline: install the package, load the pretrained Community-1 pipeline, pass audio, receive speaker-attributed segments, on any recording, with no dataset preparation or configuration required to get a first result.

python

from pyannote.audio import Pipeline

pipeline = Pipeline.from_pretrained(
    "pyannote/speaker-diarization-community-1",
    token="HUGGINGFACE_ACCESS_TOKEN")

output = pipeline("audio.wav")

for turn, speaker in output.speaker_diarization:
    print(f"start={turn.start:.1f}s stop={turn.end:.1f}s speaker_{speaker}")

The production details are in the pipeline too. output.exclusive_speaker_diarization returns one speaker per moment, which the model card describes as simplifying "the reconciliation between fine-grained speaker diarization timestamps and (sometimes not so precise) transcription timestamps," so a team wiring diarization to an STT engine gets the reconciliation-friendly view without building it. Speaker count constraints are parameters rather than config edits. And the hosted API is a further step down in setup cost: a POST to https://api.pyannote.ai/v1 with a bearer token, no local model, no GPU.

Neither workflow is wrong. They encode different assumptions about the user.

Where SpeechBrain wins: verification

The verification-versus-diarization distinction belongs here, because verification is where SpeechBrain genuinely shines. For speaker verification research (is this the claimed speaker?), SpeechBrain's ECAPA models and training recipes are a first-class choice: the released checkpoint reports 0.80% EER on the cleaned VoxCeleb1 test set, under Apache 2.0.

Diarization (who spoke when, among an unknown number of speakers, in continuous audio) is a different and harder systems problem, and it is the one pyannote is built around. Readers unsure which task they have should see our explainer on diarization, recognition, and identification, and the speaker identification tutorial for how structure and identity combine in production.

Community, maintenance, and the production path

Both projects are alive and well maintained. SpeechBrain has an active academic community, steady releases, and wide use in research across many speech tasks. pyannote's community is concentrated on one problem, which shows in the cadence of diarization-specific improvements: Community-1 superseding 3.1 with lower DER on eleven of twelve benchmark datasets, published benchmarks updated against the commercial field, and a company whose entire business is the accuracy and deployability of these models.

The production path is the cleanest difference. SpeechBrain is a toolkit, so deploying its diarization recipe in production means owning the assembly, hardening, and serving yourself, which sits outside the project's stated purpose. pyannote's open-source pipeline is production-shaped from the start, the same Community-1 model runs on the hosted API with no infrastructure at all, and Precision-2 sits above it for teams that want the accuracy ceiling on the same integration. For audio that has to stay inside your own environment, the same models deploy on-premise without a change of stack.

The honest verdict

Choose SpeechBrain when the work is research: training or fine-tuning your own speaker models, running experiments across speech tasks, teaching, or building on its verification stack, where its ECAPA lineage is a genuine strength the whole field has benefited from. Inside that purpose it is one of the best toolkits available.

Choose pyannote when the job is diarization itself: unknown speaker counts, real-world audio, overlap, production deployment, or accurate speaker attribution without assembling a system, which describes most teams making this comparison. Start with the open-source Community-1 pipeline, move to the hosted API when infrastructure should disappear, and to Precision-2 when accuracy justifies it.

Run both on your own audio. SpeechBrain is free, Community-1 is free, and the benchmark methodology documents how to make the test fair. Start with the API quickstart when you want a result in the next ten minutes.

Frequently asked questions

Can SpeechBrain do speaker diarization? Yes, as a recipe rather than a pretrained pipeline. The AMI recipe combines ECAPA-TDNN embeddings with spectral clustering, and running it on your own audio means adapting the experiment's dataset preparation and configuration. There is no one-call pretrained diarization pipeline in SpeechBrain the way there is in pyannote.audio.

Which is more accurate, SpeechBrain or pyannote? The published numbers are measured under different conditions, so a direct comparison is not available. SpeechBrain's AMI results use oracle speech/non-speech labels, which isolates clustering quality. pyannote's AMI results (17.0% DER on IHM, 19.9% on SDM) are fully automatic, with no forgiveness collar and overlapping speech included in the score. Benchmark both on your own audio under identical conditions before concluding anything.

Is pyannote's Community-1 open source and free? The pyannote.audio codebase is MIT-licensed and the Community-1 pipeline weights are released under CC-BY-4.0. Both are free to use, with attribution required for the weights. Community-1 is also available on the hosted API if you would rather not run it yourself.

Do pyannote and SpeechBrain share models? Historically the field has, since SpeechBrain's ECAPA-TDNN release became a reference implementation used well beyond the toolkit. The current pyannote generation does not: Community-1 uses WeSpeaker embeddings, powerset segmentation, and VBx clustering.