In short: Precision-3 reduces diarization error rate by 10.4% against Precision-2, and by 34.2% against the open-source Community-1 baseline. Against the strongest commercial alternative, the gap runs from 40% to 67%. On speaker-attributed transcription, Precision-3 has the lowest attribution penalty of any system benchmarked, meaning its transcripts stay accurate once you need to know who said what, not just what was said.

Raw transcription is largely solved, as every serious speech-to-text engine transcribes words with high accuracy, and word error rate alone rarely decides which provider a team picks anymore. Diarization, knowing who said what across a session with multiple speakers, is where the differentiation still happens. Treated correctly, it is not a feature layered on top of a transcription model: it is a foundational step in the pipeline, and it determines whether the transcript, the analytics, and the voice agent built on top of it can be trusted.

This is the third article in our Precision-3 launch series. The first introduced the model and its new optimization parameters. The second walked through tuning those parameters for different acoustic domains, using DIHARD as a testbed. This one is about the numbers: how Precision-3 performs against its own predecessors and against the rest of the market, on both diarization error rate and speaker-attributed transcription error rate, and what those numbers mean once diarization sits inside a production pipeline.

How Precision-3 compares on diarization error rate

Diarization error rate (DER) is the standard metric for the task. It sums three error types, each measured as a percentage of total speech time: false alarms (speech detected where there was none), missed speech (speech that went undetected, including missed overlapping speech), and speaker confusion (speech correctly detected but assigned to the wrong speaker). A DER of 14% means that roughly fourteen seconds of every hundred seconds of speech are misattributed in one of these three ways. Lower is better.

Against Precision-2 and Community-1 (8 datasets)

Averaged across the eight datasets, Precision-3 DER falls from 24.9% for Community-1 to 18.8% for Precision-2, to 16.4% for Precision-3: a 34.2% relative reduction against the open-source baseline and a 12.7% relative reduction against the previous commercial generation.

Against the market

Comparing Precision-3 to commercial alternatives is less direct, since most providers bundle diarization into a full speech-to-text pipeline rather than exposing it as a standalone score. Isolating the diarization error rate on four widely used benchmarks puts the gap in context:

Measured against whichever competitor performs best on each dataset, rather than the field average, Precision-3 comes in roughly 45% lower on DIHARD, 40% lower on AISHELL, three times lower on AliMeeting, and two and a half times lower on AMI-SDM.

For the full domain-by-domain breakdown, covering all eleven DIHARD sub-domains and the rest of the benchmark suite, see the benchmark page.

The error that matters most: Speaker Confusion

DER treats all three error types equally in the total, but they are not equally costly to fix. A missed silence break, or a stretch of unflagged non-speech, can usually be recovered by revisiting the audio. Speaker confusion, attributing a segment of speech to the wrong person, cannot be recovered the same way: once a transcript says the wrong participant said something, that error propagates into everything built on top of it, the meeting summary, the analytics dashboard, the turn-taking logic of a voice agent, with no straightforward way to catch it afterward.

This is where most of Precision-3's improvement comes from. Across the full 15-dataset benchmark, speaker confusion accounts for 53% of the total DER improvement over Precision-2, missed speech for 35%, and false alarms for the remaining 12%. In relative terms, Precision-3 assigns speech to the wrong speaker 18.7% less often than Precision-2.

DER breakdown results (15 datasets)

In practice, this is the difference between a diarization system that occasionally trims too much silence and one that occasionally puts your words in someone else's mouth. The second failure mode is the one that costs a product the most trust downstream, and it is the one Precision-3 was built to reduce.

From diarization error to transcription error: tcpWER and tcorcWER

DER scores diarization on its own by comparing the segmentation the model produces against a reference segmentation. That is the right measurement when diarization is the deliverable. In most Voice AI pipelines it isn't: diarization feeds a transcription engine, and what reaches the end user is a transcript with speaker labels on it. To measure that, you need a metric that scores words rather than time.

Word error rate is the usual one. It counts the words a system inserted, deleted, or substituted, divided by the number of words actually spoken. Two variants extend it to multi-speaker audio, and the difference between them is what makes them useful:

• tcpWER (time-constrained minimum-permutation word error rate) asks whether the system got the words right and attached them to the right speaker. A word transcribed perfectly but credited to the wrong person counts as an error.

• tcorcWER (time-constrained optimal reference combination word error rate) asks whether the system got the words right. Speaker labels are largely ignored, so a correctly transcribed word counts as correct no matter who it was assigned to.

Both are time-constrained, which means a hypothesized word only matches a reference word if it lands close enough in time. That prevents a system from being rewarded for producing the right words in the wrong place.

Speaker-attributed transcription benchmark

tcpWER results (4 datasets)

Precision-3 posts the lowest tcpWER on three of the four benchmarks. On AISHELL, AssemblyAI's Pro 3.5 model scores marginally lower, 35.3 against 36.6. Averaged across the four benchmarks, Precision-3 reduces tcpWER by 7.6% relative to Precision-2, with no regression on any individual dataset relative to its immediate predecessor.

tcorcWER results (4 datasets)

tcorcWER, the speaker-agnostic version of the metric, tells a different story. On AMI-SDM, Soniox scores 20.2 and ElevenLabs Scribe v2 scores 16.7, both lower than Precision-3's 22.3: judged purely on getting the words right, both providers transcribe that dataset more accurately.

For the pyannoteAI results, diarization is paired with off-the-shelf open-source transcription, Faster-Whisper and NVIDIA NeMo Parakeet, with no tuning. On-premises customers can use STT orchestration to combine Precision-3 with their own, including a fine-tuned engine.

The attribution penalty and what it costs you

Take one system, one dataset, and both metrics. The relative increase from tcorcWER to tcpWER is the attribution penalty: the share of the error rate that comes not from mishearing words but from crediting them to the wrong speaker.

On AMI-SDM, Precision-3 goes from 22.3 to 23.6, a rise of 5.8%. Soniox goes from 20.2 to 58.0, a rise of 187%. ElevenLabs Scribe v2 goes from 16.7 to 45.1, a rise of 170%. Both start with better raw transcription than Precision-3 and end well behind it, because their words are attached to the wrong speakers.

The pattern holds across the four datasets:

Precision-3 records the lowest penalty of every system benchmarked on three of the four datasets, and the second lowest on NOTSOFAR.

Word accuracy and speaker accuracy are separate capabilities, so a benchmark reporting word error rate alone cannot tell you which one you are buying: if your product reads the speaker field, excellent words with a 170% penalty serve you worse than weaker words with a 6% penalty.

Word accuracy and speaker accuracy are separate capabilities, and a provider can be strong at one and weak at the other. A speech-to-text benchmark that reports word error rate alone cannot tell you which you are buying. If your product reads the speaker field, a system with excellent words and a 170% penalty will serve you worse than one with slightly weaker words and a 6% penalty.

Why tcpWER is the number that matters for voice AI pipelines

The pattern holds across the other benchmarks: the providers with the best raw transcription are not the providers with the best speaker-attributed transcription. A strong speech-to-text engine paired with weak diarization still produces a transcript where the words are right, and the speaker labels are wrong. For any product that needs to know who said what, a meeting notetaker, a call center analytics tool, a voice agent deciding who to respond to- the gap between tcorcWER and tcpWER is the number to watch. A speech-to-text engine can score well on word accuracy alone and still be unusable for speaker-aware use cases if the diarization underneath it is weak. That gap is the practical argument for treating diarization as a first-class step in the pipeline rather than an assumed byproduct of transcription.

Evaluating Precision-3 on your own audio

Public benchmarks are a proxy, not a guarantee. DIHARD, AMI, AliMeeting, and the rest of the suite cover meetings, broadcast, telephone speech, and read speech, but no single domain in that list will match your production audio exactly. Codec, microphone setup, number of speakers, and background noise all shift the numbers. Before committing to a model change in production, it is worth running the same evaluation on a sample of your own audio.

The process is the same one used to produce the figures in this article. Label a small set of representative recordings, ideally thirty minutes to a few hours, with a reference RTTM file marking who spoke when. Run Precision-3 on that same audio and compute DER against the reference with a standard scoring tool such as pyannote.metrics. If your pipeline includes a transcription step, compute tcpWER and tcorcWER the same way, using a time-constrained scorer such as meeteval, against a reference transcript with speaker labels attached. Running the same comparison against your current model gives a direct, apples-to-apples result on your own data rather than an inference from public benchmark averages.

A few dozen minutes of labeled audio is usually enough to tell whether the domain gap matters for your use case. If your production audio resembles a clean, structured domain such as broadcast or telephone speech, the public numbers in this article should transfer reasonably well. If it resembles a harder domain, far-field meeting audio, dense crosstalk, in-the-wild web video, expect the absolute error rates to be higher across every system, Precision-3 included, even where the relative ranking between systems tends to hold.

Getting started

Precision-3 is available today on the API by setting "model": "precision-3" in a diarization request. Existing integrations keep working unchanged until Precision-3 becomes the default, and pricing is unchanged from Precision-2. On-premises deployments require a package update before the model becomes available to workloads.

The two articles that precede this one in the series cover the rest of the release: the first introduces the model, its input parameters, and its output scores in detail, and the second walks through tuning vadSensitivity and crosstalkSensitivity for your own audio domain. Both are available on the pyannoteAI blog, alongside the full benchmark data at pyannote.ai/benchmark.