Pyannote's diarization model is language-agnostic. A conversation in Tamil, a sales call that switches between Spanish and English mid-sentence, and a Mandarin board meeting all run through the same weights, with no language flag, no locale configuration, and no per-language model registry to maintain.

That property is easy to state and easy to underestimate. It changes how multilingual voice pipelines get built, where they break, and what your team should actually test before shipping into a global market.

Why diarization does not need to know the language

Diarization answers who spoke when. Transcription answers what was said. The two tasks consume completely different signals, and that difference is the whole reason one of them travels across languages and the other does not.

Our diarization pipeline runs in four stages:

1. Voice activity detection separates speech from non-speech, filtering out silence, background noise, and non-verbal sounds so that compute goes to actual speech. Speech and silence are acoustic categories.

2. Segmentation handles speaker turns, overlapping speech, and interruptions.

3. Speaker embeddings turn each speech segment into a high-dimensional vector capturing the vocal characteristics that make a voice distinctive: pitch patterns, formant frequencies, and speaking style.

4. Clustering groups those vectors into speakers, an unsupervised problem that has to handle varying cluster sizes, overlapping distributions, and voice-similarity uncertainty.

Not one of those stages reads a word. Pitch, formant structure, and timbre belong to a vocal tract, and a vocal tract does not change when its owner switches from Hindi to English mid-sentence. We build our models to operate on acoustic voice characteristics rather than linguistic content, which is what makes the language-agnostic property structural rather than incidental.

Two things would make a diarization system language-dependent: an architecture that leans on lexical cues for speaker assignment, and training data confined to a single language family. Acoustic modeling rules out the first by construction. The second is a data question, which is where our benchmarks come in.

What our benchmarks cover

Precision-3, our newest and most accurate diarization model, is evaluated across 15 benchmark datasets spanning Mandarin meeting and conversational corpora, French broadcast material, English meetings, telephone speech, and in-the-wild web video: AISHELL, ALLIES, AliMeeting, AMI, AMI-SDM, AVA-AVD, CALLHOME, DIHARD, INA, MSDWILD, NOTSOFAR, RAMC, SBCSAE, VoxConverse, and VoxSRC2023.

Across that set, diarization error rate drops from 16.02 with Precision-2 to 14.35 with Precision-3, a 10.4% relative reduction. The component breakdown matters more than the headline for anyone working with multilingual conversation, because the largest gains land on the hard parts:

Crosstalk is where multilingual meetings live. Interruption-heavy conversation across language groups is exactly the condition that produces overlapped speech, and both crosstalk components improve by roughly 20%.

Per-dataset numbers for the previous generation are published on the Community-1 model card, scored with fully automatic processing, no forgiveness collar, and no skipping of overlapped speech:

AISHELL-4 is a Mandarin meeting corpus and RAMC is Mandarin conversational speech. REPERE is French broadcast material. The largest relative gain over the legacy 3.1 pipeline lands on RAMC, where error drops from 22.2% to 10.5%, a 53% reduction on a Mandarin corpus.

Independent evaluation has reached the same conclusion about our models. Benchmarking Diarization Models (Lanzendörfer, Grötschla, Blaser, and Wattenhofer, September 2025) evaluated five state-of-the-art systems over 196.6 hours of multilingual audio spanning English, Mandarin, German, Japanese, and Spanish. We placed first at 11.2% DER, ahead of the next system at 13.3%.

Where code-switching actually breaks a pipeline

Here is the part that matters for anyone building on multilingual audio. The diarization layer takes code-switching in stride. The transcription layer does not.

Look at our diarize endpoint. The request body accepts url, webhook, webhookStatusOnly, model, numSpeakers, minSpeakers, maxSpeakers, vadSensitivity, crosstalkSensitivity, turnLevelConfidence, exclusive, confidence, speechProbability, crosstalkProbability, speakerProbability, transcription, and transcriptionConfig. There is no language parameter anywhere on the diarization side, because there is nothing for one to configure.

Language coverage appears exactly once in the API surface, inside transcriptionConfig, as a property of the speech-to-text model you choose:

• parakeet-tdt-0.6b-v3, the default, covers 25 languages: Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Hungarian, Italian, Latvian, Lithuanian, Maltese, Polish, Portuguese, Romanian, Slovak, Slovenian, Spanish, Swedish, Russian, and Ukrainian. Every one of them is European.

• faster-whisper-large-v3-turbo covers 100, including Cantonese, Hindi, Japanese, Korean, Swahili, Tagalog, Thai, Turkish, Urdu, Vietnamese, and Yoruba.

For a team serving Southeast Asian, South Asian, African, or East Asian markets, that table is the whole planning exercise. Speaker attribution is already solved by the same model you run everywhere else. Word-level accuracy is the constraint to design around, and the lever is the transcriptionConfig.model field.

Building on it

A single call to POST /v1/diarize runs diarization and transcription together and returns speaker-attributed output. STT Orchestration is available on Precision-3 and Precision-2, and it handles reconciliation between transcript segments and speaker turns for you.

json

{
  "url": "https://example.com/multilingual-call.wav",
  "model": "precision-3",
  "transcription": true,
  "transcriptionConfig": {
    "model": "faster-whisper-large-v3-turbo"
  },
  "exclusive": true,
  "speakerProbability": true
}

Three things are worth carrying into a multilingual deployment.

Pick the transcription model for your markets, not your defaults. Parakeet is the default and it is European-only. Any catalog reaching beyond Europe wants faster-whisper-large-v3-turbo. This is the single highest-leverage line in a multilingual config.

Exclusive mode simplifies alignment. Setting exclusive to true adds a second diarization track without overlapping speech, alongside the standard output. Alignment with speech-to-text becomes far more predictable against that clean track, which saves a lot of downstream merging logic on interruption-heavy multilingual meetings.

Use frame-level probabilities to route human review. Precision-3 can return speakerProbability, speechProbability, and crosstalkProbability every 20 ms, answering three separate questions: is this speaker active on this frame, is speech present at all, and is there overlap. Each is a request-body flag, off by default. When a pipeline spans a dozen locales and your team cannot spot-check every language, routing low-probability regions to reviewers scales better than sampling at random.

Precision-3 also adds vadSensitivity and crosstalkSensitivity, both floats from -5.0 to 5.0 with a default of 0.0. They trade precision against recall on voice activity and overlap detection, which is useful when one market's audio is consistently noisier or more interruption-heavy than another's.

One note on timing. Precision-3 becomes the default model on October 3, and Precision-2 is deprecated on October 17. Integrations that explicitly set model: "precision-2" or read the confidence output need updating before then, since Precision-3 replaces confidence with the three frame-level scores above. turnLevelConfidence is unchanged.

Frequently asked questions

Does speaker diarization work in any language? Yes. Our diarization models are language-agnostic and need no retraining per language. They model vocal characteristics rather than words.

Do I need a separate diarization model per language? No. There is no language parameter on the diarization endpoint and no per-language model variant. One model covers every language you process.

Can diarization handle code-switching within a single sentence? Speaker embeddings capture pitch patterns, formant frequencies, and speaking style, all of which stay stable when a speaker changes language mid-utterance. The transcription layer is where language coverage becomes a constraint, and it is configured separately through transcriptionConfig.

Which transcription model should I choose for non-European languages? faster-whisper-large-v3-turbo. The default, parakeet-tdt-0.6b-v3, covers 25 European languages only.

Does language-agnostic mean equally accurate everywhere? Our benchmark sets include Mandarin, French, and English corpora directly, and an independent 2025 study measured us as the most accurate of five systems across English, Mandarin, German, Japanese, and Spanish. Accuracy on your specific audio is still worth measuring yourself.

The practical takeaway

Multilingual voice AI usually gets planned as a language-by-language rollout: a model per market, a benchmark per locale, a launch gate per language. Diarization sits outside that plan. The model that attributes speakers in your English recordings attributes speakers in every other language you serve, with no retraining, and it keeps working when a speaker switches language mid-turn.

That frees your roadmap to spend its language budget where language actually lives, on transcription quality, vocabulary, and domain adaptation. Start with the quickstart, or run a multilingual file through the API playground and look at what comes back.