Written by Juan Ignacio Álvarez Trejos, Research Team, pyannoteAI

If you're preparing audio for voice cloning, you want every overlap filtered out and every silent moment trimmed. And if you're feeding a transcription pipeline, you want every word kept; even the quiet ones, even the ones spoken over each other.

Speaker diarization systems depend on two upstream components that are central to the task: voice activity detection (VAD), which determines when speech is present, and overlapping speech detection (OSD), which identifies when multiple speakers talk simultaneously.

At pyannoteAI, the same diarization model Precision-3 exposes two parameters that let users shift the model's prior beliefs about speech activity and crosstalk directly at inference time, without any retraining.

In this post, we benchmark the two inference-time parameters: vad_sensitivity and crosstalk_sensitivity, across the 11 acoustic domains of DIHARD, and give you the settings to copy-paste into your own pipeline.

Key takeaways

• VAD quality is high across the board, but only Precision lets you move the precision/recall operating point at inference time.

• vad_sensitivity has a flat sweet spot between 0.0 and +0.8 for most domains; small changes are safe, big ones aren't.

• crosstalk_sensitivity has a cliff: stay between −1 and 0 for general diarization. Push higher only if recall on overlap is your only objective (e.g., cleaning a TTS dataset).

• No single configuration wins across all 11 domains. Optimal vad_sensitivity ranges from −0.8 (Restaurant) to +2.0 (Audiobooks).

• Inference-time tuning is now a real alternative to retraining when adapting to a new acoustic domain.

The two parameters in 60 seconds

Both parameters live on the Precision-3 model and accept a float in [−5.0, +5.0], default 0.0.

vad_sensitivity : Control the strictness of Voice Activity Detection.

• ↑ higher → more precision, fewer false positives (less non-speech detected as speech)

• ↓ lower → more recall, fewer false negatives (less speech missed)

crosstalk_sensitivity : Controls the sensitivity of overlapping speech (crosstalk) detection.

• ↑ higher → more recall, more overlap detected

• ↓ lower → more precision, only clear overlaps detected

Neither parameter exists in community-1 ; they're exclusive to Precision-3. In this post, we benchmark their effect across DIHARD to understand the tradeoffs and give practical guidance on how to set them.

How well does the VAD actually work?

Before talking about tuning, it's worth checking the raw VAD quality. We compare three systems on DIHARD: Precision, our open community-1 model, and Silero VAD (a strong open-source VAD baseline), at each system's best operating point.

Both pyannote models edge out Silero, with Precision ~0.8 points above Community-1. For Precision, the operating point can be shifted at inference time via vad_sensitivity, which adjusts the model's log-priors rather than applying a post-hoc threshold.

Which vad_sensitivity should I use?

The chart below isolates VAD-only error (MISS + FA, no speaker confusion) across the parameter sweep.

Suggested chart: line plot, x-axis = vad_sensitivity from −1.0 to +1.0, y-axis = MISS+FA (%), three lines (precision-2, community, Silero rescaled to [−1, 1]).

Three things stand out:

1. precision has a flat minimum between +0.4 and +0.8 (13.33%–13.39%). You can move within that band without consequence — useful when the right setting is uncertain.

2. community follows the same shape but with double the false-alarm baseline at negative aggressiveness values.

3. Silero only catches up at the cost of substantial missed speech — its FA at low thresholds is huge.

But the global sweet spot lies — the per-domain picture is much more varied.

Optimal VAD sensitivity varies by domain

The pattern is intuitive once you see it: clean, structured audio rewards aggressive VAD (Audiobooks, Maptask, Meeting), while noisy multi-speaker environments need permissive VAD to avoid swallowing real speech (Restaurant at −0.8). A single global default cannot be optimal everywhere.

What these domains map to in production

Most of you don't ship products in a "DIHARD domain"; you ship products in industries. Here's how the benchmark domains line up with the use cases we hear about most:

• CTS (Conversational Telephone Speech) → call centers, customer support analytics, sales call coaching

• Clinical → medical transcription, doctor–patient note-taking, telehealth

• Broadcast → TV, series, entertainment, sports commentary

• Court → legal proceedings, depositions, regulatory hearings

So if you're building a call-center analytics product, the CTS row (vad_sensitivity = +0.2) is your starting point. If you're tackling medical transcription, look at Clinical (+1.2). For TV/entertainment captioning, Broadcast (+1.4). For legal-tech, Court (−0.2) — note that legal recordings reward permissive VAD, because softly-spoken witness testimony is the kind of speech you most want to preserve.

The biggest wins of precision over community are in clean, structured domains where a better model can exploit turn-taking: Maptask (10.1% → 4.1%), Court (8.3% → 4.1%), Clinical (24.3% → 14.0%). The hardest domains (Restaurant, Webvideo, Meeting) stay hard for both, but precision-2 keeps a consistent lead.

Which crosstalk_sensitivity should I use?

Silero has no equivalent here — it's single-channel with no overlapping speech detection. So we compare Precision and Community only.

At their best operating points, Precision reaches 59.7% OSD F1 vs 48.2% for Community — an 11-point gap that reflects a real capability difference, not just tuning.

Suggested chart: line plot, x-axis = crosstalk_sensitivity from −5 to +10, y-axis = DER (%), two lines (precision-2, community), with shaded band marking the recommended [−1, 0] zone.

The DER curve has a clear shape: a sweet spot around 0 to −1, then a cliff. Push above +2, and DER degrades fast as the model floods the output with spurious overlap segments. Community collapses past +7 (DER over 100%). Precision degrades more gracefully, but the cliff is still there.

Settings recipes you can paste

# General-purpose diarization (default — start here)
vad_sensitivity = 0.0
crosstalk_sensitivity = 0.0

# Downstream transcription (don't lose any words)
vad_sensitivity = -0.4
crosstalk_sensitivity = 0.0

# Voice-cloning / TTS dataset prep (filter aggressively)
vad_sensitivity = 0.6
crosstalk_sensitivity = 2.0  # high recall on overlap → discard those segments

# Noisy multi-speaker audio (Restaurant-like)
vad_sensitivity = -0.8
crosstalk_sensitivity = 0.0

# Clean, structured audio (Audiobooks, Maptask)
vad_sensitivity = 1.2
crosstalk_sensitivity = 0.0

# STT × diarization reconciliation (minimize overlap to simplify alignment)
vad_sensitivity = 0.0
crosstalk_sensitivity = -1.0

Three rules to remember:

1. vad_sensitivity is forgiving — adjust in 0.2–0.4 steps; the sweet spot is wide.

2. crosstalk_sensitivity is not — stay between −1 and +1 unless you have a specific reason to push higher.

3. Tune per domain, not globally — even a few dozen labelled minutes are enough to find a better operating point than the default.

Why this matters

Modern diarization is shifting from static models to configurable pipelines. Architectures matter, but so does giving users a knob to adapt the same model to wildly different acoustic conditions: voice cloning data prep and noisy restaurant transcription should not require two different models.

vad_sensitivity and crosstalk_sensitivity are the first two of those knobs on Precision. We're working on more.

Try it yourself

Spin up Precision-3 with these parameters in the pyannoteAI playground, or read the precision-3 launch post for the full picture.
If you'd like to learn how we measure all of this, our evaluation guide for speaker diarization walks through DER, JER, and the rest.