Blog

Live-1: From Batch pipelines to real-time intelligence

In short: After 12 years of research and six years of work on streaming pipelines, beginning with Hervé Bredin's pyannote.audio and Juan Coria's DIART, pyannoteAI is launching Live-1, its first Streaming Diarization model. A production-grade model that delivers real-time speaker attribution with sub-300ms latency, designed for the noisy, multi-speaker conditions that voice-based solution actually runs in.

How streaming became mandatory in modern Voice AI stacks

For most of its history, voice AI pipelines were built around waiting. Audio came in, processing happened, results came out. That model worked when the use case tolerated latency: transcribing a recorded meeting, indexing a support call after the fact, and captioning an uploaded file.

That tolerance is gone. Voice AI now lives inside customer service platforms, sales workflows, meeting tools, wearables, and AI companions. The pipeline is no longer processing a file. It is participating in a live conversation. Users interrupt, overlap, and expect the system to keep up. A voice agent that pauses for three seconds after every turn does not feel slow. It feels broken.

This forced a structural change. The standard speech-to-text, language model, and text-to-speech stack had to become a continuous pipeline, with each component streaming its output to the next. When STT, LLM, and TTS all run incrementally, a single batch-mode component becomes the bottleneck that undermines the whole experience.

Real-world audio makes this harder. Production voice AI runs in contact centers, open-plan offices, live events, and mobile environments. Background noise, reverberation, overlapping speech, and variable recording quality are not edge cases. They are the norm. Any component designed for clean audio will degrade in the field.
Streaming is no longer a differentiator. It is the baseline for a voice AI stack that works in production.

Why diarization is a foundational layer of every Voice AI stack

When a voice AI system processes a conversation, the first question is not what was said. It is who said it.

Speech-to-text returns a flat word stream. It is accurate at the word level but has no model of conversational structure. A multi-party conversation run through STT alone produces an undifferentiated transcript: technically correct, structurally incoherent. The system cannot tell a question from an answer, an agent from a customer, a decision from a clarification.

Every downstream component depends on that structure. A language model cannot track who committed or whose turn it is to respond. An analytics engine cannot measure talk time, interruption rate, or speaker balance. A compliance tool cannot verify that the right party made a required disclosure. A routing system cannot act on speaker identity it does not have.

Diarization restores that structure. When each segment of audio carries a speaker label and a timestamp, the stack gains the context it needs: who spoke first, who responded, how long each participant held the floor, where turns changed, and where speakers overlapped.
The challenge is doing it in real time. A batch model that needs the full recording before producing output cannot serve a live pipeline. Speaker attribution has to arrive at the same latency as transcription. Without it, the rest of the stack is blind until the conversation ends.

What brought pyannote to build a new streaming diarization model

The decision was not made in isolation. It was the convergence of clear customer demand, a market shift among STT/TTS providers, and a research foundation that made the right approach tractable.

Over the past two years, customer and prospect conversations surfaced the same requirement: accurate streaming diarization for production voice AI. The use cases vary, from contact center agents to real-time meeting transcription, live broadcast, and on-device deployment for wearables, but the need is the same. These teams process live audio and need speaker attribution that keeps up with the conversation. Several design partners specified sub-one-second latency. Others needed support beyond the two-party assumption built into most existing approaches.

At the same time, the STT market moved decisively toward real-time, with most providers launching or expanding streaming transcription. Voice AI platforms build on these streaming STT layers. The result is a stack where every component runs incrementally except, in many cases, diarization. Speaker attribution either arrives too late to use or is dropped entirely, leaving the stack without conversational context.

The pyannote story: from pyannote.audio to Live-1

For pyannoteAI, this was the logical next step, not a new problem. The work is a sequence of steps, each building on the last. In March 2016, Hervé Bredin trained the first neural networks for speaker diarization. That research produced pyannote.audio, released in November 2020, which became the most widely used open-source speaker diarization toolkit in the world: over one billion downloads on Hugging Face, 10,000 GitHub stars, and more than 275,000 users. It established the architecture (speaker segmentation, embedding, and clustering) that defined the state of the art for offline diarization and became the baseline every later model is measured against.

In December 2021, Juan Coria, now pyannoteAI's CTO, released DIART, the first rigorous open-source framework for low-latency online speaker diarization. It brought streaming diarization to engineers for the first time and was adopted widely because nothing comparable existed.
pyannoteAI was founded in April 2024 to bring this research into a commercial product. In September 2025, Precision-2 launched as the most accurate speaker diarization model on the market, setting the batch-quality ceiling the streaming model would need to match. Live-1, our streaming diarization model, launched in July 2026. It is the result of twelve years of research on speaker diarization and six years of work on the streaming pipeline, engineered for the conditions where production voice AI actually runs.

What Live-1, the streaming diarization model, does?

Live-1, the streaming diarization model, processes live audio continuously and returns speaker attribution as the conversation unfolds, without waiting for the recording to end.

The primary integration pattern runs alongside a streaming STT provider. The same audio stream goes to both: STT returns word-level transcription, Live-1 returns speaker events. Reconciling the two outputs by timestamp produces a live, attributed transcript where every word is labeled with the speaker who said it. This is the structured metadata that LLMs, analytics engines, compliance tools, and routing systems need to work correctly.

It also enables use cases that batch diarization cannot support at all. A routing system can redirect a call the moment a new speaker is detected, not after it ends. A meeting tool can display attribution live as people speak. A voice agent in a multi-party conversation can track which human it is responding to. In each case, the value of attribution depends on its arriving in time to act on.

The model handles up to eight speakers and supports up to ten parallel streams per user in the current release. It keeps speaker labels consistent across the full duration of a stream, so a speaker identified in the first minute receives the same label ten minutes later.

Technical details and integration

Live-1 connects to a pipeline over WebSocket. The input is 16 kHz mono audio, delivered in 100-millisecond chunks. The output is a continuous speaker timeline: for each speaker event, the API returns the start time, end time, and speaker label, all updated in real-time.

The architecture processes audio natively in streaming mode rather than adapting a batch model to incremental inference. Chunks of roughly 100 milliseconds are processed as they arrive, and a speaker tracking layer maintains state across them, holding consistency without access to the full conversation. This removes the redundant recomputation that made DIART's sliding-window design expensive, and it is what lets the model reach sub-300-millisecond end-to-end latency while supporting multiple concurrent streams on shared infrastructure.

Integration needs a WebSocket client and handling for two event types. A diarization_speaker_start event fires when a speaker begins a turn, carrying the speaker label and start timestamp. A diarization_speaker_end event fires when the turn ends, carrying the end timestamp. Both are emitted continuously and can be consumed without waiting for a complete result.

The model is built for the conditions production deployments actually meet: background noise, overlapping speech, variable microphone quality, and conversations with more than two participants. It was retrained on audio selected for real-world complexity rather than clean benchmark conditions, and validated against those conditions before release.