We tested ElevenLabs Scribe v2 and Scribe v2 Realtime as the transcriber on a Vapi assistant doing live NZ inbound reception. Ran them back to back on 5 real phone calls and every single one had turn taking issues that never happen on Deepgram Nova 3. The worst was Scribe v2 Realtime. The agent spoke its entire 4 sentence scripted intro as one TTS burst at t=0 with zero gaps for the caller to answer, then sat on about 47 seconds of dead air before the first tool call. The caller's whole reply arrived in the same second as the bot's whole intro. Looked like the realtime endpoint delivered the user turn as one batch and the LLM fired mid burst. On regular Scribe v2 the pattern was different but just as broken. Our customer.speech.timeout hook (the "no rush, take your time" filler) fired while the caller was actively speaking, twice in some calls, because the HTTP per chunk model has ~700ms round trips and the idle hook can't tell the difference between "caller silent" and "transcriber hasn't POSTed the next chunk yet." Callers ended up saying "hello? hello?" and hanging up. Also got 26x 401 Unauthorized on transcript webhooks on one call, even though status update and tool calls webhooks on the same URL with the same HMAC secret delivered fine in the same call. Looks like the transcript webhook auth path is different when the transcriber is ElevenLabs vs Deepgram. Net: unusable for phone reception where turn taking matters. Rolled back to Deepgram Nova 3 same day. Curious if anyone else has it working for live inbound or if it's really just an async/post call transcriber.

For live inbound reception where turn-taking is critical, use a transcriber with built-in end-of-turn detection like Deepgram Nova 3. Here's a sample configuration snippet for your Vapi assistant:

json
"transcriber": {
  "provider": "deepgram",
  "model": "nova-3",
  "language": "en"
}

This setup ensures natural conversational flow and avoids the turn-taking issues seen with Scribe v2. For more details, see the [speech configuration documentation](https://docs.vapi.ai/customization/speech-configuration). Source: - [Speech configuration documentation](https://docs.vapi.ai/customization/speech-configuration)

It's worth to note that I have not had this problem on any other voice AI platforms

Yeah, you’re not crazy this lines up with what I’ve seen in production too. ElevenLabs Scribe v2 Realtime is marketed for live agents, but in practice it still behaves like a streaming-first transcription layer, not a tightly synchronized turn-taking system. That batching/commit behavior (plus VAD timing) is likely why you’re seeing merged turns and the agent firing mid-utterance, even though latency is ~150ms on paper The regular ElevenLabs Scribe v2 issue you described is also expected it’s fundamentally closer to a chunked HTTP pipeline, so your timeout hooks will always misfire unless you build buffering logic or delay thresholds around it. On the other hand, Deepgram Nova-3 is much more production-stable for telephony and real-time turn-taking, especially under noisy or interrupt-heavy conditions, which is why it feels consistent in your tests Short answer Scribe works, but right now it’s better suited for async or controlled flows. For inbound reception where interruption + timing matters, sticking with Deepgram is the safer call. If you want, I can sow you how to stabilize Scribe (buffering, VAD tuning, or hybrid fallback), but it’s not plug-and-play yet. @Jarod

Hi Rita, that would be great, although I do think my team have exhausted our efforts on everything Vapi has to offer. I would still be interested in connecting to see what you have to offer but it would be great to hear from some Vapi staff but it looks like they dgaf about general pop.

Okay @Jarod

Hi, could you please share some of the affected call IDs?

019d841f-f52b-7000-a18f-069c0a178052 scribe_v2 83s 019d841e-53d9-7000-97e8-1978c58e1c8f scribe_v2_realtime 93s ← most dramatic failure 019d841c-7420-7000-a183-a2e4fc260947 scribe_v2 95s 019d841a-1033-7000-a772-4b937877b4ae scribe_v2 134s 019d8419-b709-7000-a94c-69ad31d37807 scribe_v2 17s ← short test drop

Hi Jarod, Thanks for the call IDs.