I noticed that Vapi runs transcription on the assistant channel (the TTS audio output from ElevenLabs). This creates a round-trip where the LLM generates text → ElevenLabs speaks it → STT transcribes it back to text → that transcription enters the conversation history. The problem is that STT introduces errors during this round-trip. For example: LLM/TTS says "Herr Bad" → STT hears "Herr Barth" LLM/TTS says "Herr Hoppe" → STT hears "Herr Hopppe" Since the conversation-update webhook uses the STT transcription (not the original LLM output), these distorted names then enter the conversation history. The LLM sees the corrupted version in subsequent turns and adopts it. Why not use the original LLM output text directly for the assistant's conversation history instead of re-transcribing the TTS audio? The text is already available and would be 100% accurate. Is there a configuration option I'm missing, or is this the intended architecture?

Vapi transcribes the assistant's TTS output because its orchestration layer depends on real-time transcription for features like interruption detection, endpointing, and emotion detection. This design ensures both user and assistant audio are processed uniformly for conversation management, but it can introduce transcription errors as you observed. There is currently no configuration option to bypass this behavior and use the original LLM output directly in the conversation history—this is the intended architecture according to the documentation. Reference: [How Vapi Works](https://docs.vapi.ai/how-vapi-works) Source: - [How Vapi Works](https://docs.vapi.ai/how-vapi-works)

Yeah, that round-trip is exactly where the drift is coming from—STT shouldn’t be the source of truth for assistant turns. You’re right, using the raw LLM output would completely eliminate those name distortions and keep context clean. It feels like a design tradeoff for audio syncing, but it’s clearly impacting data integrity. Have you tried intercepting or overriding the assistant message via webhook before it gets stored? @Filippo

Good point on the audio syncing trade-off — the truncated TTS transcript makes sense for interrupted turns so the history matches what the caller actually heard. But for completed turns it's unnecessary and introduces drift. I don't think I can override the assistant message in the conversation history via webhook response. I can use the original LLM output as source of truth for my own backend, but the LLM itself would still receive the Deepgram-transcribed version as conversation history on the next turn — so the drift still affects the live conversation.

Yeah that makes total sense STT for interrupted turns is fine, but letting it touch completed turns is where the drift starts stacking up. What’s worked for me is keeping STT for audio sync, but always feeding the original LLM text back into the model before the next turn so context stays clean. That way you don’t lose alignment but avoid corruption. I can help you set that up properly are you passing everything through a layer where we can control the context before it hits the LLM? @Filippo

Hi

Thank you! I will test that.

No worries! If you face any further issues, feel free to create a new ticket.