custom STT,TTS, VAD problem
# supportn
Nathan
11/26/2025, 2:35 PMHey guys, i having an issue when integrating my custom transcriber Yandex, to Vapi.
The problem, Yandex apparently doesn't support 2 channels, it is mono. So i am splitting two channels into two individual STT streaming session to Yandex. I am not sure if this is the right approach. Plus, Yandex doesn't even provide SDK for streaming, it onyl offers v2 gRPS streaming single channel and v3. This is means that i am entitled to control the channels. I couldn't figure out, how to stop the spill over of audio. What I mean by that: when assistant is speaking it is audio is continiously is being trancsribed, but at that moment if customer starts speaking as well, my server can't distinguish it, so yandex will send a one giant mixed transcript. Well, the expected behavior for me is, agent should stop if user interrupts the agent, well since my TTS also going to be Yandex, i can cut the TTS and silence agent channel, whenever i detect energy increase in customer channel, but that doesn't stop the Vapi agent's LLM, this can lead to inconsistency between the understanding of the user and agent.
Please any of guys have a solution to this, help me!
v
Vapi Support Bot
11/26/2025, 2:36 PMThanks for sharing the details! To better understand your setup: Are you able to programmatically mute or pause the audio sent to the Yandex STT stream for the assistant channel when you detect user speech (e.g., via VAD or energy threshold), or is the audio always mixed before reaching Yandex? This will help clarify if channel-level control is possible in your current architecture.
Sources:
- [Voice pipeline configuration](https://docs.vapi.ai/customization/voice-pipeline-configuration)
- [Custom transcriber](https://docs.vapi.ai/customization/custom-transcriber)
n
Nathan
11/26/2025, 2:47 PMI did think about adding VAD / energy detection to pause the assistant channel, but the issue is deeper than that.
When I tested with two concurrent STT streams — one for the customer channel (Left) and one for the assistant channel (Right) — the Yandex STT still produced mixed transcripts.
What’s happening is:
The assistant channel STT session is always receiving audio.
When the customer suddenly starts talking, their voice is still present in the assistant channel’s audio, so Yandex transcribes both voices together.
As a result, the customer STT stream never gets clean audio, because by the time they speak, the assistant channel stream is already “contaminated”. Well, to be correct customer STT get no audio.
So even though I have two separate sessions, it doesn’t matter because Yandex only supports mono STT. I have to downmix each channel to mono before sending it. That means both streams are seeing the same blended audio unless I fully mute one channel.
That’s the core problem:
I cannot reliably isolate customer vs assistant audio before the STT step, because Yandex STT requires mono input and I can’t prevent spillover unless I disable one side entirely.
Nathan
11/26/2025, 3:36 PMI think i have realized my mistake, I was previously tried dual separate session for each channel method but i actually forgot to mute the corresponding opposite channels for sessions, so i was basically sending the audio to both sessions, and expecting it to work!
I ll be updating my progress here please any feel free to share your opinion!
Thanks !
v
Vapi Support Bot
11/26/2025, 3:41 PMMessage marked as helpful by @Nathan! 🎉
c
Chiranjeet Mishra
11/28/2025, 12:52 AMHi Nathan,
For setting up a custom Text-to-Speech (TTS) integration with VAPI, you can follow these key steps:
1. **Setup your Web Server**: Ensure your server can handle POST requests and return audio data.
2. **Authentication**: Use Custom Credentials in the VAPI dashboard for secure communication. Example configuration:
Copy code
json
{
"voice": {
"provider": "custom-voice",
"server": {
"url": "https://your-tts-api.com/synthesize",
"credentialId": "cred_tts_auth_123",
"timeoutSeconds": 30
}
}
}