Hey everyone! I've been trying to build a multilingual voice assistant for a few weeks now, and I keep running into roadblocks. The assistant itself works great for handling basic conversations and tasks — no issues there. The challenge starts when I need to handle conversations in different languages. The built-in multilingual handling doesn't seem to provide consistent results — the assistant struggles to maintain the logic, and the flow of the conversation breaks down. To solve this, I’m leaning toward using multiple dedicated agents, each tailored to a specific language. But here's where I’m stuck: I haven't been able to create a reliable "router" agent that can detect the language and forward the call to the right assistant. If I can't make this work, I'm considering falling back on a more traditional IVR approach (press 1 for X, 2 for Y, etc.), but I'd really prefer a smoother, more natural experience. So my questions are: Has anyone successfully implemented language selection via IVR in Vapi? Is it possible to build this kind of setup without relying on an external VoIP provider? Any best practices for linking agents together without repeating greetings or breaking the flow? I'd really appreciate any tips, advice, or example setups you might have. Thanks in advance! 🙏

Building a multilingual voice assistant, particularly in effectively managing language detection and routing like; Built-in Multilingual Support. Implementing IVR for Language Selection. External VoIP Providers, external VoIP providers like telnyx can enhance functionality, especially for complex routing and language detection scenarios. This tutorial can give you a guide

I'm getting quite good results (although not very stable, so a chunk of the calls might present weird conversations if not guided enough) with the following stack : - Google 2.0 Flash as transcriber. Deepgram nova-3 multi is also an option. - gpt-4o - 11labs voices

It works with popular languages, unfortunately I need Latvian language which doesn't work with any multilingual transcriptionist...

Did you try azure?

@AiTitus :) Yes, that's what I use, the question is how to direct to the right assistant using Squads if the main assistant can't correctly recognize the language? For me, choosing a language by button works fine, but if you connect through a third-party service, it's expensive, because redirection is used, and I couldn't set it up in Vapi, everything I tried didn't work...

Is there a default language that people speak in? If there are multiple hard to detect languages maybe add a language per unique phone number? Or you could use Twilio which allows people to put in numbers and then it would direct to a specific agent direct from Twilio using getAssistant from VAPI.

is it really that expensive to just have a person call an ivr system and have ivr transfer the number to the assitant based on what lauanugae they chose in the menu /dialpad

Also with OpenAI Realtime mini maybe it can handle this part?

Also might be a bit more expensive and technical but feel finetuning a model to detect the languages would be pretty fitting task for this.. especially since there is usually a common set of phrases people say at the start of the call

I think OpenAI Realtime is pretty good at sorting these things out

Unfortunately there is no default language, some speak one, others another... I considered a separate number for each language, but the problem is that Tvilio does not offer Latvian numbers, the service I use in Vapi, Zadarma, takes 0.21 cents for transferring to another number, and this breaks everything...

ok gothca ok well its still hard to have a reliable language detection since currently it depends on the transciber which you have to set ahead of time. What langauge do you use for the First Spoken Message?

I tried everything, the idea is that the Latvian language is not recognized by multilingual models, and if you set the LV transcription, other languages ​​are not recognized.

if you usesd lartvian you could use in your first message if you want to speak latvian say latvian then you transfer to a wsquad agent that speaks it