https://vapi.ai/ logo
Join Discord
Powered by
Multilingual Voice Agent Not Responding in Caller’...
# support
k
Hi team, I’ve created a voice assistant in VAPI for a dental clinic, and I'm using a system prompt where Ellie (the agent) should automatically detect the caller’s language (French, Italian, Spanish) and respond in that same language. Here’s the relevant section from the system prompt: "Ellie automatically adapts to the caller's language. If the caller speaks French, respond entirely in French. If the caller speaks Italian, respond entirely in Italian. If the caller speaks Spanish, respond entirely in Spanish. If the caller speaks English or another language, continue speaking in English." However, even when I speak in French or Spanish during the call, the assistant keeps responding in English. It doesn’t seem like the language detection is working properly. Could you please guide me on how to ensure language auto-detection and dynamic response in the caller's spoken language works correctly in VAPI? Is there a specific setting, flag, or syntax I’m missing in the system prompt or the VAPI configuration? Thanks in advance!
v
To enable language auto-detection in Vapi, make sure your assistant uses a transcriber that supports multilingual recognition (like Deepgram or Google). In your assistant configuration, set the transcriber as follows:
Copy code
json
"transcriber": {
  "provider": "deepgram",
  "model": "nova-2"
}
This allows the transcriber to detect and transcribe multiple languages. Note: The system prompt alone is not enough—ensure your transcriber and TTS voices support the target languages for full multilingual functionality. [See more in the Vapi multilingual documentation.](https://docs.vapi.ai/customization/multilingual) Source: - [Vapi Multilingual Documentation](https://docs.vapi.ai/customization/multilingual)
k
v
To enable true multilingual detection, set your transcriber language to 
"multi"
instead of 
"English"
in your assistant configuration. For example:
Copy code
json
"transcriber": {
  "provider": "deepgram",
  "model": "nova-2",
  "language": "multi"
}
This allows the assistant to detect and transcribe French, Italian, and Spanish automatically. After updating, test again to see if the assistant responds in the caller's language. [See Vapi multilingual documentation for more details.](https://docs.vapi.ai/customization/multilingual) Source: - [Vapi Multilingual Documentation](https://docs.vapi.ai/customization/multilingual)
To enable true multilingual detection, set your transcriber language to 
"multi"
instead of 
"English (En)"
. Update your assistant's configuration as follows:
Copy code
json
"transcriber": {
  "provider": "deepgram",
  "model": "nova-2",
  "language": "multi"
}
This should allow the assistant to detect and transcribe French, Italian, and Spanish. Also, ensure your TTS voice supports these languages for proper responses. [See Vapi multilingual documentation for more details.](https://docs.vapi.ai/customization/multilingual) Source: - [Vapi Multilingual Documentation](https://docs.vapi.ai/customization/multilingual)
k
v
To enable multilingual detection, set your transcriber language to 
"multi"
in your assistant configuration:
Copy code
json
"transcriber": {
  "provider": "deepgram",
  "model": "nova-2",
  "language": "multi"
}
Also, make sure your TTS voice supports French, Italian, and Spanish—otherwise, the assistant may still respond in English even if the transcriber detects the language. [See Vapi multilingual documentation for more details.](https://docs.vapi.ai/customization/multilingual) Source: - [Vapi Multilingual Documentation](https://docs.vapi.ai/customization/multilingual)
Thanks for sharing your configuration steps. You’ve set the transcriber language to 
"multi"
, which is correct for multilingual detection. However, the assistant still responds only in English. Clarifying question: Can you confirm which TTS (voice) provider and voice ID you have set in your assistant’s configuration? The TTS voice must explicitly support French, Italian, and Spanish for dynamic language responses to work ([see documentation](https://docs.vapi.ai/customization/multilingual)). If the voice only supports English, the assistant will default to English even if the transcriber detects another language. Example voice config for Spanish (Azure):
Copy code
json
"voice": {
  "provider": "azure",
  "voiceId": "es-ES-ElviraNeural"
}
Could you check and share your current voice settings? Source: - [Vapi Multilingual Documentation](https://docs.vapi.ai/customization/multilingual)
k
v
Vapi does not currently support a single TTS voice that dynamically switches between French, Italian, and Spanish. To achieve dynamic language responses, you need to programmatically set the TTS voice based on the detected language. For example:
Copy code
json
// Pseudocode for dynamic voice selection
if (detectedLanguage === "fr") {
  voice = { "provider": "azure", "voiceId": "fr-FR-DeniseNeural" };
} else if (detectedLanguage === "it") {
  voice = { "provider": "azure", "voiceId": "it-IT-ElsaNeural" };
} else if (detectedLanguage === "es") {
  voice = { "provider": "azure", "voiceId": "es-ES-ElviraNeural" };
} else {
  voice = { "provider": "azure", "voiceId": "en-US-JennyNeural" };
}
Update your assistant's configuration at runtime based on the detected language to ensure the correct TTS voice is used. [See Vapi multilingual documentation for more details.](https://docs.vapi.ai/customization/multilingual) Source: - [Vapi Multilingual Documentation](https://docs.vapi.ai/customization/multilingual)
Your current Azure voice (
es-ES-ElviraNeural
) only supports Spanish, so dynamic multilingual replies won't work. To enable automatic language responses, switch to a TTS provider or voice that supports all your target languages (French, Italian, Spanish) in one configuration—such as ElevenLabs Turbo 2.5, which is mentioned in the UI tip for multilingual support. Example voice config:
Copy code
json
"voice": {
  "provider": "elevenlabs",
  "voiceId": "eleven_multilingual_voice_id"
}
Update your assistant's voice to a multilingual-compatible option, then test again. [See Vapi multilingual documentation for more details.](https://docs.vapi.ai/customization/multilingual) Source: - [Vapi Multilingual Documentation](https://docs.vapi.ai/customization/multilingual)
k
@Vapi Support Bot This too is not working, can we transfer the call to another assistant not to the phone number in the VAPI ?
v
Yes, you can transfer a call to another assistant in Vapi without using a phone number. Handle the 
transfer-destination-request
webhook and respond with a payload like this:
Copy code
json
{
  "destination": {
    "type": "assistant",
    "assistantId": "your-target-assistant-id",
    "message": "Transferring you to another assistant."
  }
}
This will route the call directly to the specified assistant within Vapi. Source: - [Dynamic Call Transfers Documentation](https://docs.vapi.ai/calls/call-dynamic-transfers)
Based on the documentation, Vapi supports transferring calls to phone numbers or SIP URIs, but there is no explicit method to transfer a call directly to another Vapi assistant by referencing its assistant ID. To clarify your use case: Are you looking to transfer the call to another Vapi assistant instance (by assistant ID), or do you want to transfer to a SIP endpoint or phone number where another assistant is configured to answer? This will help determine the best approach for your scenario. [See Vapi dynamic call transfer documentation for more details.](https://docs.vapi.ai/calls/call-dynamic-transfers) Sources: - [Dynamic Call Transfers](https://docs.vapi.ai/calls/call-dynamic-transfers) - [Call Forwarding](https://docs.vapi.ai/call-forwarding)
k
v
Thanks for sharing the error details. To help resolve the "Invalid Destination" issue, could you please confirm: Are you using the exact assistant ID (not just the name) of the target assistant in your transfer tool configuration? You can find the assistant ID in the Vapi dashboard under the assistant's settings. Using the correct ID (not the display name) is required for the transfer to work if this method is supported. Let me know, and I can guide you on the next steps! Sources: - [Vapi documentation: call forwarding](https://docs.vapi.ai/call-forwarding) - [Vapi dynamic call transfers](https://docs.vapi.ai/calls/call-dynamic-transfers)
To resolve the "Invalid Destination. Assistant ... Not Found" error, ensure you are using the correct assistant ID (not just the name) in your transfer tool configuration. Replace the destination with your actual assistant ID from the Vapi dashboard:
Copy code
json
{
  "destination": {
    "type": "assistant",
    "assistantId": "your-target-assistant-id",
    "message": "Transferring you to another assistant."
  }
}
Double-check the assistant ID in your dashboard and update your transfer tool accordingly. [See Vapi dynamic call transfer docs for details.](https://docs.vapi.ai/calls/call-dynamic-transfers) Source: - [Dynamic Call Transfers Documentation](https://docs.vapi.ai/calls/call-dynamic-transfers)
k
@Vapi Support Bot { "type": "transferCall", "destinations": { "type": "assistant", "assistantId": "6b2e5119-d4fd-411d-87ca-4f09c89b5a62", "message": "Please wait while I connect you to the other assistant." }, "function": { "name": "transferToAnotherAssistant" } } i used this to create the function but it throws 400 error "message": [ "destinations must be an array", "each value in destinations.property assistantId should not exist" ], "error": "Bad Request", "statusCode": 400
v
The 
destinations
field must be an array of phone numbers or SIP URIs—direct transfer to another assistant by 
assistantId
is not supported. To fix the error, use a format like this:
Copy code
json
"destinations": [
  {
    "type": "number",
    "number": "+1234567890",
    "message": "Transferring your call."
  }
]
If you want to route to another assistant, configure that assistant to answer at a specific phone number or SIP URI, and transfer the call there. [See Vapi dynamic call transfer docs for details.](https://docs.vapi.ai/calls/call-dynamic-transfers) Sources: - [Dynamic Call Transfers Documentation](https://docs.vapi.ai/calls/call-dynamic-transfers) - [Call Forwarding](https://docs.vapi.ai/call-forwarding)
The "destinations" field must be an array, and only supports types like "number" or "sip"—not "assistantId". Here is the correct format for a transfer to a phone number:
Copy code
json
{
  "type": "transferCall",
  "destinations": [
    {
      "type": "number",
      "number": "+1234567890",
      "message": "Transferring your call."
    }
  ],
  "function": {
    "name": "transferCall"
  }
}
Direct transfer to another assistant using "assistantId" is not supported in the transferCall tool according to current documentation. [See Vapi transferCall documentation for details.](https://docs.vapi.ai/call-forwarding) Sources: - [Call Forwarding](https://docs.vapi.ai/call-forwarding) - [Dynamic Call Transfers](https://docs.vapi.ai/calls/call-dynamic-transfers)
a
Ensure you’re using a multilingual transcriber, configure language-specific voices, and optionally use code or webhooks to switch voices dynamically based on the detected language, your current prompt alone isn’t enough without these settings.. [https://docs.vapi.ai/customization/multilingual](https://docs.vapi.ai/customization/multilingual?utm_source=chatgpt.com)
PreviousNext
Powered by