We got an email announcing the Google Gemini transcriber, with "enhanced multilingual speech recognition". Which languages does it work with in multilingual mode? It only seems to understand English and Spanish. What about French?

@radu The Google Gemini transcriber supports a wide range of languages. In the

GOOGLE_LANGUAGES

array, we can see that 'Multilingual' is listed as the first option, followed by 38 specific languages: 1. Multilingual (as a general option) 2. Arabic 3. Bengali 4. Bulgarian 5. Chinese 6. Croatian 7. Czech 8. Danish 9. Dutch 10. English 11. Estonian 12. Finnish ...more we can see how the multilingual mode is handled in the code:

typescript
if (language === 'Multilingual') {
  base += `You can understand and transcribe audio in any language.`;
} else {
  base += `The language of the audio is ${language}.`;
}

This shows that when 'Multilingual' is selected, the system is instructed to "understand and transcribe audio in any language." To answer your specific question about French: Yes, French is officially supported in the Google Gemini transcriber as shown in the list of supported languages. It's listed as one of the specific language options you can select. Your observation that it seems to only understand English and Spanish in multilingual mode may indicate a limitation in the actual implementation or performance of the multilingual feature. While the system is designed to handle multiple languages in multilingual mode, the actual recognition capabilities might vary or be stronger for certain languages like English and Spanish.

It doens't work at all in French, when set as multilingual. That's why I am asking if there 's a problem in its description.

That's a model problem. Most likely, it doesn't work well with French transcription. I will suggest you try using Azure once I have heard that Azure works pretty much good.

Azure does work with multiple languages either. I tried it again. I opened this support ticket because Vapi sent an email telling us about Google Gemini transcriber's support for "enhanced multilingual speech recognition". Have you guys tried it out?

To help track down this issue, could you share: - The call ID - When exactly this happened (the timestamp) - What response you expected to get - What response you actually got instead This would really help us figure out what went wrong!

@radu Upvote here for deepgram multilingual feature. https://roadmap.vapi.ai/feature-requests/p/deepgram-nova-3-multilingual-released

let me check with the team.