https://vapi.ai/ logo
Join Discord
Powered by
Gladia hardstuck on "manual" language behaviour
# support
h
Despite setting to "automatic multiple languages" language behaviour, gladia is still using "manual" , i tested and checked gladia's language behavior on their site , and checked the same language behaviour through vapi, and it's very different, no matter what language behaviour i set gladia to through vapi, it's behaving like the "manual behaviour" on gladia's site, it not properly switching to the desired language behaviour Please fix this urgent @Shubham Bajaj
v
message has been deleted
@HystericalBat I'm sorry to hear that you're experiencing issues with the language behavior in your voice assistant. It seems there might be a discrepancy between the settings on Gladia's site and the behavior you're observing through VAPI. To address this issue, I recommend checking the voice settings in your VAPI configuration to ensure that the correct voice supporting the desired language is selected. Since the language configuration in the voice section is deprecated, you should directly choose a voice from your voice provider that supports the language you want to use. If you've already done this and the problem persists, it may require further investigation. In that case, please provide me with additional details such as the specific languages and voices you are trying to use, and any error messages you might be receiving. This will help me assist you more effectively. If you need immediate assistance and the issue is critical, please reach out to our support team for more in-depth troubleshooting.
You can continue this conversation by mentioning me in the message.
Got it. Tagging @User for additional help.
s
Hey @HystericalBat To help track down this issue, could you share: - The call ID - When exactly this happened (the timestamp) - What response you expected to get - What response you actually got instead This would really help me figure out what went wrong!
z
Hi y @HystericalBat how did you set automatic multiple languages" language behaviour with Gladia on Vapi? I indeed saw that Gladia offers real time language detection and that would be great if we can get it, there s a huge expectation for offering multilanguage voice assistants.
s
@HystericalBat / @zamzam TLDR: mutli lingual transcriber is not working as per expectation's As of now the multi lingual feature by these transcribers isn't working as expected, Gladia is bugging whereas Talkscriber is currently not focusing on mutli-lingual, sadly codeSwitchingEnabled with deepgram isn't fine to use it for production. Instead you have to use language based assistant based on the call origination at telephony level, depending upon telephony used. I am really sorry guys this feature isn't feasible as per your expectations. Do let me know how can I help you guys further with language based assistants or even anything else.
z
Understood we ll have to wait then; I just tested Gladia on their playground, and the results were quite good with a conversation in 3 languages, EN, ES and FR. Thank for the feedback
h
call ID : 72ef842d-b011-4709-9637-5208ff33a9ce timestamp: 11/20/2024 9:32:38 PM, IST Expected response : مرحباً، تاج إلى مساعدة في حجز غرفة Actual Response : Hello, I need help in booking a room. Eh? when i visited gladia's playground, when i set language behaviour to automatic multiple languages , it is giving the expected response , it is only giivng the actual response when i set to manual. Through vapi i set up automatic multiple languages, but clearly working like its on manual
well its not working for me as of now lol
s
logs
🔵 16:02:34:394 Initializing Gladia. Options: { "x_gladia_key": "", "encoding": "WAV/PCM", "bit_depth": 16, "sample_rate": 44100, "language_behaviour": "automatic multiple languages", "language": "english", "transcription_hint": "", "endpointing": 100, "model_type": "accurate", "frames_format": "base64", "prosody": true, "audio_enhancer": true, "word_timestamps": true } @HystericalBat this is what we are sending to Gladia, let me check with gladia team why it's not working as expected. Allow me sometime to get back to you.
h
Sure 👍
c
.
s
@clumsy.ninja I have replied to your ticket as well.
c
Yes I just wanted to follow the thread so msged, Any update by the way?
s
Allow me sometime ⏩
c
Sure, thanks for the help..
h
@Shubham Bajaj what api end point are you using for gladia ? the pre-recorded one or live or transcription ???
@Shubham Bajaj i tried implementing gladia api from my end and it is working fine
in here it says for automatic multiple languages u have to include : { "language_config": { "languages": [], // You can now specify the expected languages in V2 as guidance to improve accuracy and latency "code_switching": true   } } and for automatic single : { "language_config": { "languages": [], "code_switching": false   } } and for manual : { "language_config": { "languages": ["en"], "code_switching": false   } }
i made a repo, u can check it out , i am assuming you used live endpoint as well, so in the readme file i mentioned how to use it
when it is arabic, it is also giving output in arabic
and for english, same
c
@Shubham Bajaj @User
@Shubham Bajaj It's working for @HystericalBat
@Shubham Bajaj
@User
its working for us, we have provided details on how to set gladia up to make multilingual work properly, please have a look
s
@clumsy.ninja@HystericalBat there is a difference this code makes an http request to the gladia api with a audio file whereas we use wss. Transcribing realtime audio and Pre-recorded audio has difference. Good part is we have corrected our code for mutli lingual behavior now you only require to pass language bheaviour for mutliingual and don't pass the language along with it and it work. Previously we were sending english only and it was flaw in our system. Do not pass or set the language along with languageBehaviour, else it will be set to english. Do let me know guys how it goes!!
Copy code
json
{
  "transcriber": {
    "provider": "gladia",
    "languageBehaviour": "automatic multiple languages",
    "model": "accurate"
  }
}
h
@Shubham Bajaj , Ye in the live folder, through http, we just initialized the session, then through websocket we sent the audio file similar to how we would send live from microphone as per their own docs
But thanks though, will give it a try
s
@HystericalBat do let me know how it goes, and thanks for correcting me :bow_down
h
@Shubham Bajaj thanks man!, for another assistant though, i tried playing around from the UI itself, and it accidentally set the language parameter, and i couldnt undo that, will you guys be making some changes from the UI side as well soon to not include language parameter ?
s
As of now it's doable using API but we will add it in future.
h
Sure 👍
z
HI, i ve been testing this and it seems to work, it s quite a good news! Although there is still latency. I am using tools. Another issue are that idle messages are in English. Is it possible to make them multilingual? Maybe i ll use this setting in a squad configuration, with language detection as main agent.
s
@zamzam can you create a new support ticket for your issue.
5 Views
PreviousNext
Powered by