Recently my agent has started saying 'what was that' at awkward/interrupting times to users. After reviewing discord, i see a few others have experienced this issue. @rocket @Higsos @jonathan lei Like @rocket , we started noticing this behavior after switching to using Azure OpenAI provider. If the team could look into this it does seem like a bug? We do have agents speaking in languages other than english, so interrupting a spanish convo with "What was that" in english can be quite jarring. The timing is also somewhat problematic, often while the user is speaking. Since its not appearing in the model messages I'm guessing this is some sort of filler phrase coming from somewhere? @Sahil @Shubham Bajaj - hoping someone on your team can take a look or let us know if there's a fix/config option to address this?

Hi there, Thank you for your message. Our team is currently out of the office. We operate Monday through Friday, from 9:00 AM to 8:00 PM Pacific Standard Time (PST). We’ll get back to you as soon as possible during our normal business hours. If your message is urgent, please mark it accordingly or include “URGENT” in the subject line, and we’ll do our best to respond promptly. Warm regards, Vapi Customer Support Team

Upvote! Also Having this problem, using Azure OpenAi and talking in german. Nothing to see in the transcripts..

@Amir76 @User - I saw a posts in which you also mentioned seeing this "What was that" issue. did you find a solution or get any feedback from Vapi on it?

thank you for the call ids. we are looking into it

Hi @kyle , thanks for responding. For reference, i do see the "what was that" as model output in the logs but given the observations of others and that this issue started after switching to Azure Openai (without any associated prompt changes) - it very much seems like it has something to do with that. ie, if its happening randomly in the middle of a german or spanish conversation it doesn't make sense that its coming from the model. This spanish call shows that particularly: 731f17cf-abc2-4031-8278-748f85325dc9

Thank you for providing this information. We’ll be looking into it further to better understand the situation and determine the next steps. Our team will update you as soon as we have more details, and we truly appreciate your patience and cooperation in the meantime.

thank you for opening up this support ticket @Alex QS

We completely understand your concern. Our team is still investigating this, and we’ll keep you updated so we can assist you accordingly.

any update on this?

Updates?

We’ll keep you updated the moment we have new information, and we’re dedicated to making sure this gets resolved for you.

Dear Vapi Support, I hope that message finds you well und you had a fantastic day so far . Unfortunately i have the same problem @kyle @Sahil @Shubham Bajaj

I tested a few scenarios in Azure AI Foundry, and there is no difference between Global Standard Deployment and Datazone Deployment, it still randomly says, "What was that?" @User

If you look at the logs, it does show up when I've looked which I find interesting. Since it only seems to happen with Azure, I'm assuming it's somewhere in the Azure OpenAI integration within Vapi's stack, and may have something to do with latency/timing? definitely feels hardcoded

We really appreciate these additional information you have provided, we will be with you as soon as we have news.

01999e90-8a7e-7ddd-9e4a-299e268eb7a3 Here is a good example, The AI Said "What was that" 5 Times in a row after the user input..

thank you

Thank you, we will be with you as soon as we have news.

Is there any news on this?

So we re-enabled Azure OpenAI for the last 2 days, the 'what was that?' is still happening. this call was particularly bad: 0199eefa-cc9f-755e-92c0-a4da2da37ae8 definitely seems to happen in clusters? ie some calls not at all, some multiple times. Maybe that's a sign its latency related?

vapi team, please fix this we are currently more then 7 people with the same problem and this support ticket was opend nearly 2 Months ago.

In this specific case, Lea from 11labs is a multilingual voice which means it could switch over at any time when strict guardrails are not put in place. To prevent this from happening: - use a language specific voice id instead of a multilingual one - lower the temperature of the assistant to 0.4 or less

@kyle we've seen this issue happen with other voices as well, including the Vapi voices. We are using a specific voice ID for that call, voiceId="Zdsf4NBMlHR5zJJ72y9q", and use the multilingual model because the agent is bilingual english/spanish. I doubt the other's in this channel are using that same voice settings. And as this is happening to others in various languages i really don't think its coming from the voice model. this one the AI is speaking sensibly then the sentence seem to get interrupted with what was that: 56933142-ffcf-400d-9008-73fe745de779 - model sends a normal response, but then at the end of the response: 4:16:49:553 (+01:09:760) [LOG] Model output: What was that? this one is Jordan's Vapi Voice: f8685f4e-2b12-4540-a22e-30178b415c9e - the conversation is in spanish, the model is consistently responding in spanish... - then randomly: - 12:22:31:446 (+04:09:551) [LOG] Model output: What was that? - then a normal spanish reply on the next step... The fact that its happening for all of us different clients, prompts, presumably models... In the logs I always see it as a single chunk exactly "Model output: What was that?" - regardless of the language/context of the rest of the response/chunks. Seems like a hard coded string getting injected into the stream of chunks somehow? Maybe the logs are misleading and its not actually coming from the model

I tried both of these and yes it helps but the issue still occurs: use a language specific voice id instead of a multilingual one lower the temperature of the assistant to 0.4 or less I think it's a problem within Vapi. I tried it on Retell and it works perfectly fine.

This was my basic Setup from the beginning, Temperature was 0.3 and the Voice from eleven labs was a specific for German, still happend (multilingual off)

Good find!!!

maybe you can also provide your call logs

If possible, please share your call logs so we can investigate further.

all the call IDs are already in the chat

@rocket Maybe double check? I don't see any IDs that you've shared/posted. I only see callIds from myself and @Higsos

https://storage.vapi.ai/019a270a-c4d7-733f-bf93-1eacb63ad87b-1761591768953-467b6aeb-152b-4cc1-9017-7887c39919e9-mono.wav 1:25 - 1:35

we have issue the same issue at email collecting frequently

Is there anything at all on this?

I think overall we've noticed the frequency decrease somewhat but it's still happening. To me that seems like its latency related

It needs to happen zero times

Maybe an interesting question: Which Region did you guys setup for Azure Openai?

GPT-4.1-2025-04-14 Data Zone deployment in the region france central

i was also using: GPT-4.1-2025-04-14 but sweden central

It has now been more than 2 months since this ticket was opened, and more than 10 people still have not received a single serious response. VAPI team, **please **forward this to your engineering team. @kyle @Sahil @Shubham Bajaj

we are looking into it and we are going to run some tests on gpt-4.1-2025-04-14 on azure to reproduce the issue

Okay please inform is on the results when the tests are finished

Saved it in my notes. Thank you

This issue had SEEMED to reduce a bit for us, but we're seeing it frequently again. 019b05a1-3d6d-7dd7-8b25-3dc83a9cd6f7 as an example Given this convo is in spanish, i do not believe at all that the llm is outputting this "what was that?" string. So vapi's reporting/logs seem to be misleading here... It's been 3 months that this thread is open - would be great if your team could actually look into it/fix this bug

Thanks Alex. I didn't have time yesterday but hopefully will have some time tonight. Will keep you posted.

Hey all - did we ever figure this out or figure out any workarounds anyone?