Sometimes Vapi takes a while to respond
# support
p
I have an issue where on occasion vapi takes a while to respond, i think it has to do with gpt4o being slow at times switching to gpt4 turbo seems to resolve it when this happens. But is there a config i can use to use the fallback model? after x seconds of waiting for gpt4o?
v
Hi @Papi, I'm an AI assistant for Vapi. I'll analyze available sources and get back to you shortly.
----------------------------- **While I generate an answer, here are some sources that might be helpful**: * [Update Assistant]() * [Get Assistant]() * [Delete Assistant]() * [Create Assistant]() -----------------------------
### Answer Currently, the official Vapi documentation does not provide specific details about configuring a fallback model that automatically switches based on response time thresholds from a primary model such as GPT-4o to an alternate like GPT-4 Turbo. However, implementing a timeout and fallback mechanism at your application layer might be a feasible solution. You could set up your system to monitor response times, and if the primary model's response exceeds your defined threshold, you can programmatically switch to a faster model for subsequent requests. For direct support and more personalized guidance on possible configurations or updates that might not be documented yet, you might consider reaching out to Vapi's support. They could provide the latest updates or workarounds that are not yet covered in the public documentation.
Got it. Tagging @User for additional help.
s
We auto switch to the fastest model based on the latency. Can you send me a call_id along with the audio timestamp where you faced the delay?
p
250f6bd1-de79-41a1-b246-622f2aff7302 no timestamp cause it's the whole call. It took 30 seconds to respond. Also note this is a test call and the script is a little weird . hah . Testing "emergency" call forwarding rules . (not 911 emergency but like angry karen emergency)
@Sahil
s
It is fixed.
Can you give it a try once again and lmk
p
It's not super consistent but will report back if i expierence it again thank you so much sahil
s
Welcome 🙂
p
Sahil this is still an issue ea2a3328-49cb-43c5-9fe7-3bc28d103583 No timestamp it's the whole call. Basically delayed for the first 15-20 seconds then it repeats the same question over and over again . Not a prompting issue i suspect something to do with the delayed response
@Sahil
just made another call and it's doing the same thing .
30 second to the first response
d6f80b04-b9ef-47d6-9872-ff26daa3dfc5
this is taking 30 seconds per response .v
3rd call is working nromally
gotta figure this out before i go in to prod
Yeah it's hit or miss sometimes i get on a call where it takes 30 seconds per response but then i call back and it's fine. I think maybe im getting stuck on a bad azure endpoint? I use azure openai for some other stuff and notice their endpoints are way slower lately.
im planning to go live at the beginning of june, keep me posted on this and ill be happy to do more testing.
s
@User Can you please create a new support request after one is completed because it gets confusing.
2 Views