Hi people, I’m experiencing problems with latency in my VAPI assistant. It is often 4x of estimated latency which is displayed in the dashboard (estimated is 875ms, acutal is 3-4sec). The higher latency is happening both during web calls (while testing directly in VAPI), and during calls via Twilio number.
It’s happening every single day since I started building a VAPI agent for my client (2 weeks), so it’s not a temporary issue.
Where can I look for cause of such an issue? Looking in logs (which I don't quite understand, except of numbers in turn latency) I see that often:
- Eleven Flash 2.5 TTS exceeds the latency (~500ms instead of declared 75ms) (I'm using a custom voice which I injected via the 11labs API),
- Azure transcriber exceeds it (~600ms instead of declared 250ms)
- LLM model exceeds it too (~800ms instead of declared 450ms).
I tried switching to a smarter LLM (claude 3.7 sonnet, 1000ms estimated latency), but it still exceeds the estimated latency at least 2x. I also switched the transcriber to speechmatics (100ms estimated latency), but then the total latency of my agent raises to 6-8sec.
For additional context, I'm building my agent in Polish language, I’m located in Poland.
Can that latency be a server location issue? Or does it have another cause? Example call ID: 15c9cea7-1575-490f-b52c-aeffbc024610
P.S. On the screenshot I provide an example log from a part of the conversation, on second screenshot there's an example turn latency value.
P.P.S. I'd appreciate any guidance on how to interpret the logs, it would be helpful for me to debug assistants on my own. I couldn't find any tutorial on that. I find these logs hard to read as user's and assistant's output are shown as small parts of the words
https://cdn.discordapp.com/attachments/1351514865679728671/1351514865990369290/image.png?ex=67daa7cd&is=67d9564d&hm=6d8b83ba1c092a59722b5a0ac9e49f1f88167d58a9c3f63596777a0e0aadf14a&
https://cdn.discordapp.com/attachments/1351514865679728671/1351514866254352415/image.png?ex=67daa7ce&is=67d9564e&hm=d27524a86718ddaed90aa576fdaf2b3588598a2ee07ffabd91d5808e03cb1306&