Hello, We have recently been experiencing noticeably high latency during the first turn after the initial message in our calls using VAPI.AI. Interestingly, with the same assistant and identical AI service configurations, performance varies significantly from day to day — sometimes responses are fast, while on other days the latency is much higher. The delay is particularly noticeable right after the first message, where the response time exceeds 3 seconds, which negatively impacts the voice call experience. After the first sentence, the general conversation flow usually becomes much faster and does not show the same delay. Below are some example Call IDs from slower cases: Organization ID: 810ba100-0c63-4b0d-a27a-80861e4a3a8a Call IDs and Observed Latencies: 1. 2990ms – 3135ms Call ID: 019cbdc3-5181-7bbd-9a65-e9f1bee22d81 2. 3337ms – 3410ms Call ID: 019cbd04-2cea-7445-8b29-ffc818e4884f 3. 3066ms Call ID: 019c9efe-284d-722b-8191-32d5daf89040 4. 3728ms Call ID: 019cb352-667b-7ee5-aa67-2f6264386820 Could you please help us investigate the possible causes of this issue and advise on what we can do to reduce the latency during the first turn? We would appreciate your quick support on this matter. Thank you.

Thanks for the detailed examples, I can help troubleshoot the first-turn latency. Typically, delays like this stem from cold-start overheads, session initialization, or model warm-up variability. A good approach is to check whether the same AI endpoint and region are consistently used, and whether any request prewarming or caching is in place. Could you confirm if these calls are all routed through the same region and endpoint? @Yavuz Kömeçoğlu

I couldn't find the region and endpoint information in the call logs. Could you direct me to check it? They are working with the same provider and AI model. TTS - Elevenlabs Turbo2_5 (voice ID may change, model is the same) STT - Deepgram nova-3 (fixed) LLM - OpenAI GPT4o Cluster (fixed)

Since you’re using fixed providers and models, the variability likely comes from endpoint routing or cold-start behavior, which isn’t always obvious in standard call logs. Ican guide you on exactly where to check these details and the optimizations we can apply to reduce first-turn latency. Let’s discuss this more privately so I can walk you through the methodology step by step. @Yavuz Kömeçoğlu

I'm looking for your support regarding cold starts and first-turn delays. Thank you.

Hi, Thanks for sharing the logs. The first-turn delay is caused by a cold speech cache miss. When the cached audio for the greeting isn’t available in Redis, the system falls back to real-time TTS generation via REST, which is slower and explains the variability you’re seeing. We recommend three changes: 1. Set

firstMessageMode

to

"assistant-speaks-first-with-model-generated-message"

. This streams the first message through the WebSocket TTS pipeline instead of the slower REST path. Move your greeting into the system prompt instead of using the

firstMessage

field. 2. Switch the voice model to

"eleven_flash_v2_5"

for faster time-to-first-byte compared to

eleven_turbo_v2_5

. 3. Set

optimizeStreamingLatency

to

4

to enable maximum latency optimization on ElevenLabs’ side.

{
  "firstMessageMode": "assistant-speaks-first-with-model-generated-message",
  "voice": {
    "provider": "11labs",
    "voiceId": "5hMqNWDlm0o8hdIWxZq7",
    "model": "eleven_flash_v2_5",
    "stability": 0.5,
    "similarityBoost": 0.75,
    "useSpeakerBoost": false,
    "optimizeStreamingLatency": 4
  }
}

Give this a try and let us know how it goes.

Thank you so much for your detailed feedback and suggestions. I will try it and let you know again.