Hello,I am experiencing a critical LLM latency issue that appears to be caused by an unwanted automatic model fallback in the Vapi infrastructure. My Assistant is currently configured to use Llama 3 70B with the Groq provider (or previously GPT-4o Mini Cluster). However, for multiple calls, the system is consistently falling back to Gemini-2.5-flash (Google provider). This is resulting in unacceptably high LLM latency, frequently ranging around 3-4sec, making the real-time conversation unusable. Evidence: Configured Model: Llama 3 70B (Groq) Observed Fallback (Log Detail): The latest call log shows the Google model being used instead of the configured one. Call ID: 019a0b78-e67c-7118-8a9f-f0a9d26b60a4 Log Data: The log for this call explicitly shows "provider": "google" and "model": "gemini-2.5-flash".Could you please investigate why the system is failing to route to my configured Groq model and why it is instead defaulting to a high-latency Google model?Thank you for your urgent attention to this issue.

Hi Daniel, Thank you for sending the call ID. Based on the information from that specific call, here’s what we’re seeing in the configuration:

{
  "cost": 0.64165,
  "type": "model",
  "model": {
    "model": "llama3-70b-8192",
    "provider": "groq"
  },
  "promptTokens": 51278,
  "completionTokens": 490
},
...
{
  "cost": 0.0002247,
  "type": "analysis",
  "model": {
    "model": "gemini-2.5-flash",
    "provider": "google"
  },
  "analysisType": "summary",
  "promptTokens": 624,
  "completionTokens": 15
}

The

gemini

model you’re seeing is only being used for post-call analysis, not during the assistant conversation itself. Your assistant’s active LLM during the call was indeed Llama 3 70B (Groq). Also, due to how our platform is structured, if Groq were to fail to complete a request, the call would end with a pipeline error rather than falling back to a different model. Currently, only OpenAI is configured to support fallback models. You can find more details in our API documentation here: 👉 [Vapi API – Fallback Models](https://docs.vapi.ai/api-reference/assistants/create#request.body.model.OpenAI.fallbackModels) If you have any other questions or need further clarification, feel free to reach out — happy to help. Best, Kyle Vapi Support Team