I'm evaluating using VAPI for a new integration with my company. I'm using the custom LLM integration. I'm testing a local integration that is being proxied through ngrok. I'm seeing the requests come into the custom LLM, but VAPI is not returning my responses to the user. When I test through the website voice or chat my response is returned from my server, but it is never parsed into something that the system will read. Am I missing a required result in the response? If I hit the custom LLM manually, this response looks correct. This also happens when using the chat so it's more than just the voice integration. An example call ID is:

0b0c9722-46d7-483a-9a05-27bd1a2a56b4

Here is an example JSON response from my custom LLM.

{
  "id": "chatcmpl-BXECyIN6BeM1HQKfpuI3KlhVCCaRI",
  "choices": [
    {
      "index": 0,
      "message": {
        "content": "I am designed to assist with a wide range of tasks, including answering questions, providing information, solving problems, and helping with decision-making. I can also interact with external tools to perform specific actions like creating tickets, fetching user information, and more. My goal is to be a helpful and efficient assistant for whatever you need.",
        "role": "assistant"
      }
    }
  ],
  "created": 1747259140,
  "model": "a6312579-10d5-4ff2-a576-1f44390c748d",
  "service_tier": "default",
  "system_fingerprint": "fp_55d88aaf2f",
  "object": "chat.completion",
  "usage": {
    "completion_tokens": 66,
    "prompt_tokens": 421,
    "total_tokens": 487
  }
}

It looks like your custom LLM is returning a static JSON object, but Vapi expects the response to be streamed using Server-Sent Events (SSE). Try streaming your response like this:

data: {"choices":[{"delta":{"content":"Your response text here."},"index":0}]}

data: [DONE]

Make sure your server sends each chunk prefixed with

data:

and ends with

data: [DONE]

. This should allow Vapi to parse and display your responses correctly ([documentation](https://docs.vapi.ai/customization/custom-llm/using-your-server)). Sources: - [Vapi Custom LLM Integration Documentation](https://docs.vapi.ai/customization/custom-llm/using-your-server) - [Vapi Discord Troubleshooting Discussion](https://discord.com/channels/1211482211119796234/1328028159949537341)

Hey, Could you please schedule a brief call at your convenience so we can discuss this matter in detail? Kindly use the following link to book a suitable time: .

Actually, the problem is what the chat bot thought it was. You're system is not sending an

Accept: text/event-stream

header which we were using to determine what to send back. When I forced it to stream events it works.

Glad you were able to find a solution!