----------------------------- **While I generate an answer, here are some sources that might be helpful**: * [Custom LLM]() * [Call Ended Reason]() * [Get Call]() * [Connecting Your Custom LLM to Vapi: A Comprehensive Guide]() -----------------------------

### Answer It sounds like you're encountering an error related to the custom LLM setup in Vapi, specifically an issue after streaming completes. This might be linked to how your server handles streaming responses to Vapi. Vapi expects the custom LLM server to handle streaming responses correctly [(1)]() . To troubleshoot and potentially resolve the issue, consider the following steps: 1. **Validate Server Response**: Ensure your custom LLM server is configured to handle streaming. If your server setup doesn't support streaming responses, Vapi might not be able to process the output correctly, leading to failures post-streaming. 2. **Check Compatibility**: Verify that your server's response structure aligns with what Vapi expects. Typically, this involves formatting the responses in a JSON structure that includes fields like

id

,

object

,

created

,

model

, and

messages

[(2)]() . 3. **Logs and Error Messages**: Review the detailed logs from both your custom LLM server and Vapi's call logs. Look for specific error messages or indicators at the point where the LLM fails. This might give clues on whether the issue is with data formatting, response timing, or connection handling. 4. **Server Configuration**: Double-check your server’s endpoint configuration, ensuring it matches the specifications required by Vapi for streaming compatibility. This includes setting up appropriate headers, response formats, and handling of continuous data streams. Here’s a basic example of how your server might format a response to Vapi:

json
{
    "id": "42",
    "object": "chat.completion",
    "created": "timestamp",
    "model": "your_model_id",
    "messages": [
        {"role": "assistant", "content": "Response text here"}
    ]
}

If these steps do not resolve the issue, it may be beneficial to simulate the streaming interaction in a controlled environment to isolate the problem. Adjusting how your server handles the 'stream' flag in requests from Vapi might also be necessary. For further detailed guidance on setting up and troubleshooting custom LLM servers with Vapi, including streaming data, you can refer to the Vapi documentation on integrating custom LLMs [(3)]() .

Got it. Tagging @User for additional help.

Glad I could be helpful. Feel free to create a new thread with any new questions you may have.