Good day, so Im trying to set up the custom LLM integration with my own server to send pre-determined responses (using AI only when necessary), but I cant seem to get the JSON format right for VAPI to read the responses. Ive looked through the documentation but found nothing, and the only available resource just re-dumps the response from the OpenAI API as JSON-Data. How can I create a custom JSON Response (Im using FastAPI) and return custom text + function calls? Thanks in advance

Hey, this usually comes down to matching VAPI’s expected schema exactly, not just returning raw JSON. In FastAPI, you’ll need to return a structured object with message, role, and optional tool_calls/function_call fields in the format VAPI parses. For custom replies, you can bypass OpenAI-style payloads and send a normalized “assistant” response with your text + function metadata. Most issues I see are missing keys or wrong nesting, so VAPI ignores the payload. Which VAPI endpoint/version are you using, and can you share a sample of your current response body? @LeoLion3

I dont know what the reaponse payload should look like. Ive tried a couple options from what Ive found on the developer forums. Is there no official documentation on this? Ive even gone as far as replicating a response from openai's completion api, it just doesnt get parsed by vapi:

python
@router.post("/chat/completions")
async def get_completion(request: Request) -> JSONResponse:
    if request.headers.get("content-type") != "application/json":
        return JSONResponse(content={"error": "Invalid content type"}, status_code=403)
    auth_header = request.headers.get("Authorization")
    if auth_header != f"Bearer {config.BEARER_AUTH_TOKEN}":
        return JSONResponse(content={"error": "Unauthorized"}, status_code=403)
    data = await request.json()
    message_obj:

# Custom, internal data model I use for easier processing. Irrelevant
ModelMessage = ModelMessage.to_obj(data)
    function_call_response = {
        "name": "end_call_tool",
        "arguments": {},
        "parameters": []
    }

    response_content = {
        "id": uuid.uuid4().hex,
        "object": "chat.completion",
        "created": int(uuid.uuid1().time),
        "model": "gpt-4.1",
        "choices": [
            {
                "message": {
                    "role": "assistant",
                    "content": "Hello world!",
                    "function_call": function_call_response
                },
                "finish_reason": "stop"
            }
        ]
    }

    return JSONResponse(content=response_content, status_code=200)

Yeah, this is a really common issue with VAPI, it’s not your FastAPI logic, it’s that their parser doesn’t fully follow OpenAI’s public schema. Your response is close, but small differences in nesting, metadata, and tool-call formatting can make VAPI ignore it. Most people who get this working end up reverse-engineering it from real traffic since the docs don’t cover it clearly. I’ve helped others align this properly so custom text and function calls are parsed consistently. If you want, we can discuss your setup and logs privately and fix this properly. @LeoLion3

@User could you please provide an expected server message json scheme?

Hi Leo, Here’s a very basic example request you can use (with

metadata.sendMode: "off"

on the Vapi side if you need the schema reference for building responses that include tool calling).

{
  "model": "your-model-name",
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant..."
    },
    {
      "role": "user",
      "content": "Hello, how are you?"
    },
    {
      "role": "assistant",
      "content": "I'm doing well, thank you!"
    }
  ],
  "temperature": 0.7,
  "max_tokens": 1024,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "book_appointment",
        "description": "Book an appointment for the user",
        "parameters": {
          "type": "object",
          "properties": {
            "date": {
              "type": "string"
            },
            "time": {
              "type": "string"
            }
          },
          "required": [
            "date",
            "time"
          ]
        }
      }
    }
  ],
  "stream": true
}

This should give you the expected structure for tool definitions. From there, your response payload (especially when streaming) needs to mirror the expected

chat.completion.chunk

format, including

delta.role

,

delta.content

, and/or

delta.tool_calls

depending on whether you're returning text or invoking a function. If you’d like, share your current response body and I can help align it exactly to what Vapi expects.