We're currently using Custom LLM feature that points to our backend's /completion endpoint that then uses openai's Completions API. We want to migrate to OpenAI's responses API but are not sure if this'll just work with Vapi, or do we need to add some glue code to make sure that our backend is always returning a response that adheres to the Completions API response spec? Theres no documentation on what Vapi expects as a response for custom LLM.

You will need to add glue code to ensure your backend returns responses in the format Vapi expects, not just the raw OpenAI Responses API output. Here's a minimal example of how to adapt your endpoint:

python
# After calling OpenAI's Responses API:
openai_response = ...  # your OpenAI Responses API call
formatted_response = {
    "output": [
        {
            "role": "assistant",
            "content": [{"type": "output_text", "text": openai_response['output'][0]['content'][0]['text']}],
            "status": "completed",
            "type": "message"
        }
    ]
}
return jsonify(formatted_response)

Format your backend's response as above to ensure Vapi compatibility. For more details, see the [Vapi Custom LLM integration guide](https://docs.vapi.ai/customization/custom-llm/using-your-server). Source: - [Using Your Server (Vapi Custom LLM)](https://docs.vapi.ai/customization/custom-llm/using-your-server)

Vapi’s Custom LLM expects your backend to behave like the OpenAI Chat Completions API — both in how it receives the request and how it returns the response. The new Responses API has a different response structure, so if you switch to it, Vapi won’t understand the raw output. So you have two real options: Option 1 — Easiest (no risk) Keep using Chat Completions in your backend. This is fully compatible with Vapi today and requires no changes. Option 2 — Use Responses API (requires glue code) If you want to migrate to responses.create(), you need a small translation layer in your backend. What that means in practice: Call OpenAI using the Responses API Extract the final assistant text from the response Return a Chat Completions–formatted JSON response back to Vapi For non-streaming, your backend must return something like this: { "id": "chatcmpl-123", "object": "chat.completion", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "Generated reply here" }, "finish_reason": "stop" } ] } If you're streaming, you must stream SSE chunks in Chat Completions format, meaning: object: "chat.completion.chunk" choices[0].delta.content for tokens If you return the raw Responses API structure instead, Vapi won’t parse it correctly.

Hi, Vapi’s Custom LLM integration expects your endpoint to behave like the OpenAI Chat Completions API, both in request structure and, most importantly, response format (including SSE format if streaming). The newer OpenAI Responses API returns a different JSON structure, so it will not work if passed through directly. If you migrate to

responses.create()

, you’ll need a thin compatibility layer in your backend that: 1. Calls the Responses API 2. Extracts the final assistant text 3. Returns a Chat Completions–formatted response back to Vapi For non-streaming, your backend must return a payload shaped like: •

object: "chat.completion"

•

choices[0].message.role = "assistant"

•

choices[0].message.content = "<final text>"

For streaming, you must emit SSE chunks formatted as: •

object: "chat.completion.chunk"

• Tokens inside

choices[0].delta.content

If you return the raw Responses API structure, Vapi will not parse it correctly. If you want zero-risk and no additional glue code, continuing with Chat Completions is the simplest path. If you’d like the benefits of the Responses API, implementing a small translation layer will ensure compatibility.