Hey guys, I have an existing chatbot that runs with an Azure OpenAI model and an Azure Vector Storage (RAG) to answer questions in a specific niche. What is the easiest way to transfer this capability to Vapi for allowing calls? I didn't see any option (in the UI at least) to integrate my Azure Vector Storage? In my chatbot I use the Responses API from Azure OpenAI with

file_search

(to query the RAG) - is it possible to do the same with Vapi? Help is much appreciated, thanks a lot!

Create a backend service that receives Vapi’s transcribed voice input, queries your Azure Vector Storage (RAG) via Azure AI Search, generates a response with Azure OpenAI, and sends it back to Vapi for voice output..

Yes, I figured to use Custom LLM. But in the web all examples are built with the Chat Completions API from OpenAI. But I would need to use Responses API and I failed to implement. not sure whether this is even supported?

Vapi supports using the OpenAI Responses API via a Custom LLM by setting up a custom endpoint that forwards requests to the Responses API and configuring your assistant to use that endpoint with the correct model.

As I understand you I have to configure my Assistant to work with the Responses API? Where can I do this? I don't find it.. 🙂

You can use the Response API by setting up a custom server to handle those requests and then selecting “Custom LLM” in Vapi’s model settings with your server URL and model details.

Yes I did that. But I am always running in an error. All tutorial use OpenAI's chat completions and I guess Response API returns a different format that VAPI cannot handle.

Vapi requires your custom server using the Response API to reformat and stream responses via Server-Sent Events in Vapi’s expected format, not the raw OpenAI format, to avoid errors. https://docs.vapi.ai/customization/tool-calling-integration

I don't find anything there regarding Vapi's expected format? Did you send the correct link?

Your server must respond to tool calls using Server-Sent Events with a JSON structure like {"results":[{"toolCallId":"X","result":"Y"}]} to ensure Vapi processes responses correctly.

But I don't wanna use a "Custom Tool" but a "Custom LLM"