hey, so I currently have no idea how to implement a custom model in Vapi. Because when I usually send post requests I do have to add an 'Authorization' part with Bearer Token into my headers. But I think it's not possible to add those in the vapi system right? So is there any workaround? This would be an important feature for me. Thanks

@OGOnkel Hello! I'm glad you reached out for assistance with implementing a custom model in VAPI. You're correct that typically, when making POST requests, you would include an 'Authorization' header with a Bearer Token. However, with VAPI, you can authenticate your OpenAI-compatible endpoint by using the

/credential

endpoint to POST your server’s API key and URL. Here's how you can do it: 1. Prepare your server's API key and the URL for the OpenAI-compatible endpoint. 2. Use the

/credential

endpoint to submit your API key and URL in the following format:

json
{
  "provider": "custom-llm",
  "apiKey": "YOUR_API_KEY_HERE"
}

Once you've done this, your server will be authenticated, and you can start making requests to your custom model through VAPI without needing to manually add the 'Authorization' header each time. If you need any further assistance or have additional questions, please let me know!

hey check the following: [1] https://docs.vapi.ai/customization/custom-llm/using-your-server [2] https://github.com/VapiAI/advanced-concepts-custom-llm

@User I successfully managed to integrate my custom model but streaming the response does not work fully. I do see in the logs those data chunks but VAPI doesn't generate speech content although it shows "Model sent end token" in the Call Log Details: """data: {"id":"chatcmpl-9lNDIiDF9tMPniynvEFZbkWz6bYaw","object":"chat.completion.chunk","created":1721077676,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_dd932ca5d1","choices":[{"index":0,"delta":{"content":"Hey"},"logprobs":null,"finish_reason":null}]} data: {"id":"chatcmpl-9lNDIiDF9tMPniynvEFZbkWz6bYaw","object":"chat.completion.chunk","created":1721077676,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_dd932ca5d1","choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null}]}

@OGOnkel please share the call id.

@User 4b5008b2-4dd9-4018-ab8b-4288c04e7f06

this is your system content, "48487478-44c0-4610-b331-2e98aa8c04d9"?

@User yes that's correct. In my custom model it's an identifier for custom data.

@OGOnkel can you please share your repo link/snippet.