https://vapi.ai/ logo
Join Discord
Powered by
Is there a way to use OpenAI's Assistants API as a...
# support
n
Right now it seems like the only option is to try and bridge the Assistant API response to the chat completion API via custom server. It would be great if there was a native way to use a custom OpenAI Assistant to power the "intelligence" part of Vapi!
v
Hi @Nick Fiacco, I'm an AI assistant for Vapi. I'll analyze available sources and get back to you shortly.
----------------------------- **While I generate an answer, here are some sources that might be helpful**: * [Custom LLM]() * [Connecting Your Custom LLM to Vapi: A Comprehensive Guide]() * [Provider Keys]() * [Create Assistant]() -----------------------------
### Answer Currently, Vapi supports the integration of any OpenAI-compatible endpoint as a custom LLM, which includes using services like OpenRouter, AnyScale, Together AI, or your own server, but it does not natively support the OpenAI Assistants API directly as a built-in option [(1)]() . To integrate OpenAI's Assistants API, you would indeed need to set up a custom server that can handle the API responses and format them to be compatible with Vapi's expected inputs. This server would act as a middleware, receiving the output from the OpenAI Assistants API, processing it as necessary, and then forwarding it to Vapi in the required format [(2)]() . For instructions on setting up your server to use a custom LLM, you can refer to the documentation on using your server with Vapi. This would involve creating an endpoint that is compatible with the OpenAI Client and possibly supports streaming completions for best results.
n
There also isn't any documentation on what Vapi expects in terms of a streaming response, though I assume it would also be the same as the chat completion API. An example of that would be super helpful!
s
You need to use the custom llm flow then you will be able to do it.
n
So basically just hack the assistants API to return data in the same format as the chat completion API?
Also how do I return a streaming response
s
n
ty!!!
d
hey @Nick Fiacco have you managed to do it?
n
I haven't yet, but will let you know if I end up implementing it!
o
Hey @Nick Fiacco Have you implemented this?
p
I am on the same mission. Almost getting there 😉
g
likewise 😉
ok it 's done. I am going to build a frontend for people that are not comfortable with coding so you can try it easily. Please send me a message if you want to beta test it.
o
You are building frontend for openai assistant api? are you with vapi team?
g
i am building a frontend so that you can put your assistant api id and it will be connected to vapi yes. All you will have to do is add a custom server url . I am not with vapi team.
o
If you are talking about vapi assistant connected with custom llm, I have already done that. I want to use "OpenAI assistant" as custom LLMs, which I made on openai playground.
g
yes, this is what i built. You can use the "OpenAI assistant" as custom LLMs, which you made on openai playground.
o
Can you please share the code
I am closing in on this but I am stuck here client.beta.threads.runs.stream( thread_id=thread.id, assistant_id = GPT_assistant_id, instructions=prompt, event_handler=EventHandler(), ) how do I format this output into vapi structure. please let me know, I am kinda stuck on this.
g
if event["event"] == "message": chunk = { "id": f"chatcmpl-{thread_id}", "object": "chat.completion.chunk", "created": int(time.time()), "model": "gpt-4o", # Adjust model as necessary "system_fingerprint": "fp_5c95a4634e", # Example fingerprint "choices": [ { "index": 0, "delta": {"content": event["data"]["content"]}, "logprobs": None, "finish_reason": None, } ], } yield f"data: {json.dumps(chunk)}\n\n" elif event["event"] == "done": logging.info("Streaming completed") yield "data: [DONE]\n\n" break
o
Thanks, Done that.
PreviousNext
Powered by