https://vapi.ai/ logo
Join Discord
Powered by
Finetune Model (e.g. llama3)
# support
u
Has anyone tried finetuning an open source model for conversation flow? To make the agent use industry-specific vocab. I'm looking for a finetuned model that uses a more casual tone when completing replies. https://cdn.discordapp.com/attachments/1243235331579449454/1243235331952607383/1_xqmeYGXHFi2EPCwxhiwFiA.jpg?ex=6650bca1&is=664f6b21&hm=4349c92fa06301340e5ce8e8320bc74ac1ead357d1c5b41b88cee6ffed2208c0&
v
Hi @.hardboiledfish, I'm an AI assistant for Vapi. I'll analyze available sources and get back to you shortly.
----------------------------- **While I generate an answer, here are some sources that might be helpful**: * [Connecting Your Custom LLM to Vapi: A Comprehensive Guide]() * [Custom LLM]() * [Orchestration Models]() * [Orchestration Models]() -----------------------------
### Answer Vapi supports using any OpenAI-compatible endpoint as the Large Language Model (LLM), including the possibility of using fine-tuned or custom models that are tailored to specific vocabularies or industries. This enables the integration of models like Llama3, provided they are compatible with OpenAI's API standards. You can set up these models by specifying your custom LLM provider and model details in the configuration of your assistants in Vapi [(1)]() .
s
You can use a finetune model using custom-llm you can refer to this code https://dump.sahilsuman.me/streaming-custom-llm-vapi.txt
m
Yeah we've done it a lot
I can give you a dataset if you want to fine tune on
it's jsonl though so fine tune 3.5
u
GPT 3.5 throws lots of hallucinations when finetuned though, is there any other way?
What data did you finetune on?
m
Higher quality data and iterate on it using what it hallucinates on
It was a custom dataset I built on conversational sciences
u
this code is just chat completion, no? not finetuning
s
Yup it is just a simple chat completion code. Basically, you need to write the complete logic yourself for the finetuning part.
And if you just need to use the openai fine-tuned model then it is pretty simple
u
is it possible to finetune GPT-4? I don't have that option in my openai platform @User
m
You can request it through the fine tune ui on OpenAI
u
But the GPT-4? Did you have to make a request to fine-tune GPT-4? I don't have that option
m
Yeah you gotta request it through the fine tune ui on openai
u
I didn't even have the option to request to fine-tune gpt4
This is what my openai finetuning UI look like
m
You may have to run more tuning jobs to get the option
4 Views
PreviousNext
Powered by