Hi. Another of what seems to be a bug. When I upload knowledge base files with LLAMA 3 or even Mistral via Groq the LLM will not execute the tools and instead follow documentation guidelines but not actually carry out the tool. Without the knowledge base the tools get executed. I tied it with GPT 4o and it worked. So I’m not sure if this is an open LLM limitation but it’s working for GPT only. Please inform me of what’s going on. Also, is there a way to use Gemini Flash or any other LLM outside of what’s available. The custom LLM option seems to be about fine-tuned OpenAI models only.

@Cyrus Hello! I'm sorry to hear you're experiencing issues with the execution of tools when using LLAMA 3 or Mistral via Groq in conjunction with a knowledge base. It sounds like there might be a specific issue with how these models are handling tool execution when a knowledge base is present. This could be due to a limitation or a bug in the system. Regarding your question about using other LLMs like Gemini Flash, VAPI's Custom LLM feature is designed to integrate with fine-tuned OpenAI models. If you're looking to use a different LLM that's not currently supported by VAPI, this would typically require additional development and integration work to ensure compatibility. For both of these concerns, I would recommend reaching out to our support team for more detailed assistance. They can provide insights into the limitations of the current LLMs and guide you on how to integrate other models if that's possible. Would you like me to assist you in contacting support for more information on these issues?

can you share call Ids

HI, Need some info on 'Custom LLM URL' , I am working on PHP, able to receive messages and my ai response according to do that, but vapi server not answering after first message, need to know what kind of parameters vapi server need in response back streaming , other than role & contents. also noticed user's content updated without any comma or separation including previous old user contents after user new questions/reply

Call ID: 66f66005-09d0-4478-8d0c-fb463b11f545 - the assistant repeats itself and asks me again for the info it needs. Without the knowledge base uploaded it works fine. This happens with LLAMA3

create an issue.

What do you mean?

You have to send response back in the openAI messages format. check structure/type

components > schemas > OpenAIMessage

from https://api.vapi.ai/api-json#