Agent not performing function calls for 25% of cal...
# support
m
Hi, for quite a few calls (~25%) recently, the agent is saying it's checking availability for a specific practitioner, but it actually doesn't check. Instead, it leaves the user waiting indefinitely, then hangup on them I get that hallucinations are a part of LLM, but is there anything I can do to try and reduce their probability? It feels high for gpt4 8438d69f-2d81-4a0d-813a-8bcfbb41bd4d a83d344c-bc77-400b-87dd-5b7442d6d0a7 a5769f90-d2d0-413f-af3a-bf5eb87d6d1b
v
Hi @manu, I'm an AI assistant for Vapi. I'll analyze available sources and get back to you shortly.
----------------------------- **While I generate an answer, here are some sources that might be helpful**: * [Delete Assistant]() * [List Assistants]() * [Create Phone Call]() * [Create Assistant]() -----------------------------
### Answer Managing hallucinations or incorrect function triggers in LLM, such as GPT-4, can indeed be challenging. While the specific issue of function calls not being executed as intended isn't directly addressed in the provided documentation, there are general strategies to possibly mitigate such issues: 1. **Prompt Engineering**: Fine-tuning the prompts sent to the model can significantly impact its performance. Ensuring clarity and specificity in prompts can help guide the LLM towards the expected outcomes. This involves carefully crafting the instructions or questions asked to the LLM, which might reduce the likelihood of hallucination or incorrect actions. According to a discussion on Discord, rigorous prompt tweaking has shown success in improving LLM performance [(1)]() . 2. **Review and Adjust Parameters**: The assistant's behavior can be influenced by various parameters, such as
temperature
, which controls randomness, or adjusting the
maxTokens
for responses. Reviewing these settings and experimenting with different configurations might help in reducing unexpected responses. Although the exact parameters aren't specified in your case, general adjustments to the model's configuration might offer some improvement [(2)]() . 3. **Feedback Loop**: Implementing a feedback mechanism to analyze calls where the assistant didn't perform as expected and adjusting the system based on these insights can be crucial. While this is more of a long-term strategy, continuously refining the system based on real-world interactions can lead to significant improvements over time. 4. **Utilize Semantic Caching**: If applicable, leveraging options like semantic caching could help in managing recurrent queries more efficiently, possibly reduc
ing errors in function calling by relying on cached, known good results. Semantic caching is mentioned as a feature for assistants, indicating its potential utility in improving response accuracy and consistency . 5. **Exploration of Different Models**: If the issue persists and is significantly impacting the user experience, exploring other models or versions might be worth considering. Although you mentioned a concern with GPT-4, there may be configurations or alternative models that strike a better balance between response latency and accuracy. Lastly, given the complexities involved with LLM and AI-based systems, consulting with technical support or the community could provide additional insights specific to your implementation and use cases. Unfortunately, the provided documentation doesn't cover a direct solution to reducing hallucinations in function calls, but applying best practices in AI model interaction and management is advisable.
Got it. Tagging @User for additional help.
s
I hope your question got answered in the general section.
m
yes - thank you