https://vapi.ai/ logo
Join Discord
Powered by
Assistant constantly repeats Questions
# support
f
Hi @Vapi, We have an Assistant that collects customer information including Full Name & Phone Number. This call went very bad and we urgently need to this resolved to ensure these calls do not occur again. In this call, the Assistant asks the customer to spell their last name which they did. She then repeats the spelling back and asks, "is this correct?". The customer says "yes" and she repeats this 7 times. We've noticed the in Transcript, the spelling is different each time. Assistant finally gets to the next stage and asks for Phone Number 2 x times. She then repeats phone number back for confirmation 3 x times. This is even after the customer provides number and confirms it is correct a number of times. CALL ID: 719b8d5d-38d7-4ce7-babc-19ce4b089bde
v
I'm sorry to hear about the experience you had with the Assistant repeating questions and confirmations multiple times during the call. I understand how frustrating and urgent this situation is for you. Based on the information available, I don't have specific details or documentation that explain why this repetitive behavior is occurring or how to resolve it. If you have any additional details about the Assistant's configuration or recent changes, sharing those might help in identifying potential causes in the future.
I'm sorry to hear about the experience you had with the Assistant repeating questions and confirmations multiple times during the call. I understand how frustrating and urgent this situation is for you. Based on the information available, I don't have specific details or documentation that explain why this repetitive behavior is occurring or how to resolve it. If you have any additional details about the Assistant's configuration or recent changes, sharing those might help in identifying potential causes in the future.
v
Hey! To help track down this issue, could you share: - When exactly this happened (the timestamp) - What response you expected to get - What response you actually got instead This would really help us figure out what went wrong!
f
Hi @Shubham Bajaj Issue starts at 00:50 Call ID: 719b8d5d-38d7-4ce7-babc-19ce4b089bde As mentioned above, the assistant is repeating itself when it shouldnt be. Thanks
v
Hey, Could you please schedule a brief call at your convenience so we can discuss this matter in detail? Kindly use the following link to book a suitable time: .
f
Hi @Kyle , the only spaces available are not until July. Are we able to have someone from support check on this?
v
Sorry about the inconvenience. Can you share the latest call ID so I can debug this issue and get back to you?
f
Hi @Sahil , we thought we had fixed this but it is happening again. Call ID: 8cbf4484-c885-4471-bc62-2d332a63f5e8 00:54 - Assistant collected customers last name and confirms spelling. Assistant keeps repeating another 3 times after customer has confirmed it is correct. Due to this, the customer has ended the call and our client has lost the lead.
We also have this in the prompting to fix: # CORRECT SPELLING HANDLING INSTRUCTIONS After confirming the user's last name or email by spelling: > Always store and use the confirmed spelling version. > Do not send the original transcription to the FunctionsEnquiry tool unless it matches the confirmed spelling. > Build the final LastName and Email tool fields using the confirmed spelling from the user, not the automatic transcription. > Once a user confirms any spelling once, DO NOT repeat or re-confirm the spelling unless the user explicitly provides a correction. DO NOT change the confirmed spelling unless asked. > For short names (e.g. 3–4 letters), confirm once by clearly repeating the letters with spacing. If the user confirms it's correct, store it and proceed. Do not reinterpret or alter the spelling unless the user corrects it. > Only proceed to call the FunctionsEnquiry tool after confirmed spelling has been confirmed with the user.
v
Hey back, this issue is happening because of your knowledge-based integration where the TRIEVE KB gets involved on each user response and eventually influence the response at the end, leading to the iteration cycle of repeatedly asking last name. I will suggest you to move from TRIEVE KB to query-tool based KB instead to handle this hallucination issue. Let me know how it goes for you.
f
@Shubham Bajaj the query-tool base KB is not a practical solution for us because of the limitations of Google as a provider with the gemini-1.5-flash model. We need Open ai with GPT 4o cluster for best performance and capabilities specficcally for this AI. Is there another way to resolve this issue?
@Vapi team, just following up on update for this as we've had another call come through with same issue: 13107f25-c711-417c-aaaa-43bd40cf7a33
v
Hey @Beck, the immediate solution to this is to create a custom knowledge base for handling it . Once it's done, you can return the response to us through the custom knowledge base. more info:
f
@Shubham Bajaj just to clarify, this method is different to the query-tool base KB and supports Open ai + GPT 4o cluster model?
v
The solution is to create a custom tool where you define a tool of type function which works similarly to the Query tool for the KB. Instead of us doing all of the KB processing and handling, you do it on your server, and finally return the response to us. This way you can use any model of your choice without it being dependent on us. This is the best solution to solve your issues. I know that you guys are dealing with this for a very long time, so that's why I rethought it again and suggested you this instead of custom KB.
f
Hi shubham, after reviewing this solution with our team, building a Custom KB is not something we can implement right away. Is it possible to have some sort of control over when the AI accesses Trieve KB?
v
Hey Beck, what do you mean by "control over when the AI accesses Trieve KB"?
f
Hi @Shubham Bajaj , as you mentioned this issue of the AI repeating itself is because it is accessing the KB. The instructions for the AI to fulfill this request is in its prompting and it does not need to access the KB to fulfill the task. So it looks like it is accessing the KB for no reason for this specific task. Is there a way to control when the AI accesses the Trieve KB? As this may help us stop the repeating. We do have the following in our prompting: > If unable to find information in your prompts and knowledge base, then “Just a sec while I take a look” and search the X website to find the information
v
Hey Beck, The Trieve KB approach differs from the Query Tool KB model in two key ways: 1. Invocation - *Query Tool KB*: You explicitly write an instruction to invoke the KB whenever you need it for the next response. - *Trieve KB*: Every incoming chunk is automatically checked against the KB, and relevant content is injected into the LLM’s context on the fly. 2. Implementation - If you’d like to continue with Trieve, you can build a custom tool that fetches the exact KB snippets you need and returns them for the LLM to consume in its next response. Let me know if you’d like me to draft the custom tool spec!
PreviousNext
Powered by