Our assistant was asked a question and it answered with information that is not in the Knowledge Base or on Website. How can I identify where exactly the assistant drew this information for their reply? Call ID: 2defde6c-6b7a-4f61-b62a-c1bc26beb723

Hey! To help track down this issue, could you share: - When exactly this happened (the timestamp) - What response you expected to get - What response you actually got instead This would really help us figure out what went wrong!

Call ID: 2defde6c-6b7a-4f61-b62a-c1bc26beb723 When exactly this happened (the timestamp): 00:15 What response you expected to get: We were happy with response in this scenario but want to better understand where information was pulled from specifically for the answer given to avoid giving mis-information to customers in future. What response you actually got instead: Assistant gave a straight "yes" to the customer's Question although this information is not provided in Knowledge Base or on Website. I have searched through Logs and I can see the information being retrieved from Trieve although there is no specific information provided for this subject.

"id": "2defde6c-6b7a-4f61-b62a-c1bc26beb723", "transcript": "AI: Thanks for calling Carina Leagues Club. This is Jasmine. How can I help?\\nUser: Hi. I just wanna know if I can wear thongs the club tonight.\\nAI: Thanks for calling Carina Leagues Club. This is Jasmine. How can I help? Oh, so sorry. My let me check that for you. Just a moment. Yes. You can wear thongs at the club. However, please ensure that your overall attire is neat and tidy. If there's anything else you need help with, feel free to ask.\\n",

@firefly_0206 as you can see in the messages-txt the response was pulled off from your KB.

Hi Shubham, can you please edit your response to remove company name for privacy. Thank you

Beck, Assistant has utilized LLM response generation capabilities and a knowledge base to generate its content. Subsequently, it developed its own generated response to address the behavior you are observing. You can set the model temperature to 0 to prevent it from being creative. Also, you can try using Langfuse to monitor your Assistant LLM responses and gain more insights.

Hi @Shubham Bajaj, thank you for this information. This clarifies where the Assistant has gotten this from!

Marking this ticket as Solved ✅

Hi @Shubham Bajaj , We recently had our Assistant share incorrect information to a customer which was our first intentions to avoid here. It will also seem impossible for us to monitor this kind of this on a large scale but sifting through every call to ensure it provides correct information. We want to keep our Assistant conversational and "less robotic" and from our understanding, dropping the Model Temperature to 0 will cause make it less conversational and sound more robotic. Is there a way to ensure Assistant only shares specific information strictly from KB and Website only but keep it conversational? CALL ID: eeb275bc-a84d-4110-9ceb-b46901a16720 What went wrong: 00:09 Assistant said there are complimentary wheelchairs available (when there is no information provided about this in KB or website) What we expect to happen: Assistant to never share incorrect information and to only share information that is provided on KB and website.

@firefly_0206 please create a new support ticket because the previous ticket has been resolved, and you new issue won't be covered under SLA and as ticket.