Knowledge base hallucinations ?
# support
z
Hi all, I'm testing deeply the knowledge base functionnality (not the file) . This is my Setup : The file I use with my knowledge base is really simple so I can test # Prestations - Plomberie - Chauffage ~ # Zone d'intervention - Bordeaux - Pessac - Mérignac ~ Here my payload to create the KB { "provider": "trieve", "vectorStoreSearchPlan": { "searchType": "hybrid", "removeStopWords": true }, "vectorStoreCreatePlan": { "fileIds": [ "a6a2b253-e999-43fb-917e-4fe4d8de8180" ], "targetSplitsPerChunk": 1, "splitDelimiters": [ "~" ] }, "name": "a6a2b253-e999-43fb-917e-4fe4d8de8180" } 🛑 If I ask first if locksmith is the service list, the assistant say no, if I ask secondly if they work in Paris, the assistant says yes and start to hallucinate 🛑 If I ask first if they work in Paris, the assistant says no, if locksmith is the service list, the assistant says yes and start to hallucinate It's like the assistant was reading the KB only for the first question 🤔 ✅ If I try with the file attached to my assistant directly without the knowledge base, the assistant can answer correctly to my 2 questions. Am I doing something wrong ?
c
Where is the score threshold?
z
Is it required ? What would you advise @Chowderr ?
c
I mean I would test it and see. I have not seen any hallucinations on either side o f creating a knowledgebase.
z
Maybe its' not hallucination, but it's like the knowledge base is access only for my first question. No problem with the file attached to my agent
c
Yea that does not sound right at all.
z
@Vapi
v
@zxdream It seems like you're encountering an inconsistency with the knowledge base responses. The score threshold is a parameter that determines the minimum confidence score for an answer to be considered valid. If it's not set, the system might use a default value, which could be too low or too high, leading to inaccurate responses. I would advise setting a score threshold that balances between precision and recall. A higher threshold will give you more precise but potentially fewer answers, while a lower threshold will give you more answers but with less precision. You can start with a moderate value and adjust based on the results you observe. If you continue to experience issues where the knowledge base only seems to be accessed for the first question, it might be worth reviewing the implementation details or reaching out to the support team for further assistance. Please ask support for more info on this if the problem persists.
s
@zxdream Hey! To help track down this issue, could you share: - The call ID - What response you expected to get - What response you actually got instead (although already shared) This would really help me figure out what went wrong!
z
b9a74cd0-b318-4b56-8abe-e294b7f3708a
s
@zxdream We have implemented the new fixes and changes (development), and it should be operational at this time. Can youtry using it now?
2 Views