https://vapi.ai/ logo
Join Discord
Powered by
Knowledge Base retrieval isn't accurate
# support
s
49ce9ece-a24b-488a-92a5-537e76d3d49e 00:44 The assistant selects the incorrect assistant from the knowledge base. I set the top K value to 1 since all I'm using the KB for is to search for the agent associated with a council number (and it's always a unique council number). Let me know if this should be changed. https://cdn.discordapp.com/attachments/1250061279800852520/1250061279968759838/Screenshot_2024-06-11_at_12.59.47.png?ex=666991ca&is=6668404a&hm=7302b6b2e225db12bfe3969629b564788c16c174caacb97f2ea4e0175bf5a4ef&
v
message has been deleted
@Saien It seems like there might be an issue with how the assistant is retrieving information from the knowledge base. Setting the top K value to 1 is appropriate if you're looking for a unique council number, as it should return the most relevant result. However, if the assistant is selecting the incorrect assistant, it could be due to a few reasons such as an error in the knowledge base entry, an issue with the search query, or a misconfiguration in the assistant's settings. To resolve this, you may want to double-check the knowledge base entries to ensure that the council numbers are correctly associated with the right assistants. Also, review the search queries to confirm they are formatted correctly and are targeting the right fields in the knowledge base. If you've already done this and the issue persists, it might be helpful to provide more details or examples of the incorrect behavior so that we can assist you further. If you need more in-depth support, please ask support for more info on this.
You can continue this conversation by mentioning me in the message.
Got it. Tagging @User for additional help.
s
s
Can you send me your KB file?
s
I tested it out today and it hasn't failed once. So perhaps this issue is resolved. Does anything look wrong with my knowledge base though and do you think that it could be formatted better?
s
The general recommendation which we provide is to provide the data in bookish manner like header followed by content which provides the best output.
s
So in my case, how should I restructure it? I tried the format of Joseph Donlon's Council Numbers: 34567 58433 etc. But the AI was hardly ever accurate so I changed it to the Q and A style.
s
Q and A style will also work like you can even try to provide data into json format
s
9b807b30-2942-409a-a640-e9817697eec4 00:43 The AI chose the wrong agent from the KB but when I asked it to check again, it chose the correct agent. Is there anything I could do to ensure that the KB retrieval works 100% of the time? https://cdn.discordapp.com/attachments/1250061279800852520/1250764076305485916/Screenshot_2024-06-13_at_12.49.08.png?ex=666c2051&is=666aced1&hm=e211f4cd8709796b94a22cb59340708478e6c9561f52a9798a57e398c2ce4107&
s
No, it is not cause we do the similarity search and your KB format isn't correct
s
It isn't correct but you said this before?
s
Like, what you can do is increase the topK to 10 or something and add in the prompt to provide you the most relevant data.
The thing is you have same answer for multiple Q
this is causing the confusion part
s
If I increase the topK to 10, won't it return 9 other chunks that don't have the council number in it (since it's unique)? What do you mean by 'add in the prompt'? Like to the KB?
s
Yeah, it will return more results and they will be appended to your LLM. Now, you will need to modify the system prompt to provide an answer that is as close as possible.
Note: It will also lead to increase in your call cost.
s
But why would I want extra chunks being fed to the LLM that don't even contain the correct council number?
s
Hey, let's do one thing let's have a meet and we can discuss all of these things.
s
I'd appreciate it a lot. I'll dm you to discuss further
PreviousNext
Powered by