- I create a VAPI assistant with a detailed system prompt mentioning user's data can be found using a knowledge base tool. - Whenever a user starts a session, I upload their history as a JSON file and register it as a “QueryKB” tool.

// Spec of query tool being created
{
  "type": "query",
  "function": {
    "name": "kb_query_for_{{sessionId}}",
    "description": "use this tool whenever you need info from user's recent activity"
  },
  "knowledgeBases": [
    {
      "provider": "google",
      "model": "gemini-1.5-flash",
      "name": "activity_for_user_{{sessionId}}",
      "description": "Use this KB for questions about user's recent activity",
      "fileIds": [
        "{{fileId}}"
      ]
    }
  ]
}

- On assistant startup, I inject that tool’s ID into the session so that the current conversation has the user's data available as a tool and it can query any information user requests.

const token = //from backend API
const knowledgeBaseToolId = //from backend API

// Vapi init
const vapi = new Vapi(token);

// FOR CONNECTING WITH ASSISTANCE
const assistantOverrides: AssistantOverrides = {
    model: {
        provider: "openai",
        model: "gpt-4.1",
        temperature: 0.7,
        toolIds: [knowledgeBaseToolId!]
    },
    silenceTimeoutSeconds: 60
};

const resp = await vapi.start(assistantId!, assistantOverrides);

The calls are connecting properly but there's something amiss. The assistant should automatically invoke the QueryKB tool whenever the user asks about their personal data, and incorporate the returned facts into its response. Despite the tool being available, the assistant never seems to call it—or returns empty or irrelevant answers. What configuration or prompt‐engineering steps might be missing or misordered? Would be great if someone can help.

Could you share the specific wording in your system prompt that instructs the assistant to use the KB Query tool? The way the tool usage is described in the system prompt is often critical for ensuring the model knows when and how to invoke the tool.

@Vapi Support Bot Tried some variants of the text like below: - The details of their recent activity can be fetched from kb_query tool. - Equipped with the knowledge base containing the user's structured recent activities, you provide real-time, personalized advice. Can you share a few examples with prompts properly mentioning and referring the knowledge-base tools? Also, another query I had- if there are multiple users doing a simultaneous call with same assistant, can each of the conversation have their own knowledge base tool? Or all of them would share the same knowledge base always?

To better invoke your KB Query tool, modify your system prompt to be more explicit and directive:

You are an assistant with access to the user's activity data. When asked about the user's recent activities, ALWAYS use the kb_query_for_{{sessionId}} tool to retrieve this information. If the answer isn't in the knowledge base, say "I don't have that information." Always cite the knowledge base when providing information from it.

This explicit instruction with the exact tool name and clear usage conditions will help ensure the tool gets invoked appropriately. Sources: - [Discord conversation about explicit KB instructions](https://discord.com/channels/1211482211119796234/1335953119275782144) - [Discord conversation about system prompt examples](https://discord.com/channels/1211482211119796234/1334889552002809928)

Message marked as helpful by @shivam! 🎉

checking if this is resolved/solved for you?

@shubham.bajaj yes, this is solved

Marking this ticket as Solved ✅

@shivam This is a great idea! I was curious, does it work smoothly in the conversation, and is there much of a delay when using it? Also, have you tried using Gemini 2.5 flash instead?