Hi there, found a few messages touching this topic but haven't really found a definitive solution. I'd like my assistant to be able to invoke client side "tools", in order to update the UI based on the conversation. For example

showImage(3)

This is purely a UI requirement, so no need for a webhook here. I was planning to use the on("message") event handler, and handle tool-calls messages in there. I've set up a tool, and added an instruction "Invoke `showImage`" in my prompt. So far this is ...not working, 🙂 And I'm wondering what the issue is: could it be because I haven't configured a webhook in my tool setup that my tool is never invoked? More generally, is this the right approach?

For client-side tool calls in Vapi, you're on the right track - no webhook is required. Use the

on("message")

event handler to listen for tool calls:

javascript
vapi.on("message", (message) => {
  if (message.type === "tool-calls") {
    // Handle the tool call
    if (message.toolCalls[0].name === "showImage") {
      const imageId = message.toolCalls[0].parameters.id;
      // Update UI to show image
    }
  }
});

Make sure your tool is configured without a serverUrl to ensure it's processed client-side. Sources: - [Discord message about client-side tool calls](https://discord.com/channels/1211482211119796234/1342336047589359618) - [Discord message about on("message") event handler](https://discord.com/channels/1211482211119796234/1260258706667343892)

Define the tool name in assistant.clientMessages, guide the assistant in the prompt to send that message, and handle it in your client using vapi.on("message")—no webhook needed.

Finally sorted it out. The gotcha here was to set the tool as async, so the call is directly sent to client.

Nice

@Kings_big💫 is there any best practices document somehwere with samples or recommendations on how to build voice agents with tools? My agent works most of the time, but I'm struggling to make it 100% reliable. Tool calls are sometimes not made, and I sometimes have the impression that the tool call interfere with the discussion, is that possible? ie the assistant interrupts itself to make the tool call...

Ensure clear prompts, implement backoff and retry strategies, configure default tools correctly, analyze call logs, and leverage community insights for best practices.

@Kings_big💫 thx ! One last question on this from me : it seems the tool call is interfering with the conversation. Ie the assistant is interrupted by the call. Typically it would start saying something, then trigger the call and interrupts itself. This is pretty annoying from a UX POV. We've tried a few things in the prompt, but so far no luck... Any idea on what we could do?

Instruct the assistant in the prompt to complete its message before triggering tools, enable silent tool calls, adjust interruption settings, and test with different models like GPT-4o for smoother behavior.

How do you "enable silent tool calls" ?

Configure the destination assistant with an empty firstMessage, set firstMessageMode to assistant-speaks-first-with-model-generated-message, and ensure the transfer is made with no greetings or small talk.