"This might take a second" message for slow tools calls with a custom LLM integration. My assistant is currently integrated with a custom LLM and tools are built within the custom LLM. I followed the instructions in https://docs.vapi.ai/customization/tool-calling-integration. It is all working well expect that I want to introduce a ""This might take a second" message that is periodically repeated ever x seconds until a slow tool call is completed. Something similar to what happens in VAPI when you use one of the existing LLMs. How do I achieve that?

Hi Lucio! 👋 I understand you want to implement a periodically repeating "This might take a second" message while waiting for slow tool calls to complete in your custom LLM integration. I can definitely help with that! The best way to implement this is by using VAPI's

messagePlan.idleMessages

feature. This is specifically designed to handle situations where you need to play messages during periods of inactivity - perfect for slow tool calls! Here's how to implement it: You'll need to update your assistant's

messagePlan

configuration using a PATCH request to the VAPI API. Here's an example:

curl -X PATCH https://api.vapi.ai/assistant/id \
     -H "Authorization: Bearer token" \
     -H "Content-Type: application/json" \
     -d '{
  "messagePlan": {
    "idleMessages": [
      "This might take a second...",
      "Still working on it..."
    ],
    "idleMessageMaxSpokenCount": 3,
    "idleTimeoutSeconds": 8
  }
}'

This configuration will: - Display one of your specified messages every 8 seconds during idle periods (like when waiting for a tool call) - Rotate through your messages (you can add as many variations as you want) - Stop after speaking a maximum of 3 messages (you can adjust this number) According to the information I found, this is actually how VAPI's existing LLMs implement the "This might take a second" message - they use this idle messages feature to ensure "long silences are never completely silent" when waiting for tool calls to complete. For your custom LLM integration, you might also want to consider: 1. Setting appropriate

timeoutSeconds

for your tool calls (e.g.,

"timeoutSeconds": 30

) to ensure they don't hang indefinitely 2. Deciding whether your tools should be synchronous or asynchronous, as this affects how timeouts are handled Is there anything specific about this implementation you'd like me to clarify further? Sources:

The message plan you are mentioning makes sense when the assistant is waiting for the user. My usecase is different. This is the assistant waiting for a tool to return.

checking if this is resolved/solved for you?

Nope. See my last message.

Do you have an example on how to do that? I can't stream two distinct sources at the same time (the pending LLMs calls the periodic updates).

checking if this is resolved/solved for you?

yeah it is! thanks