I have a custom llm with a tool that takes 5 seconds to return. I am facing an issue where if a i send another request in the time that function takes to complete I lose the output of that function. For example: Call 1 to Custom LLM -> Invokes agent/completion api -> invokes tool - waiting for response Call 2 Customer LLM (1 second after call 1) -> Invokes agent/completion api -> Returns a message saying that the tool is in progress The problem is the agent does not say the response that the first call returns after the tool is completed. I have a suspicion that VAPI processes only the latest request. Is that correct? If so how can I solve this? @Shubham Bajaj

ye if you say anything while agent "thinks"(tool run), then it will just be interrupted with a new response

any way around this?

@slickVik basically you interrupted the assistant and started another turn in the conversation because of this it will lost context to previous turn or conversation.

@Shubham Bajaj so if I had a tool that took a few seconds to run and the user interrupts the agent while the tool is running, is there any way to send back a filler message until the tool finishes?

What do you mean by filler messages here?

Something to let the know the tool is running @Shubham Bajaj

You can use a request-response-delayed message that triggers while waiting for a tool call result.

But will that also start another completion? How would I do that if I am using a custom llm that has tools (not defined in the tools array of the vapi custom model)?

@slickVik Oh, I forgot you run the tools on your end. You might try sending the chunk with some content before the function is called with finish reason, or right before the function execution try yielding some response or streaming.

Yea that’s what I do currently, but there is not guarantee that the user will stop talking. This came up recently as the user said “thank you” in between the tool call and the function got called again. We just have to hope they don’t talk while the tool is running?

@slickVik I apologize for the delay in our conversation yesterday. I was occupied with other tickets and this ticket was forgotten. Since you've already solved this, which is great, could you please share the solution you adopted? That way, I can help others if they encounter similar issues, assuming it works well for you?