Thanks King. I have a custom LLM that includes this as a langchain node, but I found that using a custom LLM adds a good amount of latency (maybe thats just because im using a local server ATM). I'll try using a websocket/live call control https://docs.vapi.ai/calls/call-features to see if that works better. prompt engineering doesn't seem to be cutting it...

Using a custom local LLM can cause latency due to hardware limits, so try optimizing with quantization or faster frameworks, and consider switching to Vapi’s WebSocket live call control for lower-latency real-time interaction, prompt alone may not be enough.

ok yeah, I tried to insert a system message with the controlUrl but that was treated as an interruption by my assistant. Is there a way to avoid that, so that its just considered the following turn? Otherwise its looking more like I need to use a customLLM (toolcalls and webhooks are not applicable). When you say "Vapi’s WebSocket live call" is that how VAPI connects to the custom LLM? or does it also apply to non-custom llm assistants?

Use background messages instead of standard system messages, Vapi’s WebSocket live call control enables real-time audio for both custom and non-custom LLMs, making it useful beyond just custom integrations..

Thanks King! I realized that I has to set "triggerResponseEnabled": false (I think the default is true). Once I did that I was able to use a combination of background messages (specifically the conversation-update) with the controlURL in order to update my state. Im still inserting the message with role "system."

@Kings_big💫 We are looking to improve latency by using Vapi live call control. If we start to stream audio using Vapi live call control, does that mean Vapi is still proxying the audio? Would we still have the latency due to vapi being in the middle?