Custom LLM Multiple Utterances
# support
j
Does the custom LLM integration not support multiple utterances? For example: "One second while I verify your identity." "Great, I've found you in our system. Let me find some open slots for you." "Ok, I've found availability next Thursday". ^ This is all one turn of the agent and each utterance should be said as soon as it is complete. Is there a way for your custom LLM setup to support this (seems like an essential feature to make use of the custom LLM)?
c
Yes, Vapi supports multiple utterances in one turn via SSE streaming, your custom LLM must return each utterance chunk in real time using
data:
events, ending with
[DONE]
, to allow Vapi to speak each part as it's ready..
j
After the first [DONE] it doesn't read any of the other utterances
And if I only do 1 [DONE] at the end, it waits until all tool calls and utterances are complete before reading any of them
c
Hey jacronk, I sincerely apologize for the delay in our responses this week. We experimented with some process adjustments that didn’t work as intended, and unfortunately, this caused some disruptions. That’s entirely my responsibility. Just a quick note—our team is unavailable on weekends. However, if something urgent or critical comes up, feel free to reach out to me directly, and I’ll assist you. The support issues will be resolved in the coming weeks, and starting Monday, you can expect more consistent and improved support. Thank you so much for your patience and for continuing to stick with us!
Vapi speaks only after
[DONE]
, so to stream multiple utterances, send each chunk sequentially and delay
[DONE]
until all messages are sent.. [https://docs.vapi.ai/customization/tool-calling-integration](https://docs.vapi.ai/customization/tool-calling-integration?utm_source=chatgpt.com)
3 Views