https://vapi.ai/ logo
Join Discord
Powered by
Bug: Assistant reacts to system messages
# support
s
Per the docs: https://docs.vapi.ai/assistants/background-messages, adding a message with the role 
system
"Designates the message origin as ‘system’, ensuring the addition is unobtrusive. Other possible values of role are ‘user’ | ‘assistant’ | ‘tool’ | ‘function’ However, the Assistant reacts to system messages just as it does to user messages. It treats it as a user interruption (I receive a "user interrupted" message) and responds. I've found no message content that works to compel it to not respond. To test: Send a system message with e.g.: 1. "Stop talking. Do not respond. It is the user's turn to speak." 2. "The user has moved to the library. Do not respond, just note this for the future." Expected: In case 1., assistant stops talking. In case 2., Assistant continues without acknowledging the message. Actual: Assistant always responds as though the user has just given this information and expects a response.
c
It is to be expected, even if you try to do it chatgpt it will generate some context around it.
s
Any suggestions for mitigating this?
So, yes: If you stop the LLM from generating its current response, and add a system message, and send just that to the LLM, for sure you expect it to generate some output even if you tell it not to. BUT if that's what VAPI is doing, it's not an optimal approach if you really do want the messages to be "unobtrusive" as in the documentation, rather than immediately interrupting the current output and responding to the system message content. Rather, system messages could be added to the conversation context, but not immediately sent to the LLM, instead waiting until there is a new turn in the conversation. Separately but related, it should be possible to send a message to VAPI to stop generating its current turn. Similar to the behavior of the "user interrupted" message, but sent from the client instead of from VAPI. Is there anything like that?
Here's another use case: I am doing local speaker identification so that multiple people can be in a conversation with the Assistant. I need to tell the Assistant who's speaking at any given time. I would presumably use a system message for this, however as it's currently implemented, this would result in an incorrect and incorrectly-timed response.
c
Hey SoulAuctioneer, The issue stems from how system messages are currently processed in the conversation buffer and immediately sent to the LLM stream. You can try using triggerResponseEnabled flag that controls whether the message should trigger an LLM response. interface AddMessageParams { message: { role: 'system', content: string }, triggerResponseEnabled: boolean } // Usage example: vapi.send({ type: 'add-message', message: { role: 'system', content: 'Current speaker: John' }, triggerResponseEnabled: false }) For stopping current LLM generation, you can use the existing control message type: vapi.send({ type: 'control', control: 'mute-assistant' })
3 Views
PreviousNext
Powered by