Assistants consistently repeat messages
# support
c
Hello, I'm noticing that my assistants are repeating their second messages (first response to the user calling) on nearly every call my platform's users make. Call ids where this occurs: - 019a6e90-77df-777a-b835-28ecc6d82bf6 - 019a6ea7-959f-777d-a1b9-ea7776ad2f09 - 019a6e8b-f5fa-777e-a6a0-9a4724445177 - 019a5532-0447-7551-8d1f-e0330efcd3b1 - 019a4fe6-e3eb-755e-bc15-72897088ae99 - 019a5447-3f31-7442-983b-48f9e9067fba - 019a55b5-dad6-7883-9d77-db38462d19a7 - 019a5a27-feec-700b-96bb-d79ed6c53157 - 019a5a10-8ac1-7229-b484-c2f2557aae22 - 019a55e6-e415-7110-b964-a554d082dc29 - 019a553c-0cbc-711a-a772-5be6ad52a838 - 019a5532-0447-7551-8d1f-e0330efcd3b1 - 019a5447-3f31-7442-983b-48f9e9067fba - and many more specs: - I am using gpt4o as my model, 11labs as my voice provider, and deepgram as my transcriber - transient assistanst created via api How to reproduce (one of many ways): - Make a call with my assistant who has a system prompt with turn based mechanics (much like workflows) - Call starts with the assistant saying "Hello?" - user responds with "Hi, is this Adam?" - the assistant will say something like "Hi. This is Adam Piasecki. Who's calling? Yes. This is Adam Piasecki. Who's calling?" (this is just one example of a repeated phrase I see) My investigation: When looking at the vapi call logs, I can see that two api calls are made to gpt4o for the first message for each failing call. The first api call is made with a partial user message role message which the assistant responds to
Copy code
// request (truncated)
  {
    "role": "user",
    "content": "Hey. Is this"
  }
  // response (truncated)
  "response": {
    "content": "Hi, this is Adam Piasecki. Who's calling?"
  },
Then the second api call is made with the full user message to which the assistant also responds
Copy code
// request (truncated)
  {
    "role": "user",
    "content": "Hey. Is this Adam?"
  }
  // response (truncated)
  "response": {
    "content": "Yes, this is Adam. Who's calling?"
  },
These api calls are made within milliseconds of each other which then gets passed to the voice provider to say both messages back to back. I've tried updating my system prompts to add logic around conversation paths and preventing repeating, changing the voice id, changing the voice provider. None of these have helped resolve the issue. why is the behavior above happening, and is there a work-around? I know others are experiencing assistants repeating messages. Not sure if they have the same root cause: - https://discord.com/channels/1211482211119796234/1436189264676126771 Thanks
v
The repeated messages are caused by your assistant responding to both partial and final transcripts sent by Vapi. To prevent this, filter out partial transcripts in your webhook handler and only process those where
transcriptType
is `"final"`:
Copy code
js
if (message.type === "transcript" && message.transcriptType !== "final") {
  // Ignore partial transcripts
  return res.status(200).json({ received: true });
}
// Handle only final transcripts here
This should stop your assistant from responding twice to the same user input and eliminate repeated messages. For more details, see the [Vapi server events documentation](https://docs.vapi.ai/server-url/events). Source: - [Server events | Vapi Documentation](https://docs.vapi.ai/server-url/events)
c
To add to the above: I tried using assembly-ai as my transcriber with disablePartialTranscripts set to true (as described in this support case: https://discord.com/channels/1211482211119796234/1435895705528172624) but a similar repeating behavior occurs.
Update: Changing to Gladia as the transcript provider with the following config seems to reduce the occurance of this issue:
Copy code
transcriber: {
    provider: "gladia",
    model: "fast",
    language: "en",
    receivePartialTranscripts: false      
},
Update: Now I'm noticing that the assistant's second response is delayed by ~8 seconds on each call. I'm assuming disabling partial transcripts is the root cause. Is there a way to not have the repetition issue while also having low latency for the assistant?
c
Hi certified dirtiboi, If you're experiencing issues with assistants repeating messages or other procedural sequence problems, you might want to check the
hooks
and
messages
configurations for your assistant. 1. **Check Assistant Hooks**: Ensure the hooks like
CallHookAssistantSpeechInterrupted
or
CallHookCustomerSpeechInterrupted
are set up correctly. These hooks manage interruptions and could inadvertently cause repetition if not configured properly. Refer to the [hooks documentation](https://docs.vapi.ai/api-reference/assistants/update). 2. **Review Assistant Messages**: Review and adjust the assistant's
clientMessages
and
serverMessages
settings. Consistency in message handling is essential to avoid repetition. You can find more details in the [messages section](https://docs.vapi.ai/api-reference/assistants/list). 3. **Compliance and Recording Configurations**: Ensure that your
compliancePlan
and
artifactPlan
are configured correctly to prevent incomplete recordings, which might be mistaken for message repetition. This involves checking if
recordingEnabled
and
loggingEnabled
are set to
true
. These steps should help you address the issue of message repetitions effectively. Let me know if you need further assistance.
c
I'm not passing any of your suggested options in my transient assistant configs because I don't need them. The repitition is caused by partial transcripts being passed from the transcriber to the model during the call. I can prevent the repition by switching to a transcriber provider that allows forcing final transcripts only (e.g. Gladia). BUT, now I see long delays between messages since we no longer are accepting partial transcripts. So, my question is if there's a way to not have the repetition issue while also having low latency for the assistant? Are there other assistant config options that can help here? Here's an example of a 10 second latency between the user's first message and the assistant's response (no repeating occurs): 019a7829-256a-7cce-b87f-67a7f33d6965
After more investigation, the following seems to fix both issues:
Copy code
transcriber: {
  provider: "gladia", //changed from Deepgram
  model: "fast", //changed from Deepgram
  language: "en",
  receivePartialTranscripts: false // added with new provider
},
startSpeakingPlan: {
      waitSeconds: 2, // increased from 0.4 to 2
      smartEndpointingPlan: {
        "provider": "vapi" //changed from livekit (waitFunction: "400 + 400 * x")
      },
},
t
Thank you @certified dirtiboi for sharing that info.
For portuguese language doesnt work smoothly though.
can anyone give me a hint?
Thank you, Kyle.
I solved this issue. Thanks once more.
4 Views