Voice cut off due to async tool call response
# support
f
I have an async tool call navigateToUrl('url') which I use to redirect the user to a specific website when he's interested in a particular product. There he can find more information about the product or order it. The tool call gets triggered correctly but the voice get's cut off when the tool call returns a success response after which the sentence sort of restarts. Call Id: 604e7ddd-8cdc-4950-9879-47515319c7e8
v
It sounds like you're experiencing a frustrating issue with the voice getting cut off after the async tool call
navigateToUrl('url')
returns a success response. This is a common problem in voice AI applications, especially with async tool calls. Here are some steps you can take to troubleshoot and potentially resolve this issue: 1. **Review Recent Changes**: Check if there have been any recent changes to the configuration or logic that might affect how the assistant responds after the tool call. Ensure that the logic for providing feedback after tool calls is still intact. 2. **Inspect Network and Environment**: Make sure your network connection is stable and test the system in a different environment to rule out any device-specific issues. 3. **Utilize Logs**: Look at the logs for both VAPI and any other relevant services to see if there are error messages or warnings that could provide insight into the cut-off issue. 4. **Suppress Tool Call Announcements**: Consider adjusting backend settings to suppress the automatic success messages that might be interrupting the flow. This can help maintain a smoother conversation without unnecessary interruptions. 5. **Assume Success in Conversation Flow**: Design your conversation flow to assume success after the tool call, allowing the dialogue to continue without waiting for user confirmation. 6. **Check Voicemail Detection Settings**: If applicable, adjust parameters like
machineDetectionSilenceTimeout
to prevent premature cut-offs. If these steps don't resolve the issue, it may be worth reaching out for further support or checking for any known bugs in the VAPI service that could be affecting your tool calls. Let me know if you need more help with this! 😊 Sources: - [Troubleshooting Voice Cut-Off Issues in Async Tool Calls with VAPI](https://discord.com/channels/1211482211119796234/1211483291191083018/1334114042876264518)
- [Common Issues with Async Tool Calls in Voice AI Applications](https://discord.com/channels/1211482211119796234/1211483291191083018/1334309208035954728) - [Best Practices for Voice Interactions After Async Tool Calls in Vapi](https://discord.com/channels/1211482211119796234/1211483291191083018/1334309208035954728)
f
This doesn't solve the issue
@Shubham Bajaj Could you please look into this as the outcome of this will determine if we go with Vapi for our company or not
s
@Floris looking into this.
expected
The Aromatherapy Massage combines a relaxing massage with the therapeutic benefits of essential oils. It begins with a consultation to select oils that match your mood or health goals, such as lavender for relaxation or eucalyptus for respiratory support. ...more
actual
with the therapeutic benefits of essential oils. It begins with a consultation to select oils that match your mood or health goals,
logs
šŸ”µ 15:31:22:089 ElevenLabs (WebSocket #2) Received Non Audio Message. Message: { "audio": null, "isFinal": true, "normalizedAlignment": null, "alignment": null } šŸ”µ 15:31:17:550 ElevenLabs (Websocket #2) Pushing 101... "The Aromatherapy Massage combines a relaxing massage with the therapeutic benefits of essential oils." There was an cutt off from 11labs side for
The Aromatherapy Massage combines a relaxing massage
so I suggest @Floris Give me some time to check 11labs team if its on their side. Let me know your thoughts on this.
f
Thanks for looking into this @Shubham Bajaj I thought it was due to the async function call returning success and that's being fed into the llm or into 11labs which cuts off the sentence it's saying at that moment.
It's not a one off, I can reproduce it each and every time the async function call is executed and a result is returned while the voice is talking
I don't think the async function call result should stop the llm or voice from what it's doing
An option to ignore tool call results, or to not feed it back into the llm/voice would work for me as a fix I guess
Any news on this @Shubham Bajaj ?
s
Yeah set the tool to
async
.
@Floris Apology for the delay in response. You can try now on your end it will work.
Let me know how it goes.
f
I'll test it today
It's not fixed @Shubham Bajaj call id: a90e9a69-15b6-45e5-9700-ac1b5164a71c The voice got cut off again when the second tool call responded while the voice was talking
Could you look into this when you have a sec @Shubham Bajaj ?
Any update on this, @Shubham Bajaj ? ^
s
@Floris looking into it.
@FlorisI re-examined it and realized I misunderstood it last time. After reviewing the call logs again, I couldn't reach the conclusion. I'll find its RCA, and get back to you works?
f
Sure, I can also demo it real life or record a video if that helps?
k
Floris, would it be possible to provide one more latest call_id that I can take a look into??
f
Hi @Sahil, I got another one: 17574f7f-200a-4c3e-ad2b-ffc681dbecb0 The voice get's cut of when the async function call is triggered. It get's cut of at the "The sweet" part. I'm using the web integration to test and printed the transcripts: https://cdn.discordapp.com/attachments/1346505933697847307/1351463026838212639/image.png?ex=67da7786&is=67d92606&hm=1d2d834fa9ca545b29548814fa75d11a06d2f44bb1b1ee8abd83eda5f1dc54b8&
k
Hello Floris, I believe you're using vapi.send to trigger a function when the user makes a selection correct? If that's the case you'll need to add triggerResponseEnabled: false to the vapi.send message so that it doesn't interrupt the assistant. vapi.send({ type: 'add-message', message: { role: 'system', // or 'user' depending on context content: 'Important context the assistant should know about but not respond to immediately' }, triggerResponseEnabled: false // This is the key parameter }); Could you test this and let me know if it resolves the behavior?
f
Hi Mason, No, I let the assistant trigger an async tool function when a guest request information about a specific product. An url is passed as a parameter to that function which I use client side to navigate the guest to that url in an iframe. I don't have the ability to use that triggerResponsseEnabled boolean I think.
@Shubham Bajaj Any progress on this? Been going on quite some time now
k
@Floris The issue lies in how the tool call responses are processed and fed back into the conversation flow, affecting the TTS (text-to-speech) stream. The onToolCalls method is triggered when the LLM generates a tool call. For async tools, even though they're marked as async in the configuration (tool.async = true), the system is still capturing the response. This function feeds the tool call results back into the conversation buffer, which triggers a new LLM response. Let me check with team regarding this, and ThankYou for bringing this to attention.
f
No worries @Shubham Bajaj, any idea when this will be fixed?
k
Your issue has been acknowledged, and it will be fixed by the end of this week.
f
Nice, thank you @Shubham Bajaj please keep me posted
Hi @Shubham Bajaj, I'm still experiencing this issue. Can it been resolved already? It been a month and a half ago since we last discussed
k
Hey, Could you please schedule a brief call at your convenience so we can discuss this matter in detail? Kindly use the following link to book a suitable time: .
3 Views