Hey everyone,
I am building an AI agent specifically designed to navigate IVR phone trees. I am running into a critical failure case with IVRs that loop continuously without pausing (specifically the Costco customer support line).
The Issue: The Costco IVR reads a very long list of hours and then loops the menu options repeatedly without any silence gap between the end of the menu and the start of the loop.
Because the IVR never stops speaking for more than a fraction of a second, the agent's VAD (Voice Activity Detection) or Endpointing logic never triggers the "User Finished Speaking" state. Consequently, the LLM is never called to process the transcript, and the agent sits there recording the loop indefinitely until the call fails or times out.
What I have tried: I have updated the System Prompt with strict instructions to "Interrupt immediately upon detecting a repeated phrase" and "Do not wait for silence," but this hasn't worked. I suspect this is because the System Prompt is only evaluated after the turn is considered complete (which never happens due to the continuous audio).
My Questions:
- Is there a configuration setting to make the agent process the transcript stream in real-time and trigger a tool while the other party is still speaking?
- Can I adjust the VAD silence threshold to be more sensitive (e.g., trigger on < 500ms gaps)?
- Is there a way to force an interruption or "barge-in" programmatically if the user/IVR has been speaking for X seconds without a break?
Example Transcript (Looping): "...To reach the pharmacy press 4. To reach the administrative staff press 1. For warehouse hours... [Loop continues instantly]"
Any guidance on how to handle continuous audio streams without silence gaps would be appreciated.
Thanks in advance!