Caller-side audio hallucination
# support
r
Hey Vapi team, can you help us understand a possible false-positive transcript/barge-in issue? Call ID: 019e3c18-c915-7002-9725-04072519956c https://dashboard.vapi.ai/calls?callId=019e3c18-c915-7002-9725-04072519956c Time: May 18, 2026 around 13:18:43 ET In the Vapi logs we see a
Deepgram final transcript
event at ~4.3s with transcript: “Yeah.” That then appears in the
conversation-update
payload as a user message: role: user message: "Yeah." secondsFromStart: 4.338 duration: 320ms wordLevelConfidence: 0.85595703 From listening to the recording / looking at the user channel, we don’t hear anything that should have produced “Yeah.” There was a
user-interrupted
event shortly before it, while the assistant was speaking the recording disclosure, so our guess is this may be an ASR or barge-in false positive, possibly caused by echo/noise from the assistant audio. Questions: 1. Is this expected behavior for short false barge-ins with Deepgram/Nova-3? 2. Is there a way to inspect the raw caller-side audio segment Vapi sent to Deepgram around 4.3s? 3. Are there recommended settings to reduce these false positives, such as interruption thresholds, denoising, endpointing, or transcript confidence filtering? 4. Should we treat short one-word transcripts during assistant speech differently on our side, or is there a Vapi-side mitigation? Relevant config from the call: - transcriber: Deepgram nova-3, language en - backgroundSpeechDenoisingPlan.smartDenoisingPlan.enabled: true - startSpeakingPlan.smartEndpointingEnabled: true - smartEndpointing provider: livekit - stopSpeakingPlan.voiceSeconds: 0.5
m
This really looks like a false barge-in from echo/background audio while the assistant was speaking. We’ve seen short transcripts like “Yeah” happen occasionally with Nova-3, especially around interruptions or disclosure playback. I’d probably test slightly stricter interruption/endpointing settings and ignore very short transcripts during assistant speech unless confidence is consistently high. Curious if you’re seeing this more on speakerphone or certain devices/carriers? @rod007vrr
g
We are experiencing the same issue with unexpected call interruptions (VAD) while using Nova-3. I also have a question regarding the "Smart Endpointing" feature. I noticed it appears in both the "Advanced Start Speaking Plan" settings and the transcription configuration. Could someone clarify the functional difference between these two configurations and how each one impacts interruption handling or speech detection behavior?
s
Hi, Thanks for the detailed write-up and the call ID - we were able to trace the exact sequence in the logs.
What happened This is a VAD (Voice Activity Detection) false trigger, not a Deepgram transcription error. Here's the precise timeline: At 17:18:38 the assistant began speaking the recording disclosure. At 17:18:41 the VAD detected speech activity on the user channel and upgraded to "Utterance Detected," which fired the user-interrupted event and cleared the pipeline. At 17:18:43 Deepgram returned a final transcript of "Yeah." with 0.856 confidence.
Deepgram correctly transcribed whatever audio it received - the 0.856 confidence actually suggests it received clear audio that phonetically resembled "Yeah." rather than random noise, which is consistent with echo from the assistant's audio bleeding into the user channel.
To your specific questions This is not expected behavior for Nova-3 specifically - the issue is upstream in the VAD layer, not in Deepgram's transcription. Regarding raw audio inspection, your call has pcapEnabled: true with custom S3 storage. Check your S3 bucket for call ID 019e3c18 around 17:18:41–43 UTC and you should be able to hear exactly what was sent to Deepgram.
Recommended settings to reduce false positives Increase stopSpeakingPlan.voiceSeconds from 0.5 to 0.8–1.0, which requires longer sustained speech before an interruption is triggered. Also increase Deepgram's endpointing from 300ms to 400–500ms to give the system more patience before committing to a transcript.
Let us know if anything else comes up.