https://vapi.ai/ logo
Join Discord
Powered by
Vapi Tool Voicemail Detection Not Working Properly
# support
t
Hello, as the title suggests, the Vapi Tool voicemail detection is not working properly. Call 1 ID: 650e7e43-ff63-490b-a18d-d574169915f3 (silence timed out) Call 2 ID: 6f11422a-ad0f-462e-ae5c-ace256fef044 (voicemail) Call 1 just isn't working at all... Call 2 is labeled as voicemail, it sends the message through, but isn't waiting until the voicemail is actually read on the phone's side.... The request/tools:
Copy code
'assistant': {
          "backgroundSound": noise,
          "firstMessage": info.info.agent.firstMessage,
          "firstMessageMode": speakFirst,
          "endCallPhrases": [info.info.agent.endphrase],
          "model": {
            "provider": "openai",
            "model": "gpt-4o-mini",
            "tools": [
              {
                "type": "voicemail",
                "async": true,
                "function": {
                  "name": "AnsweringMachineDetection",
                  "strict": false,
                  "description": "Detects if a machine answered the phone."
                },
                "messages": [
                  {
                    "type": "request-complete",
                    "content": info.info.agent.voicemail_message,
                    "endCallAfterSpokenEnabled": true
                  }
                ]
              },
            ],
            "messages": [
              {
                "role": "system",
                "content": prompt
              }
            ]
          },
          "voice": {
            "provider": "11labs",
            "voiceId": storage.voices[info.info.agent.voice]
          },
        },
        'phoneNumberId': numberToCallID,
        'customer': {
          "numberE164CheckEnabled": false,
          'number': '+1' + info.phone_number,
        },
        },
@User @Chowderr @KingBear
v
message has been deleted
@Trent™ I'm sorry to hear you're experiencing issues with the voicemail detection feature. Let's try to troubleshoot the problem together. For Call 1 with ID 650e7e43-ff63-490b-a18d-d574169915f3, where silence timed out, it's possible that the machineDetectionSilenceTimeout parameter might need adjustment. The current setting is 6000 milliseconds (6 seconds). If the silence at the end of the machine's greeting is longer than this, it could cause a timeout before the beep is detected. For Call 2 with ID 6f11422a-ad0f-462e-ae5c-ace256fef044, which is labeled as voicemail but isn't waiting until the voicemail is actually read, it seems like the issue might be with the timing of the message delivery. The 
endCallAfterSpokenEnabled
parameter is set to true, which means the call will end as soon as the voicemail message is spoken. If the voicemail system on the other end requires a beep before a message can be left, this could be causing the message to be cut off. To address these issues, you might consider the following steps: 1. Adjust the 
machineDetectionSilenceTimeout
to a higher value to accommodate longer silences. 2. Ensure that the 
machineDetectionSpeechThreshold
and 
machineDetectionSpeechEndThreshold
are set correctly to detect when the machine greeting has finished. 3. If the voicemail system requires a beep before the message, you may need to set 
endCallAfterSpokenEnabled
to false and use a different method to detect when the message has been successfully left. If you'd like, I can assist you in updating these parameters or exploring other troubleshooting steps. Please let me know how you'd like to proceed.
You can continue this conversation by mentioning me in the message.
t
To add onto this, Call 1 ID: 4709d8de-2881-4453-8598-6d0a7e294e9b Call 2 ID: fe12d5a3-aff0-4099-ba5b-a1661f4ac1d6 I added twilio's voicemail detection alongside Vapi's
Copy code
"voicemailDetection": {
             "provider": "twilio",
             "voicemailDetectionTypes": [
               "machine_end_beep",
               "machine_end_silence",
               "human",
               "fax",
               "unknown",
               "machine_end_other"
             ],
             "enabled": true,
             "machineDetectionTimeout": 45,
             "machineDetectionSpeechThreshold": 2400,
             "machineDetectionSpeechEndThreshold": 2400,
             "machineDetectionSilenceTimeout": 6000
           },
           "voicemailMessage": voicemail_message,
Now it seems to be actually matching the call end reason to voicemail better.... but doesn't actually leave a voicemail, the voicemail message is said at the end of the call correctly. No voicemail is left though.
s
- For the first call ID set the startSpeakingPlan to 0.8 value and then it leave the voicemail message and end the call. "endedReason": "voicemail", [currently] "message": "Hello, Alex. This is the voice mail message. Bye.", [expected] "message": "This is the voice mail message. Bye.", - For the second call ID it is same as first Call ID. > 🔵 18:36:18:632 [user LOG] Model called tool: AnsweringMachineDetection() The tool is used for detecting voicemail, and I see both calls have ended reason as voicemail, @Trent™ can you calrify what do you mean by "**No voicemail is left though." **
t
What I mean is that in the transcript the voicemail message is there.... But it's not actually being logged as the voicemail for the call itself.
@Shubham Bajaj Any thoughts?
Changing the speaking to 0.8 did not have any effect. new call ID: 21a01032-2ac3-48c5-932b-3b93a7c4a630
The transcript shows the bot is saying the actual voicemail message, but it's like it's not waiting long enough to actually get past the voicemail message from the phone's end, know what I mean?
I changed my tool up a bit, just exactly what Vapi's documentation is: "tools": [{ type: "voicemail" }], I also change the twilio detection to the documentation, same issue.
s
@Trent™ I just looked into your call ID and for sure the voicemail is being detected but as you mentioned the firs message is voicedout first and then the voicemail message. There is an solution to it increase the start speaking time using custom endpoting. Let me know if you want to give it a try.
t
Yes, how can we do this? @Shubham Bajaj
v
Hey, we will be having an Office Hour in 20 minutes. Can you hop in and get your query resolved during it?
t
My apologies as I am at work.
@Shubham Bajaj would you be able to give me some notes and or direction on this? So I can do this later?
s
@Trent™ What's happening right now: I can see that while your voicemail is being detected, the timing is a bit off - the system's jumping in too early with messages and not waiting for the voicemail greeting to finish. That's definitely frustrating! Here's how we can fix it: First, let's set up your voicemail detection properly. Here's what your configuration should look like:
Copy code
typescript
const assistant = {
  voicemailDetection: {
    provider: 'twilio',
    enabled: true,
    // Gives enough time for most voicemail greetings
    machineDetectionTimeout: 45,
    // Standard thresholds for speech detection
    machineDetectionSpeechThreshold: 2400,
    machineDetectionSpeechEndThreshold: 2400,
    // Wait time for silence detection
    machineDetectionSilenceTimeout: 6000,
    voicemailDetectionTypes: [
      'machine_end_beep',
      'machine_end_silence',
      ...existingVoicemailDetectionTypes
    ]
  },
  model: {
    tools: [
      {
        type: 'voicemail'
      }
    ]
  },
  // This is the key part that fixes the timing
  startSpeakingPlan: {
    customEndpointingRules: [
      {
        type: 'assistant',
        regex: '.*voicemail.*',  // Adjust this based on your voicemail message
        delayMs: 3000  // Adds a 3-second delay
      }
    ]
  }
}
Pro Tips: 1. Make sure to adjust the regex pattern to match your actual voicemail message content 2. The 3000ms delay (delayMs: 3000) is a good starting point, but you might need to adjust it based on your specific voicemail system 3. If you're still seeing timing issues, try increasing the machineDetectionTimeout or machineDetectionSilenceTimeout values This setup should make your system: - Wait for the complete voicemail greeting - Properly time the message delivery - Prevent any premature message playing Give this a try and let me know if you need any adjustments! We can fine-tune the timing parameters if needed.
t
Copy code
data = {
 "name": info.info.agent.id + "-" + random(10),
 'assistant': {
 "backgroundSound": noise,
 "firstMessage": info.info.agent.firstMessage,
 "firstMessageMode": speakFirst,
 "endCallPhrases": [info.info.agent.endphrase],

 "voicemailDetection": {
 "provider": "twilio",
 "voicemailDetectionTypes": [
 "machine_end_beep",
 "machine_end_silence",
 "human",
 "fax",
 "unknown",
 "machine_end_other"
 ],
 "enabled": info.info.agent.voicemail,
 "machineDetectionTimeout": 45,
 "machineDetectionSpeechThreshold": 2400,
 "machineDetectionSpeechEndThreshold": 2400,
 "machineDetectionSilenceTimeout": 6000
 },
 "voicemailMessage": info.info.agent.voicemail_message,

 "model": {
 "provider": "openai",
 "model": "gpt-4o-mini",
 // "tools": [{ type: "voicemail" }],

 "tools": [
 {
 "type": "voicemail",
 "async": true,
 "function": {
 "name": "AnsweringMachineDetection",
 "strict": false,
 "description": "Detects if a machine answered the phone."
 },
 "messages": [
 {
 "type": "request-complete",
 "content": info.info.agent.voicemail_message,
 "endCallAfterSpokenEnabled": true
 }
 ]
 },
 ],
 "messages": [
 {
 "role": "system",
 "content": prompt
 }
 ]
 },
 "voice": {
 "provider": "11labs",
 "voiceId": storage.voices[info.info.agent.voice]
 },

 },
 'phoneNumberId': numberToCallID,
 'customer': {
 "numberE164CheckEnabled": false,
 'number': '+1' + info.phone_number,
 },
 "assistantOverrides": {
 variableValues,
 "analysisPlan": {
 "structuredDataSchema": {
 "type": "object",
 properties
 ,
 "required": analysisRequired
 },
 "structuredDataPrompt": "Extract all of the information needed for the desired properties of the call."
 },
 "startSpeakingPlan": {
 "waitSeconds": 0.4,
 "smartEndpointingEnabled": false,
 customEndpointingRules: [
 {
 type: 'assistant',
 regex: '.*voicemail.*', // Adjust this based on your voicemail message
 delayMs: 3000 // Adds a 3-second delay
 }
 ]
 },
 },
 }
That is my complete JSON object I'm sending to VAPI for a call request, I am getting the error: "Bad Request", error 400 now. With the added startSpeakingPlan, to my assistantOverrides.
Copy code
"assistantOverrides": {
          variableValues,
          "analysisPlan": {
            "structuredDataSchema": {
              "type": "object",
              properties
              ,
              "required": analysisRequired
            },
            "structuredDataPrompt": "Extract all of the information needed for the desired properties of the call."
          },
          "startSpeakingPlan": {
            "waitSeconds": 0.4,
            "smartEndpointingEnabled": false,
            customEndpointingRules: [
              {
                type: 'assistant',
                regex: '.*voicemail.*',  // Adjust this based on your voicemail message
                delayMs: 3000  // Adds a 3-second delay
              }
            ]
          },




        },
I looked in the docs and "startSpeakingPlan" is suppose to go under "assistantOverrides", so that part is correct.
I wish I got an error message that told me what was wrong in the request. When I remove the startSpeakingPlan, my request works perfectly. Even when I keep:
Copy code
"startSpeakingPlan": {
            "waitSeconds": 0.4,
            "smartEndpointingEnabled": false,
            
          },
The request works.
@Shubham Bajaj
Also, is this an issue EVERYONE has? Why does it seem like it's just my problem?
Copy code
"startSpeakingPlan": {
            "waitSeconds": 0.4,
            "smartEndpointingEnabled": false,
            "customEndpointingRules": [
              {
                "type": "both",
                "assistantRegex": "customEndpointingRules",
                "customerRegex": "customEndpointingRules",
                "timeoutSeconds": 1.1
              }
            ],
          },
This works fine with my request.
I don't have a callID to give you, but my orgID is: 1bb31438-59e0-43ab-b674-fe04f4f83c36 My post requests are green 201 status, but under the "Miscellaneous" tab, the Error is: "{"message":"Bad Request Exception"}".
I am looking forward to a response, thanks for all of your help thus far.
v
Hey Trent did you added the voicemail tool as well?
t
Yes
s
@Trent™ Sorry for mixing up the properties.
t
Ok I got it working finally.
Another question before we close this ticket, I am having an issue where the voicemail detection thinks someone picked up, and is waiting for the user to respond, but it doesn't because it's in the voicemail tone..... and the call goes on for 34 seconds or so until it times out. Call ID: 07b2ba7c-53ca-427f-83a2-84b0cc4051ca Any thoughts on fixing this? End Reason: Silence Timed Out
v
Use gpt-4o and set the temperature to 0.3 and it will solve your issue.
t
Thanks
2 Views
PreviousNext
Powered by