SuccessEvaluation not working
# support
b
Hello, My name is Alessio and I’m using Make.com with a custom http request to trigger a VAPI call. For some reason the end of call report doesn’t include the SuccessEvaluation only the summary. I also tried it without adding the “successEvaluationPrompt” but it didn’t work either. Am I missing something here? I would appreciate any help! I’m using this request here: { "name": "Test Call", "assistantId": "{{2.assistantId}}", "assistantOverrides": { "model": { "messages": [ { "content": "{{6.message}}", "role": "assistant" } ], "provider": "openai", "model": "gpt-4o" }, "firstMessageMode": "assistant-waits-for-user", "analysisPlan": { "successEvaluationPrompt": "Du bist ein Experte im Analysieren von Anrufen und bekommst ein Transkript des Anrufs. Bitte entscheide anhand des Ziels des System Prompts, ob der Anruf erfolgreich oder nicht war." } }, "phoneNumberId": "{{2.phoneNumberId}}", "customer": { "number": "{{2.callNumber}}" } }
v
@balistooo Hello Alessio, It's great to hear that you're integrating VAPI with Make.com. If the end of call report is missing the SuccessEvaluation, it could be due to a few reasons. Let's check a couple of things: 1. Ensure that the
successEvaluationPrompt
is correctly set in your request body. It should be part of the
analysisPlan
object, as you have it in your JSON request. 2. Verify that the language of the
successEvaluationPrompt
matches the language of the call. If the call is in English, the prompt should also be in English for accurate evaluation. 3. Confirm that the call is reaching its natural conclusion and that the system is generating a transcript that can be evaluated. 4. Check if there are any errors or messages returned by the API that could indicate why the SuccessEvaluation is not included. If everything seems to be configured correctly and you're still not seeing the SuccessEvaluation in the report, it might be a good idea to reach out to the VAPI support team for further assistance. Since you've already tried without the
successEvaluationPrompt
and it didn't work, it seems like there might be an issue that needs to be looked into more deeply. If you need further assistance or if there's anything else I can help you with, please let me know!
You can continue this conversation by mentioning me in the message.
Got it. Tagging @User for additional help.
s
Try increasing successEvaluationRequestTimeoutSeconds to 30 seconds and this parameter i'ts partof anaysisPlan.
b
Thanks for your help! I will try your approach.
Hi Shubham, I've tried your recommendation but it didn't solve the problem. The analysis was still without the successEvaluation. I also tried it without any analysis and timeout configuration because I thought it would use the default from the assistant then but also this didn't work. This is the call id from one of the calls: 22083378-253f-4394-a71f-38709b3aec2a
I've tried adding all possible parameters for the successEvaluation and now the analysis is completely empty. { "name": "Test Call", "assistantId": "{{2.assistantId}}", "assistantOverrides": { "model": { "messages": [ { "content": "{{6.message}}", "role": "assistant" } ], "provider": "openai", "model": "gpt-4o" }, "firstMessageMode": "assistant-waits-for-user", "firstMessage": "{{2.firstMessage}}", "analysisPlan": { "successEvaluationPrompt": "Du bist ein Experte im Analysieren von Anrufen und bekommst ein Transkript des Anrufs. Bitte entscheide anhand des Ziels des System Prompts, ob der Anruf erfolgreich oder nicht war.", "summaryRequestTimeoutSeconds": 10.5, "successEvaluationRequestTimeoutSeconds": 10.5, "successEvaluationRubric": "PassFail" } }, "phoneNumberId": "{{2.phoneNumberId}}", "customer": { "number": "{{2.callNumber}}" } } https://cdn.discordapp.com/attachments/1260620572849737781/1260956571827961876/Screenshot_2024-07-11_at_15.51.05.png?ex=669134d2&is=668fe352&hm=538133169af674810a198b04c98df28f4405de13abf8caeea38c38bb0befd0ba&
s
@balistooo is this 22083378-253f-4394-a71f-38709b3aec2a recent call id?
b
Yes, one of them. I did multiple tests where it all didn't work.
This is another one: 8e32d1eb-6884-415f-9949-d5a69b0f602c
s
hey please make an api request to increase time to generate success evaluation metric.
Copy code
json
    "analysis_plan": {
      "summaryPrompt": "Du bist ein deutschsprachiger Experte im Notizen machen und sollst bitte den Anruf in 2-3 Sätzen auf Deutsch zusammenfassen.",
      "successEvaluationRubric": "PassFail"
    },
b
Hello 🙂 Great, now it works! I added the maximum of 20.
4 Views