https://vapi.ai/ logo
Join Discord
Powered by
Analysis Structured Data Not Working During Real C...
# support
m
Analysis Structured Data Not Working. When we do a test phone call using the call button it works perfectly. When we call a real number it fails every time. Here is a video: https://www.loom.com/share/53b8aedfb7aa4a8ca04d6c12bad6a5ee?sid=66577037-6f25-489f-bf39-b78d49bf69a6
v
@MadHatMedia I'm sorry to hear that you're experiencing issues with the Analysis Structured Data feature. It seems like there might be a discrepancy between the test environment and the live environment. To assist you better, could you please provide me with the following information: 1. The structured data schema you are using. 2. Any error messages or logs that you receive when the feature fails. 3. The version of VAPI you are currently using. Once I have this information, I'll be able to help you troubleshoot the issue more effectively. If you're not sure where to find this information, please let me know, and I can guide you through the steps.
You can continue this conversation by mentioning me in the message.
m
Please watch the video, it explains the issue better. We are not getting a "failed" message. however anytime the call is to a real live number, the Analysis is always empty. @User
s
update your assistant summary, succcess evaluation and structured data generation timeout to 20 seconds and it will work https://docs.vapi.ai/api-reference/assistants/update-assistant
m
I dont udnerstand'
ok i think we did the patch correclty but it still did not work
Hello, we tried adding the PATCH. but that did not help. Here is the most recent call ID - 42e22ef3-2beb-4b54-b389-a637f1a2af8a The most recent has the PATCH Applied. here is a video to show what we did and how we added the patch:https://www.loom.com/share/5ae08addd24443bda098d802546760ec?sid=e1e79252-5de0-4ab8-97ac-b8dfdc5fde24 All all other live calls did not do the analysis correctly..Call ID:f1caf648-0506-4a8a-bc7c-f17b84019f96 a01e8480-aa68-48ff-b690-382ac4d805b3 f0d34da7-9dcf-4746-bb53-2b1309ec129e e984f199-2426-4de4-944d-f907dee83bcb
s
update assistant analysis required field mentioned. 
your_current_assistant_analysis
"analysis": { "summary": "Danny from the verification team of merchant support called to confirm details about Icy Freeze's merchant account. He verified the company name, Robert's role, the credit card processor (Optical), and the machine used (RTX 300). Danny informed that they would follow up in 3-5 business days to check for any additional credits owed.", "successEvaluation": "true" },
add the missing parameters into analysis property and it will work.
m
Why is this marked as Resloved. Its still not working. here is our code: { "analysisPlan": { "summaryPrompt": "You are an expert transcript evaluator. You will be given a Transcript of a call and you must determine if the call was a success. The most important details are the answers from the two questions the AI participant asks: ---QUESTIONS--- ~“And who do you process your credit card machine with?” and ~“What is the machine that you guys are using?” ---END QUESTIONS--- It is vital that you identify the answers to these two questions. if there is no answer then the call was a failure.", "summaryRequestTimeoutSeconds": 20, "structuredDataRequestTimeoutSeconds": 20, "successEvaluationPrompt": "It is vital that you identify the answers to these two questions. if either question is not answered then the call was a failure. ---QUESTIONS--- ~“And who do you process your credit card machine with?” and ~“What is the machine that you guys are using?” ---END QUESTIONS---", "successEvaluationRubric": "PassFail", "successEvaluationRequestTimeoutSeconds": 20, "structuredDataPrompt": "You are looking for the USER answers to the \"CCMP\" and the \"MN\" in the transcript. Find the user answers for the following questions: ---QUESTIONS--- ~“And who do you process your credit card machine with?” and ~“What is the machine that you guys are using?” ---END QUESTIONS--- ---STRUCTURE-- 1. The answer to: And who do you process your credit card machine with? is the \"CCMP\". 2. The answer What is the machine that you guys are using? is the \"MN\". ---END STRUCTURE-- --GOAL DATA-- - Credit Card Machine Processing (CCMP) - Machine Name (MN)", "structuredDataSchema": { "type": "object", "properties": { "CCMP": { "type": "string", "description": "The company or service used for processing credit card machines." }, "MN": { "type": "string", "description": "The name of the machine that is used for credit card processing." } }, "required": ["CCMP", "MN"] } } }
s
can you share the call id?
m
any call in our system
check any of them
we did 1000 calls and none of them worked
s
can you share the call id please, it's required to help you.
m
ok here is one
42cfa46e-620a-4d71-ab64-44a237857372
this one is better: fa084510-913d-4553-b7d0-0fc442da45fb
@Shubham Bajaj what is the best time to connect with you on here. we already lost a week and no progress.
j
Is that really your structured data prompt? Because there are issues with unescaped quotation marks surrounding CCMP and MN in it. Yours: 
"structuredDataPrompt": "You are looking for the USER answers to the "CCMP" and the "MN" in the transcript. Find the user answers for the following questions:  ---QUESTIONS--- ~“And who do you process your credit card machine with?” and ~“What is the machine that you guys are using?” ---END QUESTIONS---  ---STRUCTURE-- 1. The answer to: And who do you process your credit card machine with? is the "CCMP".  2. The answer What is the machine that you guys are using? is the "MN".  ---END STRUCTURE--  --GOAL DATA-- - Credit Card Machine Processing (CCMP) - Machine Name (MN)"
Corrected: 
"structuredDataPrompt": "You are looking for the USER answers to the \"CCMP\" and the \"MN\" in the transcript. Find the user answers for the following questions:  ---QUESTIONS--- ~“And who do you process your credit card machine with?” and ~“What is the machine that you guys are using?” ---END QUESTIONS---  ---STRUCTURE-- 1. The answer to: And who do you process your credit card machine with? is the \"CCMP\".  2. The answer What is the machine that you guys are using? is the \"MN\".  ---END STRUCTURE--  --GOAL DATA-- - Credit Card Machine Processing (CCMP) - Machine Name (MN)"
m
what do you mean
j
I mean the JSON you pasted is malformed. Where did it come from?
m
Copy code
{
  "analysisPlan": {
    "summaryPrompt": "You are an expert transcript evaluator. You will be given a Transcript of a call and you must determine if the call was a success. The most important details are the answers from the two questions the AI participant asks:  ---QUESTIONS--- ~“And who do you process your credit card machine with?” and ~“What is the machine that you guys are using?” ---END QUESTIONS---  It is vital that you identify the answers to these two questions. if there is no answer then the call was a failure.",
    "summaryRequestTimeoutSeconds": 20,
    "structuredDataRequestTimeoutSeconds": 20,
    "successEvaluationPrompt": "It is vital that you identify the answers to these two questions. if either question is not answered then the call was a failure.  ---QUESTIONS--- ~“And who do you process your credit card machine with?” and ~“What is the machine that you guys are using?” ---END QUESTIONS---",
    "successEvaluationRubric": "PassFail",
    "successEvaluationRequestTimeoutSeconds": 20,
    "structuredDataPrompt": "You are looking for the USER answers to the \"CCMP\" and the \"MN\" in the transcript. Find the user answers for the following questions:  ---QUESTIONS--- ~“And who do you process your credit card machine with?” and ~“What is the machine that you guys are using?” ---END QUESTIONS---  ---STRUCTURE-- 1. The answer to: And who do you process your credit card machine with? is the \"CCMP\".  2. The answer What is the machine that you guys are using? is the \"MN\".  ---END STRUCTURE--  --GOAL DATA-- - Credit Card Machine Processing (CCMP) - Machine Name (MN)",
    "structuredDataSchema": {
      "type": "object",
      "properties": {
        "CCMP": {
          "type": "string",
          "description": "The company or service used for processing credit card machines."
        },
        "MN": {
          "type": "string",
          "description": "The name of the machine that is used for credit card processing."
        }
      },
      "required": [
        "CCMP",
        "MN"
      ]
    }
  }
}
ok what part is bad? it validates so i am not sure what part is bad
j
I repeat: that JSON is malformed. Where did it come from?
no it does not validate...
I already answered this in detail, giving you the part that is wrong and a correction.
m
i have tested it on two differnent validators, both are saying valid
j
https://jsoneditoronline.org test there if you don't believe me
I'm telling you it's not valid JSON. I can see it with my eyes.
"structuredDataPrompt": "You are looking for the USER answers to the "
☝️ is that not clearly a mistake?
what validators are you using?
where did the JSON come from?
You need to either escape those quotation marks or use the same curly quotes you do in the rest of your prompt. Better yet just remove them all. 👍
m
sorry for the delay i got stuck on a call
i am confused
it comes up valid for me, what exactly should i do?
j
please scroll to the structuredDataPrompt key-value pair and show me if that is validated on that website
m
j
ok I can see the quotes are escaped in that window, which is different to what you pasted in here
ok I see the issue actually, you aren't pasting the JSON in backticks so they get removed
when you share code or structured data like JSON in Discord I'd recommend pasting it wrapped in 
backticks
in the future
` is a backtick
m
ok
i am new to discord, sorry
Copy code
{
  "analysisPlan": {
    "summaryPrompt": "You are an expert transcript evaluator. You will be given a Transcript of a call and you must determine if the call was a success. The most important details are the answers from the two questions the AI participant asks:  ---QUESTIONS--- ~“And who do you process your credit card machine with?” and ~“What is the machine that you guys are using?” ---END QUESTIONS---  It is vital that you identify the answers to these two questions. if there is no answer then the call was a failure.",
    "summaryRequestTimeoutSeconds": 20,
    "structuredDataRequestTimeoutSeconds": 20,
    "successEvaluationPrompt": "It is vital that you identify the answers to these two questions. if either question is not answered then the call was a failure.  ---QUESTIONS--- ~“And who do you process your credit card machine with?” and ~“What is the machine that you guys are using?” ---END QUESTIONS---",
    "successEvaluationRubric": "PassFail",
    "successEvaluationRequestTimeoutSeconds": 20,
    "structuredDataPrompt": "You are looking for the USER answers to the \"CCMP\" and the \"MN\" in the transcript. Find the user answers for the following questions:  ---QUESTIONS--- ~“And who do you process your credit card machine with?” and ~“What is the machine that you guys are using?” ---END QUESTIONS---  ---STRUCTURE-- 1. The answer to: And who do you process your credit card machine with? is the \"CCMP\".  2. The answer What is the machine that you guys are using? is the \"MN\".  ---END STRUCTURE--  --GOAL DATA-- - Credit Card Machine Processing (CCMP) - Machine Name (MN)",
    "structuredDataSchema": {
      "type": "object",
      "properties": {
        "CCMP": {
          "type": "string",
          "description": "The company or service used for processing credit card machines."
        },
        "MN": {
          "type": "string",
          "description": "The name of the machine that is used for credit card processing."
        }
      },
      "required": [
        "CCMP",
        "MN"
      ]
    }
  }
}
j
since that's not the issue, I'll leave it to the support team 🙂
just in case this is a issue on the Vapi side, and since your summaryPrompt works, as a workaround you could try changing your summaryPrompt to output pipe-separated data as 
summary|ccmp|mn
, and then on your backend instead of processing the structured data, split the summary by the pipe character
in case you found a bug, that's just a workaround until they get around to fixing it
in fact your summaryPrompt appears to not be being used to summarise anything, so instead of asking it to check if the questions are answered you can simply ask it to extract them and put it the format 
{ccmp}|{mn}
m
well right now as a work around i have used chat GPT, which uses the same prompt, then reads the transcript then uses that exact prompt to get the info i need and puts it into a speard sheet
but i really need it to work inside of vapi
also the propet works 100% of the time when calling from the "call button" inside the Assistnat. it only does not work when we do calls to real numbers
s
this is what your sending "analysis": { "summary": "I apologize, but the transcript you provided is very brief and does not contain enough information for me to summarize in 2-3 sentences. The transcript only includes an introduction from Danny, who is calling from a merchant support verification team. There is no further conversation or content to summarize.", "successEvaluation": "false" }
@MadHatMedia already solution is given to you, check on your end.
closing this ticket for now.
8 Views
PreviousNext
Powered by