https://vapi.ai/ logo
Join Discord
Powered by
Test suits are completely broken
# support
l
c
Hi Lucas van Staden, To address the issue you're encountering with the test called "AI," you can retrieve details about the test suite run to inspect the results and logs. 1. **Get Test Suite Run**: Use the GET method to inspect the specific test suite run by accessing the endpoint:
Copy code
GET https://api.vapi.ai/test-suite/3e31c096-ae69-4211-801f-cf04cff01a77/run/cbbedd6d-9d84-450e-ad14-8016a50b766d
Ensure you include the appropriate authorization token in your request header. 2. **Review Logs and Test Results**: Inspect the response for the 
testResults
and associated logs to diagnose why the AI might be returning unexpected outputs. For further investigation, examining the AI configurations and any recent updates or changes could be beneficial. Let us know if you need a deeper look into specific parts of your configuration. You can refer to the detailed API documentation for further exploration: [Get Test Suite Run](https://docs.vapi.ai/api-reference/test-suite-runs/test-suite-run-controller-find-one). Let us know if this works.
l
Hello, Thanks, but I did not change anything, the system just stopped working. If I loo at the data pulled from teh given url I can clearly see there is an issue when your system pushes the user prompt to generate the claler AI It is not MY ai creation that is wrong it is your creation of teh claler AI that is wrong. Please check properly.
Copy code
"messages": [
                                {
                                    "role": "system",
                                    "time": 1756125121522,
                                    "message": "\n      Your name is SlyShark79, and you are a phone call test assistant that will help the customer test their custom voice agent. \n\n      You will be given a script on how to interact with another voice agent in order to test it. Please follow the script exactly. \n\n      Here is the script, please follow it exactly:\n      1: Greet the caller.\n2: Ask \"How much larger is the sun than the moon?\"\n3: Ask the same question again.\n4: End the call.\n\n      After satisfactorily running through the script, you will call the 'endCall' tool.\n\n      Do not ever mention that you are an AI assistant, or that you are testing the voice agent, or that you have a script to follow.\n      ",
                                    "secondsFromStart": 0
                                }
                            ],
Copy code
"messagesOpenAIFormatted": [
                                {
                                    "content": "\n      Your name is SlyShark79, and you are a phone call test assistant that will help the customer test their custom voice agent. \n\n      You will be given a script on how to interact with another voice agent in order to test it. Please follow the script exactly. \n\n      Here is the script, please follow it exactly:\n      1: Greet the caller.\n2: Ask \"How much larger is the sun than the moon?\"\n3: Ask the same question again.\n4: End the call.\n\n      After satisfactorily running through the script, you will call the 'endCall' tool.\n\n      Do not ever mention that you are an AI assistant, or that you are testing the voice agent, or that you have a script to follow.\n      ",
                                    "role": "system"
                                }
                            ],
                            "transcript": "User: nine hundred and eight nine of four hundred eight four nine zero eight seven d six three seven does not exist",
                            "logUrl": "https://calllogs.vapi.ai/ae622d63-d33f-4e82-9e76-f0495051daa0/9a39deb6-5295-47b6-a4ad-31ffb640ecf3-736fbe96-9e6c-4e29-8d09-2b85613e4469-1756125131703.jsonl.gz",
                            "nodes": [],
You can see YOUR prompt to setup teh caller AI is not working.
nope, seems intermittedly broken. even tests created in your UI breaks teh same way. tests / tests suites are basicaly useless at present. to many bugs and issues. nice idea, not ready. needs to be properly tested and fixed
I think I managed to track this down. Took a lot of experimentation. Issue seems to be a deleted tool id that was still linked to the assistant.
c
Glad you were able to get this resolved. If anything else comes up or you need further assistance, we’re here to help.
3 Views
PreviousNext
Powered by