Whoops, there was an error running the test suite. Please try again or reach out to us. Vapi AI #support

Whoops, there was an error running the test suite....

Lucas van Staden

06/29/2025, 1:27 PM

Hello, I am creating test suite, tests and test runs via API I am however getting back a fail to run tests, but no real indocation in what coudl be wrong. All I get is : Whoops, there was an error running the test suite. Please try again or reach out to us. So, I must be doing soemthingw rong/not linking soemthing up right, but no clue what, everything loosk good to me. if I call thw test result from api

Copy code

{
  "id": "6f1a14ca-4763-4064-b271-d3b30930993c",
  "testSuiteId": "1b36f447-e9cf-4f72-afd4-3858954083d0",
  "orgId": "5f8d7237-69c6-4884-a7bb-f9f88bfcee4f",
  "testResults": [
    {
      "test": {
        "id": "7d776f79-bfa5-4377-8b1d-8788fe49a3c6",
        "name": "dynamic test: reschedule",
        "type": "voice",
        "orgId": "5f8d7237-69c6-4884-a7bb-f9f88bfcee4f",
        "script": "Greet the assistant politely.\nI want to reschedule my missed appointments to tomorrow\nConfirm it was rescheduled\nEnd the call politely.",
        "scorers": [
          {
            "type": "ai",
            "rubric": "The assistant should mention about missed appointments, and allow user to reschedule."
          }
        ],
        "createdAt": "2025-06-29T13:15:28.507Z",
        "updatedAt": "2025-06-29T13:15:28.507Z",
        "numAttempts": 1,
        "testSuiteId": "1b36f447-e9cf-4f72-afd4-3858954083d0"
      },
      "attempts": [
        {
          "metadata": {
            "sessionId": "117399"
          },
          "scorerResults": []
        }
      ]
    }
  ],
  "status": "failed",
  "createdAt": "2025-06-29T13:15:28.886Z",
  "updatedAt": "2025-06-29T13:15:29.254Z"
}

any help to debug what issue may be will be much appreciated. https://cdn.discordapp.com/attachments/1388873549501956257/1388873549699223623/image.png?ex=686290ca&is=68613f4a&hm=1a0e68893df7bb690e6b9234c190be0c09b9acc7d549f22eaa65cd0c88dd31a3&

Lucas van Staden

06/30/2025, 3:41 AM

I have managed to solve this issue, and here is what I changed in cas eit is of help to someone The problem was in creating the test suite. In my setup i have a phone number to which I am assigning an assitant that I want to test The test suite is setup to call that number. My setup xml was incorrect

Copy code

'name' => 'dynamic test: reschedule',
                'targetPlan' => [
                    'assistantId' => $assistant['id'] ?? null,
                ],
                'phoneNumberId' => $_ENV['test_phone_id'],

as aposed to

Copy code

'name' => 'dynamic test: reschedule',
                'targetPlan' => [
                    'phoneNumberId' => $_ENV['test_phone_id'] ?? null,
                ],

i dropped the assistentID setup and moved phone number into the target property

Kyle Brunker

07/01/2025, 4:39 AM

Nice work! If you need any additional support let us know

Previous Next