Is the test suite API still avaliable? Looks like all the documenation has been removed. Am I missing something?

Hi falseVacuum, The Test Suite feature for automating the testing of AI voice agents is available, and you can find detailed [documentation here](https://docs.vapi.ai/test/test-suites). It includes creating, configuring, and executing test suites. Tools like TestSuiteRunTestAttempt can now accept

callId

and

metadata

for better reference and tracking. If you're looking for updates, the changelog on [March 30, 2025](https://docs.vapi.ai/changelog/2025/3/30) highlights recent enhancements. Let me know if you need further assistance with specific aspects of the Test Suite functionality.

I was refering to the API (to programmatically manage test suites). As referenced here: https://docs.vapi.ai/changelog/2025/2/25 it appears all the links to the API documentation are now broken, and it's not in the swagger spec.

Test suites are unfortunately no longer accessible via the API since they are a work-in-progress. It is still available via the dashboard, but generally we recommend not using it for production use-cases. For performing tests, either run a manual test call or use a third party AI voice testing service

Thanks for the update!

Hello, Sorry for the inconvenience and thanks for your patience. Don’t hesitate to contact us again let us know if you need further assistance.

Is this still the case? I dont see any notice in the dashboard related to this and I've been struggling to get them working. If they are expected to work, how would I override my assistant variables https://docs.vapi.ai/assistants/dynamic-variables#overview ?

Could you please elaborate on what you are trying to accomplish with assistant variables? More specifically: - what is the expected result - what is the actual result

We have variables in our system prompt that we pass in at runtime via the API. In the test suite theres no way to pass those variables in as part of a test case. The test runs with the templates unhydrated and fail. The expected experience is that I can provide variables to each test case. I've hear that others are solving this by making a clone of their assistant to test against.

In the current state of tool tests, you can define some default values for the variables but mostly they will be replaced with simple values like 0 or 1 for integers and the first defined string value for all strings just to make sure the server is able to communicate even in error. The best way to test is just by running a short call with just the tool alone then reviewing the logs afterwards

I wanted to circle back here to note that the recent update to the evals feature solved the variables issue I described above, thank you 👍