Facing issues with inconsistent test results
# support
r
I’m experiencing inconsistent test outcomes - the same set of tests sometimes pass and sometimes fail. What could be causing this behavior? Additionally, is there a way to run individual tests from a test suite? https://cdn.discordapp.com/attachments/1440681151940268152/1440681152183799889/Screenshot_2025-11-19_at_6.02.00_PM.png?ex=6942a2e1&is=69415161&hm=b5af7e46dbd7d0f4e0c9ca8432bd43d60c26a233e44222da6e7ae6472bd0425e&
v
Inconsistent test results often occur due to the AI tester's simulation variability and the LLM-based evaluation process, which can yield different outcomes on repeated runs—even with the same script and rubric. Tip: To reduce variability, set the "Attempts" value for each test case to 1 and ensure your scripts and rubrics are as specific as possible. For more details, see the [Test Suites documentation](https://docs.vapi.ai/test/test-suites). Source: - [Test Suites documentation](https://docs.vapi.ai/test/test-suites)
3 Views