Inconsistent test results often occur due to the AI tester's simulation variability and the LLM-based evaluation process, which can yield different outcomes on repeated runs—even with the same script and rubric.
Tip: To reduce variability, set the "Attempts" value for each test case to 1 and ensure your scripts and rubrics are as specific as possible.
For more details, see the [Test Suites documentation](
https://docs.vapi.ai/test/test-suites).
Source:
- [Test Suites documentation](
https://docs.vapi.ai/test/test-suites)