I am using custom LLM and the VAPI logs show that the call would have 600 latency with my own LLM, which is much less than if I used one of the already integrated providers. I need to know if that latency is real.

@ZEGOTECK That's great to hear that your custom LLM is showing promising latency times in the logs! To confirm if the 600ms latency is accurate, you might want to conduct real-time tests under various conditions to see if the performance is consistent with the logs. If you have any specific scenarios or further details you'd like to test or discuss, please let me know, and I'll be happy to assist you further. If you need a more in-depth analysis or verification, please ask support for more info on this.

@Vapi How do I do those tests?

@ZEGOTECK To conduct real-time tests for your custom LLM with VAPI, you can follow these steps: 1. **Test Environment Setup**: Set up a controlled environment where you can simulate the use of your custom LLM with VAPI. This should mimic the production environment as closely as possible. 2. **Test Scenarios**: Define a set of test scenarios that cover typical use cases of your voice application. This should include a variety of inputs to test the robustness of the LLM. 3. **Automated Testing**: If possible, automate the testing process to send voice requests to your VAPI integration and measure the response times. This can be done using scripts or testing software. 4. **Manual Testing**: In addition to automated tests, perform manual tests by interacting with the voice application in real-time to get a sense of the user experience. 5. **Monitoring Tools**: Utilize monitoring tools to measure the latency of the requests. These tools can often provide detailed insights into where delays may be occurring. 6. **Analyze Results**: After conducting the tests, analyze the results to see if the latency consistently matches the 600ms reported in the logs. Look for any outliers or patterns that may indicate performance issues. 7. **Optimization**: If you find that the latency is higher than expected, consider optimizing your LLM or the network configuration to improve performance. 8. **Support**: If you're unsure about how to proceed with any of these steps or if you encounter issues, don't hesitate to reach out to VAPI support for assistance. Remember to test under different network conditions and loads to ensure that your latency measurements are accurate and representative of real-world usage. If you need further assistance with setting up the tests or have any other questions, please let me know!

Hey @ZEGOTECK To help track down this issue, could you share: - The call ID - When exactly this happened (the timestamp) This would really help me figure out what went wrong!