Hi, we're using automated testing to stress our app to see what happens when we go over our concurrency limit (currently set at 35). But before hitting our concurrency limit we're getting these error messages: "Create assistant failed: HTTP 429 - Rate limit exceeded" (we create a new assistant before each call, and delete it after). Are there any more details on API rate limits? I assume at some point they will become the bottleneck rather than the concurrency?

Hi @TimG, The 429 error occurs when you cross your default API rate limit of 50 requests per second. The rate limit applies to all endpoints and it will send 429 status when the limit is reached without additional information. You have mentioned that you are creating a new assistant and deleting it after each call, have you looked into transient assistant configurations? This will help you create temporary assistants for each call. More information about it here: https://docs.vapi.ai/assistants/concepts/transient-vs-permanent-configurations

Hi @Bharath , thanks for the info, much appreciated. Yes I looked into transient assistants a while ago, but they didn't really support our use case. Maybe things have changed now? The problem we were having is that we need to include info in the assistant config which we don't want accessible to the user. We're using the web sdk to create calls. I previously looked into if there was a way of generating calls with a transient assisstant on our backend and passing the call to the frontend, but that wasn't supported at the time. Do you have any ideas for creating transient assistants via the web sdk without sending the confidential parts of the config through the frontend?

Hi @TimG, yes, transient assistant will not help when you are dealing with webcalls and you do not want to send your prompts and other details to the client side. Could you look into websocket transport? This might require significant changes to your architecture if you are not already using websockets much. However, websocket transport will help you for this specific use case. You can have the client make a call, the call request will be sent to your server -> then your server will take required information and create a transient assistant configuration, and send an API request to Vapi for initiating a call. -> Now Vapi returns you with a websocketCallUrl -> which you can then return it to the client side -> Now your client can directly connect to the websocket call. This will prevent the you from exposing your prompts and tools to the client, and it will also reduce the API calls you make to Vapi, reducing your rate limiting errors. For more information about it, please check out the websocket transport here: https://docs.vapi.ai/calls/websocket-transport

Thanks @Bharath , I haven't seen the websocket option. Will check it out, thanks!

You're welcome, feel free to create a new ticket if you are facing any issues with Vapi in the future. We are here to help.