While this used to work, it seems that since around last Thursday or Friday, every agent using Anthropic Opus 3, Sonnet 3.5, or Haiku 3.5 now fails if the agent includes a tool call with the following error: "An error occurred in this call: pipeline-error-anthropic-llm-failed"

@Jack C I'm sorry to hear that you're experiencing issues with the Anthropic models. It seems like there might be a problem with the pipeline for those specific models. To assist you better, could you please provide more details about the context in which the error occurs? Are you calling a specific function within the VAPI environment when this happens? Any additional information you can provide will help us troubleshoot the issue more effectively.

Calling any tool call causes it to fail

Is anyone looking into this? Is there a solution or workaround for this ?

@Achal I understand the urgency of the situation. Rest assured, the appropriate team would typically be notified of such widespread issues and would be working on a resolution. In the meantime, as a workaround, you might consider using alternative models that are not experiencing this issue, if that's a possibility for your application. If you need immediate assistance or a status update on the issue, I would recommend reaching out to the support team directly. They would have the most current information on any ongoing incidents or maintenance that might be affecting the Anthropic models. Would you like me to assist you with anything else at the moment?

Yes we switched to OpenAI for the time being but Anthropic was better at sticking to the system prompts overall..

@Achal I'm glad to hear that you've found a temporary solution by switching to OpenAI models. It's understandable that you'd prefer the performance of Anthropic models for adherence to system prompts. The differences in model behaviors can indeed impact the user experience and the effectiveness of your application. While the issue with the Anthropic models is being resolved, you might want to experiment with fine-tuning the OpenAI models or adjusting the prompts to better align with your desired outcomes. Prompt engineering can sometimes yield improvements in how models adhere to instructions. If there's anything specific you'd like assistance with, such as prompt design or other aspects of working with OpenAI models, please let me know, and I'll do my best to help.

We experienced the same today and lost a customer because all their calls were just hanging. There needs to be some monitoring and alerting when things break

@Jack C @Achal @olivaw can you share the call ids with failures?

@User 08703842-bd24-48d1-838d-f133df1abdd5 6efa5f6c-f071-4e75-a663-3c7f8ba150e9

Here are some IDs form us (Jack & Achal) b3117316-433c-4a70-8e35-b8754b4ace25 162546ea-399a-496d-89d5-a897bb0e7033 59345b3b-3b84-4200-8efc-ea2a91135112

@Achal @olivaw @Jack C here's the ticket created for it https://linear.app/vapi/issue/VAP-1808/error-process-anthorphic-kb-request

Thanks for the update Shubham!

@Achal this issue fixed will be realease by end of today.