Hey,
After investigating there are the following problem we see:
1- ROOT CAUSE: Langfuse injects full KB context on EVERY turn
Fix: Langfuse should only log/observe, it must NEVER inject context mid-call. Move all Langfuse writes to async post-call hooks. The LLM context should be managed solely by Vapi, not by a logging layer.
2- Anthropic 429 Rate Limit (4M tokens/min hit)
Fix: Fix the Langfuse injection issue (Issue #1) this will immediately reduce token usage by ~80%. Also implement token-efficient context trimming: only send the last N turns + system prompt, not the entire history every time.
3- Google Gemini post-call analysis timed out (5s)
Fix: Increase the post-call analysis timeout beyond 5s (10–15s is reasonable). Use a lighter model for structured data extraction