https://vapi.ai/ logo
Join Discord
Powered by
Turn-level metrics per call
# support
r
How can we get turn-by-turn metrics for each call (e.g., latency for STT, LLM, TTS, time to first byte, etc.)? I know Vapi has an integration with Langfuse, but how can we get turn-by-turn trace metrics in Langfuse when connecting to Vapi? Currently, Vapi doesn't seem to report detailed traces to langfuse (such as a detailed breakdown for each turn, such as STT latency, LLM latency/time to first token, TTS latency/time to first byte) https://vapi.ai/community/m/1361884993697222846 -- this seems to be a pretty well-known issue, and im honestly surprised something as big as Vapi doesn't offer this (or at least make it clear how this is done)
c
Hey real.spidey, Thanks for reaching out! We're currently swamped with a bunch of tickets and it's taking us longer than usual to get back to everyone. I know waiting isn't fun, but we want to make sure we give your issue the attention it deserves rather than rushing through it. We'll circle back with you soon to get this sorted out.
Hi real.spidey, To obtain turn-by-turn metrics for each call with Vapi, you can integrate with Langfuse for enhanced telemetry monitoring. However, Vapi's native integration with Langfuse currently doesn’t provide detailed breakdowns for each turn such as STT, LLM, and TTS latencies directly from Vapi. For obtaining detailed metrics like these, you might need to manually enhance the information sent to Langfuse. You can utilize the metadata and tags enrichment feature provided by Vapi to include more granular data in your traces. Here's how to manually incorporate more details: 1. **Enrich Traces with Metadata and Tags**: You can add custom key-value pairs for metrics like STT latency, LLM processing time, and TTS latency using the 
assistant.observabilityPlan.metadata
field in Vapi.
Copy code
json
   {
     "assistant.observabilityPlan.metadata": {
       "stt_latency": "value",
       "llm_processing_time": "value",
       "tts_latency": "value"
     }
   }
2. **Use Tags for Filtering**: Attach searchable labels to differentiate the types or stages of data in your traces through the 
assistant.observabilityPlan.tags
field. 3. **View in Langfuse**: Once you have enriched your traces with these details, they should appear in your Langfuse dashboard, allowing for better analysis and debugging. For more detailed setup information, please find it in the [Langfuse Integration with Vapi](https://docs.vapi.ai/providers/observability/langfuse) documentation. This page includes instructions on adding metadata and tags to enhance your tracing capabilities.
PreviousNext
Powered by