I want to understand how VAPI actually works with LLM tokens. For example, if I have given a 1000-token system prompt in Assistant settings, does it send the entire system prompt along with the history to the LLM for processing? Or does it only affect the process the first time? I noticed that calls of the same length show different prompt tokens, and I am wondering why that is. Also, is there any benchmark for how many LLM tokens can be used per minute, minimum and maximum? Does anyone know of any range?

VAPI sends the entire system prompt and conversation history to the LLM with every request. Here's a code snippet from the documentation showing how the system prompt is always included:

python
response = openai.ChatCompletion.create(
    model="gpt-4.1-mini",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        # ... (Add messages from conversation history and current prompt)
    ]
)

This means your 1000-token system prompt will be counted in the prompt tokens for each LLM call. For token usage benchmarks per minute, no official range is provided in the documentation. Source: - [Using Your Server Vapi Docs](https://docs.vapi.ai/customization/custom-llm/using-your-server)

VAPI sends the entire system prompt and conversation history to the LLM with every request. The cost of different calls differs because a different amount of tokens is used depending on the conversation. Also, there's no such range as tokens per minute, minimum, or maximum. You can share call ids here if you have any specific cases that don't make sense.

Can you provide my token consumption history of following calls? I need to understand why it spent this much tokens in these calls. 019b8d1a-c3f3-7995-aa4f-4a9e26955847 019b8d12-5108-777a-857a-7d273091984f 019b8d10-caa6-777a-8579-65598a7c22fe**

--- Campaign / Session ID: Total Cost: $0.1676 Breakdown: 019b8d12-5108-777a-857a-7d273091984f - Transcriber: $0.011 - Model: $0.069 - Voice: $0.0248 - Vapi · Normal: $0.0523 - Analysis: $0.0105 Model-specific details: - Prompt Tokens: 27,129 - Completion Tokens: 119 - Cached Prompt Tokens: 3,584 --- 2- 019b8d1a-c3f3-7995-aa4f-4a9e26955847 Total Cost: $0.1718 Breakdown: - Transcriber: $0.0114 - Model: $0.052 - Voice: $0.0406 - Vapi · Normal: $0.0538 - Analysis: $0.0141 Token usage: - Prompt Tokens: 20,088 - Completion Tokens: 179 - Cached Prompt Tokens: 6,272 The cost of each call varies due to multiple factors. For example, in Call #1, more tokens were consumed due to multiple tool calls, but the bot spoke less overall, which lowered the voice cost. In Call #2, there were fewer turns (10 turns), so fewer tokens were used, but the bot spoke more per turn, increasing the voice cost. You can check the transcripts in your call logs for reference.