Token difference between two calls of same assista...
# supporta
Ahmer
08/15/2025, 1:41 PMCall ID: 6f9854d9-081f-4a23-953e-e3293a761a00 (Why does it utilizes 23000 tokens)
Call ID: 8f78e6ec-f26a-41e0-bfe6-cd7a7a4dd01d (Whereas this call of same duration and more words spoken has only 5000 tokens)
Why did it happen and how can I resolve this issue?
Do let me know in case you want any other info from my side.
c
Chiranjeet Mishra
08/19/2025, 1:32 AMIn the call ids you provided, the one that used more tokens has a tool call and the cheaper one doesn't. Some LLM requests use more tokens than others because tokens measure the text length of both your input and the model’s output. A few key reasons:
1. Input size – Longer prompts, context, or system messages use more tokens.
2. Output size – The more detail or longer the response requested, the more tokens are consumed.
3. Language & formatting – Certain words, punctuation, or formatting (like code) break down into more tokens than plain text.
4. Extra context – Features like conversation history, function/tool calls, or hidden metadata can add tokens even if you don’t see them.
In short: the bigger or more complex the request and response, the more tokens it uses.
a
Ahmer
08/20/2025, 9:45 AMBoth of them have same input prompts and almost the same number of words but the token difference of the calls is huge. Kindly look into it. @Vapi
c
Chiranjeet Mishra
08/22/2025, 8:14 AMWe have looked into it further and determined that the tool call is the biggest difference in token usage. Because one of the ids called a tool, it also has to process a toolCallResult which uses more tokens. If both call ids had the same tool calls, they would have the same cost.
This is not an issue but rather expected behavior. If you have any questions regarding this matter, please let us know.
a
Ahmer
08/22/2025, 7:42 PMWhy the tool call has thousands of tokens, I don't think so because I am not doing anything that utilizes so much tokens.
c
Chiranjeet Mishra
08/26/2025, 4:54 AMThanks for raising this — I understand why the numbers look surprising. The difference between your call that used \~5,000 tokens and the one that used \~17,000 tokens isn’t only the tool call itself. Here are the key factors:
- System prompt size: Your assistant’s system prompt alone is already quite large (about 2,000–5,000 tokens), and this gets included each time the model is called.
- Prompt duplication: When requests are converted into the OpenAI message format, parts of the prompt are duplicated to maintain context, which further increases the token count.
- Tool call payload: When a transfer tool call is triggered, the JSON payload plus any returned data are also counted as tokens (both as output and then again as input when passed back into the model).
That’s why you’re seeing higher totals — the tool call isn’t “thousands of tokens” by itself, but combined with the system prompt size and duplication, the numbers can add up quickly.
a
Ahmer
08/26/2025, 5:30 PMI don't understand that if there are same duration of a call and almost same number of input tokens are being used then why is this happening. The point is if there are almost 5000 tokens in the sytem prompt then why did the second call tokens are around 5000??
c
Chiranjeet Mishra
08/28/2025, 5:24 PMWe will retrieve the logs and gather the cost data per turn. Will get back to you shortly
a
Ahmer
09/03/2025, 12:51 PMDid you find anything?
c
Chiranjeet Mishra
09/06/2025, 3:04 AMLLM Token Cost Per Turn Analysis - call 8f78e6ec-f26a-41e0-bfe6-cd7a7a4dd01d
Overall Call Metrics:
- Total LLM Cost: $0.0141225
- Total Prompt Tokens: 5,473
- Total Completion Tokens: 44
- Model: gpt-4o-2024-11-20
Turn Breakdown:
Turn 1
- **User Input**: "Is this Solomon Rogers?"
- **AI Response**: "No. This is Lisa from legal services. How can I assist you today?"
- **Response Length**: \~13 words / \~17 tokens (estimated)
Turn 2
- **User Input**: "Sorry for the call."
- **AI Response**: "Thank you, Sharon, calling legal services. This is Lisa. How can I help you today?" (Initial greeting)
- **Response Length**: \~14 words / \~17 tokens (estimated)
Cost Calculation:
Based on GPT-4o-2024-11-20 pricing:
- Input: $0.0025 per 1K tokens
- Output: $0.01 per 1K tokens
Estimated Cost Per Turn:
Copy code
Total Input Cost: 5,473 tokens × $0.0025/1K = $0.0137
Total Output Cost: 44 tokens × $0.01/1K = $0.0004
Total: $0.0141 ✓ (matches the cost breakdown) Copy code
Turn 1: ~$0.0070 (approximately half of total cost)
Turn 2: ~$0.0071 (approximately half of total cost)Chiranjeet Mishra
09/06/2025, 3:09 AMLLM Token Cost Per Turn Analysis - Second Call (with Tool Calls)
Overall Call Metrics:
- Total LLM Cost: $0.0679 (significantly higher than first call)
- Total Prompt Tokens: 17,326 (3x more than first call)
- Total Completion Tokens: 19 (less than first call)
- Model: gpt-4o-2024-11-20
Conversation Flow:
1. **Turn 1**: User says "Yes. Customer service." → AI responds "Do you want me to forward your call to our customer support representative?"
1) **Turn 2**: User says "Yes." → AI makes tool call to transferCall function
Calculate LLM token costs per turn including tool call overhead
Cost Calculation:
Based on GPT-4o-2024-11-20 pricing:
- Input: $0.0025 per 1K tokens
- Output: $0.01 per 1K tokens
Breakdown:
Copy code
Input Cost: 17,326 tokens × $0.0025/1K = $0.04332
Output Cost: 19 tokens × $0.01/1K = $0.0002
Total: $0.04352
Total with Overhead costs: $0.06792 Views