Massive Token Discrepancy – Vapi Key vs. Own OpenA...
# support
d
Hi Vapi Team, We’re actively using your GPT-powered phone assistant service with a fully token-optimized setup – streamlined prompts, highly compressed tool context, and minimal turn count. However, we recently uncovered a massive and reproducible discrepancy in token usage between your OpenAI key and our own key, even for identical calls. ### 🧪 What we did We ran the same test call twice: 1. Once with your Vapi OpenAI key 2. Once with our own OpenAI API key, so we could monitor usage on OpenAI’s dashboard directly. > Both calls used: > Identical prompts and context > Same toolkits > Identical user interaction > GPT-4.1 (gpt-4-turbo) ### 🧾 Call IDs * Vapi Key Call: x-call-id: a847c3e6-cfbc-497f-bf62-21e397a32d95 * Own Key Call: x-call-id: 4f004ae9-9149-4736-a13f-a29b5eaed245 ### 📊 Cost Breakdown – Discrepancy Example | Description | Prompt Tokens | Completion Tokens | LLM Cost (USD) | Notes | | --- | --- | --- | --- | --- | | Vapi Key | 21,586 | 42 | 0.0435 | 🚨 Wildly inflated | | Own OpenAI Key | 10,811 | 40 | 0.0000 | ✅ As expected | > 💥 This is not just 2x – the discrepancy escalates with longer calls, growing exponentially the more tokens are exchanged. This is just the smallest call example.
🧵 Comparison – Actual Call Content (Bot + User Messages) 🔴 Vapi Key Call (a847c3e6-...)
Copy code
json

{
  "role": "bot",
  "message": "Guten Tag, mein Name ist Mia vom Kundenservice der BT Group. Wie kann ich Ihnen helfen?"
},
{
  "role": "user",
  "message": "Tell me, how are you?"
},
{
  "role": "bot",
  "message": "Thank you for asking. I'm here and ready to support you. How can I assist you today?"
}
✅ Own Key Call (4f004ae9-...)
Copy code
json

{
  "role": "bot",
  "message": "Guten Tag, mein Name ist Mia vom Kunden der BT Group. Wie kann ich Ihnen helfen?"
},
{
  "role": "user",
  "message": "Hello, Mia. How are you?"
},
{
  "role": "bot",
  "message": "Hello. I'm doing well. Thank you. How can I help you today?"
}
🧠 These are semantically identical conversations – same length, same system prompt, same structure. 🚨 This Indicates a Severe Accounting Problem • Token usage with your key almost doubles even for tiny calls. • With longer calls, the token cost difference scales up drastically, indicating something is deeply off. • According to your own logs, 21,586 prompt tokens were used for a 15-second call with no tools, no long chat history. • With our own OpenAI key, the exact same content only used 10,811 prompt tokens – and was free under OpenAI’s token quota. 🧭 Our ask Please investigate: • How tokens are being calculated internally • Whether prior instructions or tool metadata are accumulating or duplicated • Whether a full LLM call payload (visible to us) could be exposed for debugging We’re fully prepared to assist with debugging this issue – but at the current rate, token usage (and therefore pricing) is non-viable for production-grade deployments.
### LLM Prices are also wrong! Also the LLM Prices are wrong! From your pricing page: Large Language Model (LLM) Advanced language processing and generation OpenAI GPT-4.1 - $0.06 / 1M tokens GPT-4.1 Mini - $0.01 / 1M tokens GPT-4.1 Nano - $0.01 / 1M tokens GPT-4.5 Preview - $2.12 / 1M tokens GPT-4o Mini - $0.01 / 1M tokens GPT-4o - $0.07 / 1M tokens ChatGPT-4o (Latest) - $0.14 / 1M tokens ## Example CallID: a847c3e6-cfbc-497f-bf62-21e397a32d95 From Statistic: llm $0.04 ($0.15 / min) ‣ 21586 prompt tokens ‣ 42 completion tokens > This is not 0.06 / 1m token, this is 1.85 / 1m token! > This are 3080% By your price list it would be: 0.06 / 1.000.000 * 21586 = 0.00129516 $ In normal Calls it's a very huge discrepance! CallID: 2afe8baf-3eb4-4b8f-b47f-46a1e95de288 llm $0.46 ($0.19 / min) ‣ 227522 prompt tokens ‣ 384 completion tokens > This is not 0.06 / 1m token, this is 2.02 / 1m token! > This are 3366% By your price list it would be: 0.06 / 1.000.000 * 21586 = 0.01365132
c
Thank you for sharing your information! I’ll keep you posted as we check on our end. Your patience is truly appreciated..
d
Hey, any news why the price is wrong added to the GPT usage?
c
Hey! To help track down this issue, could you share: - The call ID - When exactly this happened (the timestamp) - What response you expected to get - What response you actually got instead This would really help us figure out what went wrong!
d
Hey, the prices are on all calls wrong when we are using the Vapi GPT (not with our own OpenAI Key): CallID: 2afe8baf-3eb4-4b8f-b47f-46a1e95de288 Example: gpt-4.1 used: $0.46 ($0.19 / min) ‣ 227522 prompt tokens ‣ 384 completion tokens Price in Vapi Price list: GPT-4.1 $0.06 / 1M tokens 0.06 / 1.000.000 * 227522 = 0,01365132 (0.46 invoiced) Call: dd920365-358b-4609-aedb-19a86b308941 pt-4.1 used: llm $0.63 ($0.20 / min) ‣ 311505 prompt tokens ‣ 526 completion tokens Price in Vapi Price list: GPT-4.1 $0.06 / 1M tokens 0.06 / 1.000.000 * 311505 = 0,0186903 (0.63 invoiced) ,... You can see this in all calls! There is not a single call where the tokens are calculated correct to your pricing table. https://cdn.discordapp.com/attachments/1399869312558174249/1405474794043998252/image.png?ex=689ef5e7&is=689da467&hm=f3fe7bab4843bb552d3add0707df31443dddad3a3cb3de16f1990646e2985eb6&
c
Looking into it
d
Any news?
3 Views