When using gpt-4.1 model, I'm always, for all my calls the costs that correspond exactly to $2 per M input token and $8 per M output token. It's exactly what is being shown here https://platform.openai.com/docs/models/compare?model=gpt-4.1 But that would mean that no input token is cached, which is surprising to me. Is that normal ? That no input token is cached for any call ? You can check on any call from my orgs. Here are a few examples from my org id 94a0b476-b4f6-4f9c-89e8-7a8475c70adb - Call id 0298ba6e-cd87-4a79-92dd-9f6e66f5e4a1 - Call id 33aaac31-0a0a-489a-a98b-56bbbb8af13c - Call id 9f14b1dc-fae7-49b2-81d7-eef2b33c6e86

Hey @Jeebs from eva.be, prompt caching is done directly from OpenAI's end, not from our end https://platform.openai.com/docs/guides/prompt-caching, and yes, we don't account for it at the moment.

@Sahil indeed, and I'm pretty sure they actually do cache most of the prompt, because our conversation is just a big repeat with a new message at each turn of the conversation 😬 So openai is probably invoicing quite a lot less than the cost you are passing on us, or do I understand something wrong ?

The way we calculate the LLM cost is by taking the original price and multiplying it by the number of tokens used on the OpenAI side, since we don't receive any discounts. However, we can offer you some coupon credit for it. Could you give me a rough estimate of how many tokens you've used?

@Sahil thanks for sharing and for transparency. Quick question : if I use my own openai key, I will get the "real" openai price, which will be lower thanks to the cached input price. But are you still going to apply these magic tricks described here if I am using my own key instead of yours ? https://vapi.ai/blog/how-we-solved-latency-at-vapi

Assuming that you 30% of your total token is cached the total refund amount is coming to be 10.53+14.09 approximately $25.39

Sure, thanks. I'm not even sure how much openai would cache, but I guess we won't be making a refund each month so you might want to implement this cost tracking ? Or should I rather use my own key (but again, afraid of losing your optimizations made here : https://vapi.ai/blog/how-we-solved-latency-at-vapi)

Currently, adding the key is only option and we are pretty buried in other priority items. So, it will take good amount of time to fix it.

@Sahil FYI I've made a test with one of my vapi org, putting my OpenAI API key. I have a huge difference with vapi, because >50% of my input tokens are cached, which is wayyyy cheaper than non-cached input. That means that using openai key from vapi is actually a lot more expensive. https://cdn.discordapp.com/attachments/1398012661144223985/1412469481510539385/image.png?ex=68b86836&is=68b716b6&hm=34523a9e7c7f01e5169a481a38a4e218eb08f42d66ef0460b85c7bf60688a998&

will get this added.