Quick question: Does vapi trims the context at some point? We use mostly the 4o-mini model, which has a 128k context, but after some digging, some calls might be consuming more than that for a single completion. So, can you confirm if there's any prompt trimming in place when the context gets too big?

@Nomadiction Yes, VAPI does implement prompt trimming to ensure that the input stays within the limits of the model's context window. For the 4o-mini model with a 128k token limit, if the context exceeds this size, VAPI will trim the context to fit within the allowable token range. This ensures that the model can process the input without running into errors due to an oversized context. If you have any specific requirements or need further assistance with managing context sizes, please feel free to ask!

@User can you confirm?

@Nomadiction Could you please provide the call ID associated with the prompt trimming issue you are experiencing? We do not trim the prompt.