Hey, I am trying to monitor the max
# general-englishm
marem
06/27/2024, 7:58 AMHey, I am trying to monitor the max token on my side. I was wondering what is the difference on the cost breakdown between "prompt tokens" and "completion tokens"?
https://cdn.discordapp.com/attachments/1211484481848873020/1255794351607386154/Screenshot_from_2024-06-27_09-56-56.png?ex=667e6d20&is=667d1ba0&hm=aa62f67f9caa8f1647627a5cce8ef734dc94962dbdd0814f6cdf5faea3b80016&
marem
06/27/2024, 9:01 AMwhat I have in mind:
- 23178 is the cumulative sum of tokens sent to the LLM, for instance if during the conversation I have 24 calls to the llm with a window of 1k tokens
- 327 is the number of tokens answered by the llm during the full conversation
l
L2.
06/27/2024, 2:38 PMPrompt = input
Completion = output
Ur prompt seems pretty large compared to your completion tokens, what is your use case?
s
SajithSPY
06/28/2024, 1:53 AMif you use knowledge base, you can expect this much input(prompt) tokens
3 Views