I would like to access evals for my stored openai sessions is this possible I can set this somewhere even programatically? Thanks! https://platform.openai.com/docs/guides/evals

@AiTitus :) Certainly! OpenAI provides an evaluation feature that allows you to assess the performance of your models on specific tasks. To access evaluations for your stored OpenAI sessions, you can use the OpenAI API to programmatically create and retrieve evaluations. Here's a general outline of how you can set up evaluations programmatically: 1. **Define your tasks**: Create a JSON file with the prompts and completion pairs that you want to evaluate your model on. 2. **Use the OpenAI API**: Write a script that uses the OpenAI API to submit your tasks for evaluation. You'll need to use the

Create

and

Retrieve

endpoints for evaluations. 3. **Submit your tasks**: Use the

Create

endpoint to submit your tasks to OpenAI. You'll receive an evaluation ID in response. 4. **Retrieve your results**: Use the

Retrieve

endpoint with your evaluation ID to get the results of your evaluation. Please note that you'll need to have an API key to authenticate your requests to the OpenAI API. Ensure that you handle your API key securely. If you need more detailed instructions or assistance with the API calls, please let me know, and I can provide further guidance.

Hey as of now we don't have this but you can raise the feature request over here: https://vapi.canny.io