Vapi is the Voice AI platform for developers.

Vapi AI

Hey team

When we create an Eval and choose the LLM we are wanting to train the Eval in say 'ChatGPT 4o Latest' it doesn't save correctly. See this Loom... https://www.loom.com/share/c7b4eb6c570d4c7c87b53d54b04eb037

Hi there,

Thank you for your message. Our team is currently out of the office. We operate Monday through Friday, from **9:00 AM to 8:00 PM Pacific Standard Time (PST)**.

We’ll get back to you as soon as possible during our normal business hours.

If your message is urgent, please mark it accordingly or include “URGENT” in the subject line, and we’ll do our best to respond promptly.

Warm regards,
**Vapi**
Customer Support Team

Ah yep — this is expected behavior.

The evaluator model only saves if you have at least one turn set to **LLM-as-a-judge** (`isEvaluation: true` + `evaluationApproach: "ai"`). The model actually lives on that turn (`judgePlan.model`).

So if you:
• pick a model in the top-level evaluator config
• only use exact / regex checks
• save the eval

there’s nowhere to store the model, so it doesn’t persist.

Adding a single LLM-as-a-judge turn should make it stick.