https://vapi.ai/ logo
Join Discord
Powered by
Evals stuck in "Running" state indefinitely / new ...
# support
n
Hey team, having an issue with running any evals, I am unable to run them at all. What's happening: * Since May 23, 2026 ~03:21 PM, I have a few eval that's been stuck in "Running" state on the Vapi dashboard — it never completes or fails out. * Every new eval I try to run now also gets stuck. Locally I see retries on 429s, then a timeout after 60 attempts:
Copy code
[warning] retry: got response with status 429, will retry in 1000ms, 3 attempts left
  [warning] retry: got response with status 429, will retry in 2000ms, 2 attempts left
  [warning] retry: got response with status 429, will retry in 4000ms, 1 attempt left
  [error] Failed to run transient eval: "Eval run timed out after 60 attempts"
The 429s I am getting are trying to get the result of evals (i.e. polling 
GET /eval/run/{id}
) Could you please help me with cancelling all the "Running" evals and start running new evals. Account info: * Email: davidson.harris1990@gmail.com * I can't provide a specific eval ID — the stuck runs never returned one to me on the client side. If you query the last 10 evals on this account, any of them should show the issue. Happy to share more logs or run anything on my end. Thanks!
s
Hi, Thanks for the detailed logs - we looked into this thoroughly. Your eval runs aren't actually stuck. Every single one completed successfully on the backend - most finished in under 54 milliseconds. The issue is that your client never got to read the "completed" status because it was hitting our API rate limit while polling for results.
Here's what happened: when you triggered multiple evals concurrently, each one opened its own polling loop calling GET /eval/run/{id} roughly every second. With 5+ runs polling simultaneously, you quickly hit the rate limit ceiling, all status checks started returning 429, and the client timed out before it could read the final results. That's why your dashboard shows "running" - it's stale data, not an actual stuck state.
How to avoid this going forward: Run evals sequentially instead of concurrently. Wait for each one to finish before starting the next:
Copy code
javascript
// Instead of this (saturates rate limit):
await Promise.all([runEval(id1), runEval(id2), runEval(id3)])

// Do this (one at a time):
for (const id of evalIds) {
  await runEval(id)
}
This keeps your polling at 1 request/second instead of 5+, which stays well under the rate limit.
The "running" entries in your dashboard are stale but the underlying data is complete. If you hit 429s again on a single sequential run, just wait a few seconds and retry - a single run polling at 1 req/second should not saturate the limit under normal conditions.
Let us know if you have any questions.
m
Hi team, I am getting a similar issue. Even when I start a new eval run, it stays running indefinitely and never resolves. The following are a few logs for debugging. Org ID: 5524944c-9133-4dee-8ef7-61921e7ad354 Affected Run IDs: 53525984-4a78-4443-b524-c15ed10ee2a6 f3ad04b6-c4c6-4bc2-b95b-ad462b17896d Assistant ID: aa31b41d-fe3b-4d4d-9dd7-a2372d5f2ee1 Issue: Eval runs immediately end with endedAt before createdAt, status stuck as queued, no endedReason returned, and no results produced.
Following are the logs for the above issue in case this helps in understanding the issue:
Copy code
curl -X GET "https://api.vapi.ai/eval/run" -H "Authorization: Bearer ****"                                     
{"results":[{"id":"f3ad04b6-c4c6-4bc2-b95b-ad462b17896d","orgId":"5524944c-9133-4dee-8ef7-61921e7ad354","type":"eval","status":"queued","target":{"type":"assistant","assistantId":"aa31b41d-fe3b-4d4d-9dd7-a2372d5f2ee1"},"startedAt":"2026-05-25T10:07:14.449Z","endedAt":"2026-05-25T10:07:14.320Z","createdAt":"2026-05-25T10:07:14.321Z","updatedAt":"2026-05-25T10:07:14.450Z"}],"metadata":{"totalItems":1,"itemsPerPage":100,"currentPage":1}}
s
Hi, Thanks for sharing the detailed logs - we've looked into both run IDs.
What happened: The eval that was set up for assistant aa31b41d was deleted from your account around 8:05 AM UTC on May 25th. After that deletion, every new eval run you triggered - including 53525984 and f3ad04b6 - had no test cases to execute. The runs start, find zero evals attached to the assistant, and exit almost instantly without ever updating the status from queued. That's why they appear stuck with no results and no endedReason.
How to fix it: You need to re-create the eval before triggering new runs: Go to the Vapi dashboard → Evals Create a new eval for assistant aa31b41d with your test cases and judge plan
Trigger a new run - it should now execute and produce results normally
n
Thank you for looking into this so quick. I think @Mohit Charkha your issue is separate from mine, belongs in a separate ticket. Regarding original issue, I have tried to run 1 eval at a time, I even increased polling to be every 5 seconds, but I still see the same issue (i.e. stuck in running). This morning I have run 3 evals separately and none of them ever finished, after waiting for more than a minute. Note, previously I was always running them sequentially (always worked) apart from the day of the issue where I tried asynchronously. After that time, I no longer can run evals. Could it maybe be because I have evals stuck in "Running" state? Maybe clearing them would fix the issue?
m
I created a completely new account and set up assistants, squads, and evals from scratch. However, I am still facing the same issue — the evals remain stuck in the "running" state on the dashboard indefinitely. From the logs, I can see that the eval status is actually "queued". Additionally, the 
endedAt
timestamp is present and is a few milliseconds earlier than both 
startedAt
and 
createdAt
. I suspect this timestamp inconsistency may be why the eval is not being executed, even though it still appears as "running" on the dashboard. I have attached the new logs for reference:
Copy code
curl -X GET "https://api.vapi.ai/eval/run" -H "Authorization: Bearer ****"

{
  "results": [
    {
      "id": "541d6964-c2bd-48de-8805-d20df954d23c",
      "orgId": "9d432532-a64d-479c-8651-f8099e8957c2",
      "type": "eval",
      "status": "queued",
      "target": {
        "type": "assistant",
        "assistantId": "7e0770f8-aa6f-427d-aa01-328667b91ab7"
      },
      "startedAt": "2026-05-26T07:18:51.289Z",
      "endedAt": "2026-05-26T07:18:51.088Z",
      "createdAt": "2026-05-26T07:18:51.089Z",
      "updatedAt": "2026-05-26T07:18:51.290Z"
    }
  ],
  "metadata": {
    "totalItems": 1,
    "itemsPerPage": 100,
    "currentPage": 1
  }
}
@N3squik Eventually, the issue we are both facing is similar: evals stuck in the "Running" state indefinitely. So, added it in the same thread.
n
Okay interesting, maybe the whole Evals are down. Interested to see what @Shaunak has to say more on this. I can confirm, I am also seeing status "queued" when polling, and "Running" is seen on the dashboard.
s
Hi N3squik, Thanks for the follow-up - understood that running sequentially didn't resolve it either.
Could you share your Organization ID and any eval run IDs from the recent attempts? You should be able to pull them via the API:
That will help us trace what's happening on the backend with your specific runs.
n
Org id - 
78466729-70a1-4dea-b857-0d921c435e0f
Recent eval run id (run that I just did) - 
d2d71fdb-403b-4ad5-8718-694f356ed0fa
s
Hi N3squik, We've checked your eval run d2d71fdb-403b-4ad5-8718-694f356ed0fa - it actually completed successfully in under a second. The dashboard is showing it as "Running" due to a display issue where both queued and completed statuses can render as "Running."
Copy code
GET https://api.vapi.ai/eval/run/d2d71fdb-403b-4ad5-8718-694f356ed0fa
n
@Shaunak This is still an issue for me. Running -
Copy code
curl -s "https://api.vapi.ai/eval/run/d2d71fdb-403b-4ad5-8718-694f356ed0fa" \
    -H "Authorization: Bearer $VAPI_API_KEY" -H "accept: application/json"
Returns me -
Copy code
{"id":"d2d71fdb-403b-4ad5-8718-694f356ed0fa","orgId":"78466729-70a1-4dea-b857-0d921c435e0f","type":"eval","status":"queued", ...
Still in 
queued
state and does not have the Eval result. I am still unable to run any Evaluations because they all get stuck in 
queued
state. @Mohit Charkha can you confirm if you still having the same issues? If so, @Shaunak maybe you can try running Evals via API to see if its an API issue or its my account issue. Note: I tried running an eval today again and same issue.
@Shaunak Are you still looking into this? Evals are really important for our team, because of this issue we are unable to verify changes we make to the AI's prompt and our customers are raising issues with AI's behaviour that we cannot easily validate because we have no access to Evals.
s
Hi N3squik, Sorry for the delay.
We've confirmed your evals are actually running and completing successfully on the backend. The issue is purely with the status not updating correctly in the database, so the dashboard and API show queued instead of ended.
Your eval results are being generated, they're just not surfaced properly through the status field. You can continue running new evals as normal - they will execute and produce results even though the status appears stuck.
n
@Shaunak Thank you for the reply. I understand they are succeeding on the backend, but its impossible for us to get them. When I try to get Eval's response (i.e. 
GET /eval/run/68f9281e-3c57-4607-ba9c-03e7581550fe
), That payload has:
Copy code
"status" => "queued"
"startedAt" => ...
"endedAt" => ...
"target" => ...
But it does not include the finished eval output fields we need, especially:
Copy code
"results"
"endedReason"
"endedMessage"
"cost"
Could you please help me retrieving that finished result (example Eval 
68f9281e-3c57-4607-ba9c-03e7581550fe
).
@Shaunak is there a timeline when Evals are going to be fixed? Because it would seem that everyones evals are not working now for nearly a month.
PreviousNext
Powered by