Bug Report: UI Tool updates overwrite system role ...
# supportj
Jeff
02/19/2025, 5:35 PMHello,
Just reporting a bug with the UI here that impacts the messages leveraged by tools. Here is a summary of the issue along with steps to reproduce:
1. Create a tool with custom messages. Leverage the "system" role with the "request-complete" status to instruct the LLM to generate a message based on the instructions provided. Example included below for a tool used to fetch availability (Step 1. Example).
2. (Optional) Test the tool with an assistant. The assistant will successfully generate a message based on the instructions provided in the content. Alternatively, just verify that the role is set to "system" when you fetch the tool from the API
3. Navigate to the tool's management page in the UI.
4. Update any part of the tool in the UI and save (e.g., update the description). Even if the messages section was not updated, any custom message with a "system" role will have been overwritten to an "assistant" role. This causes very awkward behavior in production until caught because the assistant recounts the "system" prompt rather than generating a message from it.
Step 1. Example:
"messages": [
{
"type": "request-start",
"content": "Let's see what openings we have in the system."
},
{
"type": "request-response-delayed",
"content": "Hmmm... I'm still looking around for something."
},
{
"role": "system",
"type": "request-complete",
"content": "You've fetched appointment availability. First ask if the date and time from option 1 works. Example: 'It looks like we have an opening on at . Does that work for you? '"
},
{
"type": "request-failed",
"content": "I'm really sorry but I'm having trouble accessing our system."
}
],
s
Sahil
02/20/2025, 6:51 PMJust wanted to confirm, you created a tool with "system" role messages using the API directly (as shown in their JSON example)
When later accessed and updated the tool through the UI, even without changing the messages section, the "system" role was overwritten to "assistant"?
j
Jeff
02/22/2025, 7:42 PMThat is correct @Shubham Bajaj
Jeff
02/22/2025, 7:45 PMI've (accidentally) replicated this accross multipe tools now where I:
1. Create a tool with a system message via API
2. Update something unrelated in the tool in the UI (e.g., update the Server URL)
3. That updates the message type from system to assistant
4. Since the message type was unintentionally updated to assistant, the assistant regurgitates the entire system prompt when the tool succeeds in production, rather than generating a message based off the prompt.
This has caused some awkward convos with customers
s
Sahil
02/24/2025, 10:27 AMHey @Jeffrey, thanks for reporting the issue I have created an ticket for it. I will let you know once it's resolved.
j
Jeff
02/24/2025, 6:44 PMThanks!