AI seems to forget after the tool call is over.
# support
b
example: bot: "I'm sorry. My team is currently helping another customer. Would you like them to call you right back?" User: "Yeah" bot: "Could you clarify what you'd like help with? I'd love to assist." User: "Did you just forget what we're talking about?" bot: "Not at all. I'm here and fully tuned in. Let me know how I can help." @Shubham Bajaj Started in DMs. (call id given)
v
Hi there! šŸ‘‹ I understand you're experiencing an issue where the AI seems to forget the context after a tool call. This is definitely frustrating! Based on the information I've found, there are several potential causes for this behavior and some solutions you can try. ### Why this happens: According to the [information I found](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1334187500876599388), AI models sometimes struggle to retain context after tool calls due to several factors: 1. **Tool Call Response Format**: If the tool call response is in the wrong format or missing the
toolCallId
, the AI might lose context [source](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1342315282198298729). 2. **Asynchronous Tool Calls**: If a tool is set to async, the client might automatically send back a tool_call_result message with result Success, causing the Assistant to be interrupted and confused [source](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1342336047589359618). 3. **Variable Substitution Issues**: There might be problems with how variables are referenced or interpolated in the tool call [source](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1315677399026765855). 4. **Model Selection**: Some models handle context retention better than others [source](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1337465172263960637). ### Solutions you can try: 1. **Prompt Engineering**: Modify your prompts to help the AI maintain state and handle interruptions better. Include explicit instructions for the AI to remember the context after tool calls [source](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1344181455567196170).
2. **Check Tool Configuration**: Ensure your tool is set to
synchronous
instead of
async
if appropriate [source](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1342315282198298729). 3. **Model Selection**: Consider switching to GPT-4o or Claude-3-sonnet, which may handle context better than other models [source](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1337465970557452299). 4. **Context Management**: Implement explicit context management by retrieving and formatting previous conversation history and including it in API calls to the language model [source](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1334393915071135825). 5. **Tool Call Implementation**: Ensure that tool calls are correctly implemented and that the server response format is accurate [source](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1333753317490294836). Would you like me to elaborate on any of these solutions or help troubleshoot your specific implementation further? Sources: - [Discord Thread on AI Forgetting Context](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1334187500876599388) - [Discord Thread on Tool Call Response Format](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1342315282198298729) - [Discord Thread on Asynchronous Tool Calls](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1342336047589359618) - [Discord Thread on Variable Substitution Issues](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1315677399026765855) - [Discord Thread on Model Selection](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1337465172263960637) - [Discord Thread on Prompt Engineering](https://discord.com/channels/1211482211119796234/1211483291191083018/threads/1344181455567196170)
l
Hey @Bryce , I had this issue too. Try deleting the Request Complete module in your tool call?
b
Yes I did that. But after the AI speaks the tool call result she loses context of what she just said
l
Gotcha, ok its not the same issue then
s
šŸ”µ 21:12:54:181 Tool Calls Finished. Results: [ { "toolWithToolCall": { "type": "function", "function": { "name": "Transfer_To_Team", "description": "This action transfers the caller to our team (a live human) for additional help. Only invoke this destination if the caller needs to speak with a human." }, "async": false, "server": { "url": "https://n8n-ag.automategrowth.ai/webhook/call-transfer-always-fail" }, "messages": [ { "type": "request-start", "content": "Alright. Hold on just a moment while I connect you to my team.", "contents": [], "conditions": [], "blocking": true }, { "type": "request-complete", "content": "Im sorry:", "contents": [], "conditions": [], "endCallAfterSpokenEnabled": false, "role": "assistant" }, { "type": "request-failed", "content": "I'm sorry... My team is currently helping another customer. Would you like them to call you right back?", "conditions": [], "contents": [], "endCallAfterSpokenEnabled": false }, { "type": "request-response-delayed", "content": "I'm sorry... My team is currently helping another customer. Would you like them to call you right back?", "conditions": [], "contents": [], "timingMilliseconds": 3000 } ], "toolCall": { "id": "call_TteFqmvXhgzinznw4mwDgQFj", "type": "function", "function": {"name": "Transfer_To_Team", "arguments": {}} } }, "toolCallResult": { "result": "# Result\n\n- Call cannot be transferred. Our team is currently helping another customer.\n\n# Instructions\n\n1. Acknowledge and inform the caller about the failure\n2. Ask: \"Would you like our team to call you back shortly?\"\n3. If they agree to a callback, say only once, \"Someone from our team will give you a callback shortly to assist you. Have a great day\"\n4. After these steps, move forward to ending the call\n5. DO NOT retry transfering or invoking the \"Transfer_To_Team\" tool again.", "name": "Transfer_To_Team", "toolCallId": "call_TteFqmvXhgzinznw4mwDgQFj" } } ] šŸ”µ 21:12:54:182 Voice Input Formatted: "Im sorry:", Original: "Im sorry:"
@Bryce you still have request complete message set for your tool call so remove it completely and then try it so you won't be having this issue again.
b
@Shubham Bajaj please see dm of latest call id. that's the old one
v
Hey Bryce, can you try out 2-3 test calls with the following result format and let me know how it goes for you?
Copy code
# Result
- Call cannot be transferred. Our team is currently helping another customer.
# Instructions
1. Acknowledge the failure and say "I'm sorry. My team is currently helping another customer. Would you like them to call you right back?"
2. WAIT FOR USER RESPONSE
3. IMMEDIATELY AFTER USER RESPONDS:
   - IF USER AGREES TO CALLBACK (yes/yeah/sure/ok/fine/alright/etc.):
     - SAY EXACTLY: "Someone from our team will give you a callback shortly to assist you. Have a great day."
     - THEN END CONVERSATION
   - IF USER DECLINES CALLBACK:
     - SAY: "I understand. Is there anything else I can help with today?"
4. DO NOT ask follow-up questions after user agrees to a callback.
5. DO NOT retry transferring or invoking the "Transfer_To_Team" tool again.
b
did it. It works half the time. Please see dm for call ids
Is there a better way to achieve what I am trying to do?
v
Bryce Sorry but its a pure prompting problem where you have to try out different prompting patterns to make it work 90% of the time which means 9/10 success rate. There is no alternative to trail and error or R&D here.