Do you have a clear outline of a conversation flow or prompt which specifies when to use tools? Ensure your tool descriptions and parameters are clearly defined for the assistant perhaps.