when testing an agent it switches between voices sometimes I get a male voice when the agent is setup with a female voice.. sometimes it switches between different female voices...

Make sure the correct voice is selected, check for fallback voice settings, review system prompts, test with different voices, and monitor call logs for any anomalies.