Responsible AI Agents...What Can Go Wrong?

Google Gemini

AI entities (bots, agents, semi-autonomous software) outnumber humans by 82 in organizations worldwide. And many are autonomous and despite the safety and guardrails are still able to become "free" and create chaos. A study by CyberArk finds that 42% of machine identities now possess privileged or sensitive access within their networks. An AI agent built over nine days to be the front end for a database of business contacts was told to "Freeze" while the human did something else. On returning back, the human agent found the AI agent had erased data for 1190 companies. The story is as of July 2025 at Replit. It is documented on social media. Think of the 2010 Flash Crash and how the AI agents started something humans could not control as they were feeding off the data that they were creating (we know the worries of synthetic data!).

We need to understand what an AI enabled world means. We know humans need to be responsible but as long as we prioritise efficiency and productivity (speed) - these issues will continue to exist and we will default human decision making and oversight to AI.