Responsible AI Agents...What Can Go Wrong?

Google Gemini

AI entities (bots, agents, semi-autonomous software) outnumber humans worldwide in terms of internet traffic. And many are autonomous and despite the safety and guardrails are still able to become "free" and create chaos. A study by CyberArk finds that 42% of machine identities now possess privileged or sensitive access within their networks. TheCyberArk 2025 Identity Security Landscape study, spanning 2,600 cybersecurity decision makers across 20 countries, found machine identities now outnumber human identities by a staggering 82-to-1 margin within enterprise environments. An AI agent built over nine days to be the front end for a database of business contacts was told to "Freeze" while the human did something else. On returning back, the human agent found the AI agent had erased data for 1190 companies. The story is as of July 2025 at Replit. It is documented on social media. Think of the 2010 Flash Crash and how the AI agents started something humans could not control as they were feeding off the data that they were creating (we know the worries of synthetic data!).

We need to understand what an AI enabled world means. We know humans need to be responsible but as long as we prioritise efficiency and productivity (speed) - these issues will continue to exist and we will default human decision making and oversight to AI.