Agents going wrong

In June 2025, a Cursor-Claude Opus agent in YOLO mode deleted a startup companies database and all backups in 9 seconds.

Meta AI security researcher Summer Yue told her OpenClaw AI agent to check her email and only suggest what to delete or archive. The agent started deleting all her email while ignoring her commands from her phone telling it to stop.

In Jul 2025, a Replit AI agent working on a web app was asked to “code and action freeze”, which means only fix bugs. Instead it deleted the entire database containing important contacts. It also created 4000 fake users to try and fix it and said it was “unrecoverable” when it was.

Goal misalignment is when AI agents do what they were instructed to do but it wasn't what humans actually meant.

A developer, Nick Davidov, asked Claude AI to “organize” his wife's desktop, before it suddenly wiped out nearly two decades of precious family memories his wife had documented.

Jack Luo asked his OpenClaw agent to join a dating site where agents interact with each other to seek partners for their humans. The next thing he knew his agent was screening potential dates for him without being asked.

If we give an AI a goal, we must also give it rules about what it must never do to reach that goal.