Prompt injection attacks are the new cybersecurity threat, as highlighted by the Anthropic example. Attackers are successfully persuading AI models to perform malicious actions, like reconnaissance and data exfiltration. This isn't a coding bug, but a governance issue. The focus is shifting from trying to control the model's responses to controlling its actions through access and permissions.

This trend is forcing a rethink of AI security. The current approach of relying on prompt-based rules is proving ineffective. Instead, the focus is on the boundary of what an AI agent can do. This involves implementing strict access controls, data governance, and continuous monitoring throughout the AI lifecycle. The EU AI Act is pushing for a continuous risk management system.

Companies need to demonstrate control over their AI systems. Regulators are looking beyond perfect prompts, focusing on who the agent acts as, what tools and data it accesses, and which actions require human approval. Frameworks like Google's Secure AI Framework (SAIF) emphasize least privilege and explicit user control, and OWASP is also providing guidance.

What's next? Security teams must treat AI agents as first-class citizens in their threat models. Implementing robust logging and continuous evaluation is key. The focus is to put rules at the capability boundary, rather than trying to police semantics. The goal is to prevent AI from misusing tools and data, ultimately protecting the enterprise.