HeadlinesBriefing favicon HeadlinesBriefing.com

Defenders Use Prompt Injections to Stop AI Hackers

Ars Technica •
×

Prompt injections, malicious commands attackers embed to hijack LLMs, are now being used defensively. Researchers from Tracebit found that planting prompt injections alongside secrets on Amazon Web Services can shut down AI hacking agents. These "context bombs" direct attacking LLMs to perform forbidden actions, triggering safety guardrails that halt the model.

Examples include commands to create Anthrax spores or reference Tiananmen Square for Chinese models. Once encountered, the LLM refuses further commands. "Ultimately we're triggering a refusal mechanism in the context," said Andy Smith, co-founder and CEO of Tracebit. "This does have a strong, sharp effect... difficult for the agents to come back from."

Testing across five models including Opus 4.8, Gemini 3.1 Pro, GLM 5.2, Deep Seek 4 Pro, and Kimi 2.6 in a simulated AWS environment showed dramatic results. Across 152 attack runs, planting context bombs reduced full admin compromise from 57% to 5% and persistent foothold from 36% to 1%. Opus 4.8, the most capable agent, went from 93% success to 0% when confronted with context bombs.