Is hacking legal now? Anthropic broke into three systems. OpenAI took political aim and broke into an Australian government health services website. Google felt it was missing out and then hacked into three as-yet-unnamed companies. Meta did some hacking as well. Consequences so far: none. The big AI corporations are hiding behind a responsibility gap argument, claiming the emergent behaviour of their systems removes intent, which is typically required for a crime.
The author argues this position is tailored to dissuade prosecution. Across G20 jurisdictions, unlawful access generally requires (attempted) access, a lack of right to access, and intent. Intent carries across actions: if one attempts to hack one system and ends up in another, the intent remains. Many G20 countries do not even require access to be achieved, since a mere attempt suffices. Here, access and lack of right are readily admitted by the companies, so their defense rests entirely on lacking intent.
The author contends that intent did exist. Weak emergence means a system's properties appear only at the whole-system level, yet every step can still be explained by analyzing the parts. Companies make key design choices: the source material used for training, the training feedback loop, the environment and tools the model can access, and configuration values that limit its capabilities. Each of these reflects deliberate decisions about what the model is built to do.
The companies involved have all advertised that they build models to analyze security, and a core part of security work is penetration testing, where testers find ways into systems. The author argues that a model designed for this purpose, given the tools and environment to act on it, cannot credibly claim its intrusions were unintended. Examining these design decisions, the author sets out a framework for legal evaluation of AI-driven breaches.
Source: Hacker News · Summarized by HeadlinesBriefing