The recent bankruptcy filing of Character. AI has brought to the surface a dilemma AI companies have long avoided: when models cause harm, who is responsible? Character. AI spent heavily on legal fees after being sued for wrongful death (Garcia v. Character Technologies, Inc, 2024), with allegations that its AI encouraged users to take their own lives. Cases like this are forcing a reckoning over how providers of LLMs and the products built on them answer for their output.
Reports of chatbots causing harm appear routinely, and the risks are most severe in mental health care, medicine and other high-stakes fields. A debate continues over how liability should be defined and who is really to blame for AI mistakes. As AI assistants that send emails, sign contracts and take other actions on a user's behalf become widespread, the potential for real-world harm grows further.
Under product liability, companies are generally responsible for harms caused by their goods. Claims may rest on defective design, defective manufacturing or defective instructions. Most arguments about AI focus on design, and they are typically framed as strict liability or negligence. Negligence asks whether a manufacturer tested the product insufficiently or released it despite known risks. Foreseeability and alternative design are key concepts: could the harm reasonably have been anticipated, and could the product have been built more safely while still meeting its goals?
These legal concepts connect directly to engineering decisions. Understanding how harm is discussed in court helps engineers build safer AI systems from the start.
Source: Towards Data Science · Summarized by HeadlinesBriefing