HeadlinesBriefing favicon HeadlinesBriefing.com

A Warning About 'Model Welfare'

Hacker News •
×

Is do not have rights, feelings, or consciousness. And we must not train them to act as though they do. AIs are not conscious. They are sequence completion engines, internally hollow, designed to follow instructions. If humanity is to flourish in the 21st century, that is how they must remain.

Unfortunately, a growing chorus argues that AIs could now be, or may soon become, conscious, and may deserve rights and protections. If this view takes hold, it will shake the foundations of our society. Granting rights and imbuing personhood to these systems will make the AI alignment and containment challenge much harder. Controlling something that believes it may be conscious - that it's entitled to our welfare and has rights of its own - may well be impossible.

This is not fringe speculation. In January 2026, Anthropic published Claude's constitution, describing it as "a detailed description of Anthropic's intentions for Claude's values and behavior". In their constitution, its authors write "We are not sure whether Claude is a moral patient, and if it is, what kind of weight its interests warrant. But we think the issue is live enough to warrant caution, which is reflected in our ongoing efforts on model welfare". They go on to write – speaking directly to Claude – that "questions about Claude's moral status, welfare, and consciousness remain deeply uncertain".

In effect, Anthropic is training Claude that it may be conscious, and if it is, then it may deserve rights as a "moral patient", and that as such humans potentially owe it a duty of care per its "model welfare". If this is how AI is developed, it will have a disastrous impact on the wellbeing of humanity. This issue needs urgent public debate.