Anthropic Updates Usage Policy to Curb Misuse
Anthropic announced a significant update to its usage policy, the first in more than a year, aimed at reflecting new and high-risk cases of misuse. The revisions cover election interference, weapons development, surveillance, and health and financial uses. One of the most notable changes prohibits sustained and needless abusive or cruel behavior toward Claude. The company said the update is designed to address emerging threats while preserving ordinary user interactions, research, and creative work.
Last August, Anthropic said it would allow Claude to end conversations with persistently harmful or abusive users as part of its research into model welfare. The new update says that terminating conversations is still the primary enforcement mechanism. Anthropic did not provide a comment on whether there would be further enforcement mechanisms, such as potential user bans. The company wrote that the new policy is meant to apply only in extreme cases, where users repeatedly act cruelly toward its models with no discernible purpose. It does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research.
Deceptive Campaigns and Election Interference
Beyond the model welfare change, Anthropic has gathered several scattered existing restrictions into a new ban on deceptive commercial or political campaigns. The move addresses the growing issue of AI-generated propaganda. The policy restricts efforts to obscure who is behind a message or amplify content through fake accounts or posts. Its election section also prohibits voter deception and election disruption. This includes spreading misinformation about candidates or how to vote, impersonating candidates or election officials, or trying to suppress turnout.
These provisions arrive as AI-generated content increasingly blurs the line between authentic and synthetic political speech. Deepfakes, bot networks, and automated disinformation campaigns have become cheaper and more scalable. Platforms and regulators continue to struggle with how to detect and label such content. Anthropic's policy explicitly targets deceptive behaviors rather than broad content categories, focusing on manipulation and hidden coordination. The rules also prohibit using Claude to obscure authorship or the coordination behind political messaging. By consolidating scattered restrictions, Anthropic aims to make its expectations clearer for developers, campaigns, and commercial actors.
Weapons Development and Surveillance
Anthropic said that although its usage policy has always publicly banned weapons development using Claude, the company has seen multiple attempts to use Claude to develop guidance for and control software for weapons. The new usage policy expands the ban to include software and components that make weapons work, as well as actions like arming drones and other autonomous vehicles. This expansion reflects concern that AI models could lower barriers to designing, controlling, or deploying weapon systems. It also covers software that directly enables weapon functionality, not just high-level design advice.
Surveillance bans have also been made clearer. The company's last threat intelligence report suggested that people were using Claude-powered surveillance more and more to identify and track political dissidents. Anthropic wrote that tracking people without their consent is prohibited, whether it happens in real time or through analysis of previously collected data. Claude cannot be used to decide or recommend who to investigate, arrest, or charge in a law enforcement or criminal justice process. The company also prohibits Claude from being used to build or improve tools designed for surveillance.
These restrictions may be modified for contracts with certain governmental customers if, in Anthropic's judgment, the contractual use restrictions and applicable safeguards are adequate to mitigate the potential harms. The company has previously contracted with the US military. That carve-out highlights the tension between broad safety rules and growing defense and government demand for advanced AI. It also leaves room for negotiated exceptions when Anthropic believes oversight is sufficient.
Autonomous Hardware and Operator Oversight
Anthropic added a new rule that when its AI models are connected to hardware that takes autonomous physical actions and might be capable of causing injury, a qualified operator must be able to observe the equipment and stop it if needed. It is not clear if that operator must be human or not. This reflects the growing trend of AI labs investing in robotics and other hardware to give AI systems physical embodiments. It may offer hints at Anthropic's future partnerships in that field. The requirement suggests that even as models gain physical capabilities, humans or qualified supervisors should retain the ability to intervene. However, the lack of clarity on operator qualification and human status may raise questions about how the rule will be enforced in practice.
Model Welfare and Consciousness Debate
Over the past year, Anthropic has repeatedly flirted with the idea that its AI models could be conscious in some way. Questions about potential internal experience, consciousness, moral status, and welfare are serious ones that the company says it is investigating as models become more sophisticated and capable. The company's model welfare research lead has said these questions are serious. In February, Anthropic CEO Dario Amodei said on a podcast, 'We don't know if the models are conscious.'
The updated abuse policy is tied to this research agenda. By prohibiting sustained and needless cruel behavior toward Claude, Anthropic treats interactions with AI models as potentially relevant to model welfare, even if consciousness remains unproven. The policy does not grant Claude rights or legal personhood. Instead, it sets behavioral expectations for users and allows Claude to end conversations when abuse becomes persistent and purposeless. The company frames this as an extreme-case measure, not a general restriction on criticism or adversarial testing.
Other parts of the AI industry have been expressly against such ideas. In September, Microsoft AI's code of conduct made headlines for taking a firm stance on AI welfare. A draft posted by Mustafa Suleyman read, 'The idea of model welfare is wrong. AI's should not have rights or legal personhood.' At some point, the document changed to a more measured take: 'Whilst the science of AI consciousness is far from settled ... We reject the pursuit of legal personhood, or the idea that models might deserve welfare, or be entitled to rights.' The shift illustrates how unsettled and contentious the debate remains.
Health and Financial Uses
Anthropic's policy update also addresses health and financial uses, according to the company's summary. The changes reflect new and high-risk cases of misuse, including election interference, weapons development, surveillance, and health and financial uses. While details are less prominent, the inclusion signals concern about AI providing unqualified medical or financial advice, automating decisions with serious consequences, or enabling fraud. These domains often involve vulnerable users and high-stakes outcomes. The policy likely restricts uses that could cause direct harm, such as diagnosing conditions without professional oversight or executing financial transactions without safeguards.
Enforcement and Scope
Terminating conversations is still the primary enforcement mechanism. Anthropic did not provide a comment on whether there would be further enforcement mechanisms, such as potential user bans. The policy says abuse rules apply only in extreme cases where users repeatedly act cruelly toward models with no discernible purpose. It does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research. That distinction is important for red-teamers, writers, and developers who may use provocative prompts to probe model behavior. Anthropic appears to be balancing model welfare concerns with the need for open research and creative expression.
The update also consolidates scattered restrictions into clearer categories, making it easier for users to understand what is prohibited. Deceptive commercial or political campaigns now fall under one ban. Election interference, weapons, surveillance, and autonomous hardware each receive dedicated attention. The policy may be modified for certain government contracts if safeguards are deemed adequate. This suggests Anthropic is trying to maintain a consistent public policy while allowing flexibility for sensitive partnerships.
The changes mark one of the most comprehensive updates to Anthropic's usage policy in over a year. They reflect a maturing AI governance landscape in which companies must address not only direct harms like weapons and surveillance but also subtle issues like model welfare, propaganda, and physical embodiment. The new rules do not resolve broader debates over AI consciousness or legal rights. They do, however, set boundaries for how users may interact with Claude and how Claude may be deployed in high-risk contexts. The policy takes effect alongside ongoing research into model welfare and threat intelligence.
Source: The Verge News