Our commitments to child safety
Anthropic takes a Safety by Design approach to child safety. We are signatories to Thorn and All Tech Is Human’s Safety by Design principles for generative AI and report on our progress in the Transparency Hub. In practice, this means that we apply safety training designed to make our models refuse requests that sexualize or endanger minors, operate detection and monitoring systems across our services, and report apparent CSAM and other forms of child exploitation to the National Center for Missing & Exploited Children (NCMEC). Our first-party consumer services are restricted to users 18 years and older, and Claude does not produce photorealistic image or video output.
These protections apply across our platform. Because you control your product's design, user base, and content flows, you are best positioned to address risks specific to your deployment.
Your obligations under Anthropic's Usage Policy
All developers building on Anthropic's API are required to comply with our Usage Policy, which prohibits using our products or services to compromise children's safety. This includes, among other things, creating, distributing, or promoting CSAM, including AI-generated CSAM; facilitating grooming, trafficking, sextortion, or other exploitation of a minor; and sexualizing minors in any context, including fiction or roleplay.
These obligations apply regardless of your platform's size or use case. While you may already prohibit this activity, your end users or customers may upload CSAM, elicit sexualized content involving minors, or use outputs to facilitate grooming or sextortion. You are responsible for implementing appropriate safeguards against these risks on your deployment. We monitor API usage for violations of the Usage Policy and take enforcement action consistent with our policies and legal obligations.
Additionally, if your product allows minors to interact directly with our models, please refer to our Guidelines for Organizations Serving Minors, which sets out the additional safeguards Anthropic requires for that audience. This guidance covers a range of possible measures, including but not limited to age verification systems, content moderation and filtering, monitoring and reporting mechanisms, and regulatory compliance and disclosure requirements.
Whether you've received a notification from Anthropic about prohibited content or are proactively building out your safeguards, the resources below can help you get oriented in standing up detection, reporting, and response infrastructure appropriate to your platform.
Building a child safety program
There is no single correct architecture for a child safety program. The viable approach depends on your platform; its size; the surfaces where users can engage, submit, or generate content; and your jurisdiction. Many platforms have already built prevention and response capabilities suited to their own contexts, and the field has developed shared guidance, tooling, and reporting infrastructure that newer programs can draw on. The organizations below offer resources that many platforms use as a starting point. Anthropic does not require you to use any specific vendor or program.
Design principles. Thorn and All Tech Is Human have published Safety by Design for Generative AI, a set of principles and practical mitigations for preventing child sexual abuse across the develop, deploy, and maintain phases of an AI product. Anthropic is a signatory to these principles, and we encourage developers building on our models to consider them when designing their own safeguards.
Detection & blocking services. If your deployment allows users to upload or generate images or videos, or interact with other users, you can implement detection on your own infrastructure. Organizations may choose to build custom classifiers tailored to their threat model, though for many, a vendor or partner solution may be the more practical path. There are several providers that offer hash-matching, image, video, or text classifier solutions. For example, one widely deployed option is Thorn's Safer, which provides hash-matching against known CSAM and classifiers for novel material and grooming signals. The Internet Watch Foundation (IWF) offers its member companies a suite of services for detecting, blocking, and responding to CSAM on their platforms, including hash lists and a URL blocklist. The Tech Coalition's Pathways program (described below) can help companies evaluate and access detection options suited to their platform.
Program development and operational guidance. The Tech Coalition is an industry alliance focused on preventing online child sexual exploitation and abuse. Its free Pathways program is designed for startups and small to mid-sized platforms and provides templates for building a child safety program, guidance on CSAM detection and NCMEC reporting, insight into global regulatory requirements, and support on emerging risks such as financial sextortion and AI-generated harms. Anthropic is a Tech Coalition member. You can contact the Tech Coalition team for an initial consultation.
Reporting. US-based platforms with actual knowledge of apparent CSAM are legally required to report to NCMEC's CyberTipline. Platforms can register with NCMEC to gain access to the CyberTipline reporting system, and NCMEC publishes additional legal resources on reporting obligations. Platforms operating in other jurisdictions should consult local reporting requirements; INHOPE maintains a directory of national hotlines. As you build your program, consider incorporating logging for the purposes of transparency reporting.
This page is provided for informational purposes only and does not constitute legal advice or endorsement by Anthropic. Consult your own legal counsel regarding your specific obligations.