Search Anthropic

Anthropic's approach to training AI systems using a set of principles rather than relying solely on human feedback. The model learns to critique and revise its own outputs based on these principles.
Widening the conversation on frontier AI
Over the past several months, we’ve been organizing dialogues with groups whose work and traditions bear on the questions raised by AI.
New research on how we've reduced agentic misalignment.
Claude's new constitution
A new approach to a foundational document that expresses and shapes who Claude is
Claude's Constitution
A detailed description of our vision for Claude’s values and behavior.
Next-generation Constitutional Classifiers: More efficient protection against universal jailbreaks
Constitutional Classifiers++ improve on our previous approach, yielding a system that is even more robust, has a much lower refusal rate, and is dramatically cheaper to run.
Strengthening our safeguards through collaboration with US CAISI and UK AISI
Over the past year, we've collaborated with the US Center for AI Standards and Innovation (CAISI) and UK AI Security Institute (AISI), government bodies established to measure and improve the security of AI systems.
Testing our safety defenses with a new bug bounty program
A new bug bounty program with HackerOne challenges red-teamers to find universal jailbreaks in safety classifiers we haven't yet deployed.
Constitutional Classifiers: Defending against universal jailbreaks
A paper from Anthropic describing a new way to guard LLMs against jailbreaking.
Specific versus general principles for Constitutional AI
Models trained with Constitutional AI can generalize from a single broad principle to produce harmless assistants.
Collective Constitutional AI: Aligning a language model with public input
Anthropic and the Collective Intelligence Project recently ran a public input process involving ~1,000 Americans to draft a constitution for an AI system. In this post, we share the resulting publicly sourced constitution, as well as what happened when we trained a new AI system against it using Constitutional AI.
Anthropic partners with BCG
Anthropic and Boston Consulting Group partner to bring Claude to BCG's enterprise clients for knowledge management and data analysis.
Towards measuring the representation of subjective global opinions in language models
In this paper, we develop a quantitative framework to evaluate whose opinions model-generated responses are more similar to.
Claude’s constitution
A detailed description of our vision for Claude’s values and behavior.
Constitutional AI: Harmlessness from AI feedback
Constitutional AI trains a harmless but non-evasive assistant through self-critique and RL from AI feedback, with far fewer human labels.
Anthropic is a Public Benefit Corporation. Our mission is the responsible development and maintenance of advanced AI for the long-term benefit of humanity.