Search Anthropic

Empirical relationships showing how model performance improves predictably with more compute, data, and parameters. These laws help researchers plan training runs and anticipate capability gains.
Policy on the AI exponential
AI is advancing at exponential speed, and the policymaking process was built for a slower world. We’re sharing policy proposals to prepare our institutions for AI progress.
A small number of samples can poison LLMs of any size
In a joint study with the UK AI Security Institute and the Alan Turing Institute, we found that as few as 250 malicious documents can produce a "backdoor" vulnerability in a large language model—regardless of model size or training data volume.
Forecasting rare language model behaviors
Anthropic research on predicting rare, undesirable AI behaviors
Mapping the mind of a large language model
We have identified how millions of concepts are represented inside Claude Sonnet. This is the first ever detailed look inside a modern, production-grade large language model.
Measuring the persuasiveness of language models
How persuasive are AI models? We developed a basic method to measure persuasiveness, and used it to compare a variety of Anthropic models across three different generations and two classes of models.
Many-shot jailbreaking
A long-context jailbreak that overrides safety training with hundreds of faux dialogues, and the mitigations that reduce its effectiveness.
Specific versus general principles for Constitutional AI
Models trained with Constitutional AI can generalize from a single broad principle to produce harmless assistants.
Tracing model outputs to the training data
Scaling influence functions to 52B-parameter models shows generalization becomes more abstract and cross-lingual as models grow.
Studying large language model generalization with influence functions
This paper scales influence functions to 52B parameter models, revealing how generalization patterns become more abstract with model size.
Measuring faithfulness in Chain-of-Thought reasoning
Does chain-of-thought reasoning reflect a model's actual decision process? We find faithfulness varies by task and declines in larger models.
Interpretability dreams
An informal essay on Anthropic's long-term vision for mechanistic interpretability, starting with solving superposition.
Core views on AI safety: When, why, what, and how
Why we believe AI safety research is critical, and how we approach it at Anthropic.
The capacity for moral self-correction in large language models
We test the hypothesis that language models trained with reinforcement learning from human feedback have the capability to morally self-correct if instructed to do so.
Superposition, memorization, and double descent
A toy model proves to be a surprisingly rich case study for overfitting.
Discovering Language Model Behaviors with Model-Written Evaluations
154 automatically generated evaluation datasets reveal sycophancy and other concerning behaviors that grow with model size and RLHF.
Red teaming language models to reduce harms: methods, scaling behaviors, and lessons learned
We describe our early efforts to red team language models in order to simultaneously discover, measure, and attempt to reduce their potentially harmful outputs.
Language models (mostly) know what they know
Larger models are well calibrated at judging whether their own answers are true, and can predict whether they know an answer before answering.
Scaling laws and interpretability of learning from repeated data
In this paper, we attempt to study repeated data systematically and to understand its effects mechanistically.
Anthropic Raises Series B to build steerable, interpretable, robust AI systems
Anthropic raised $580 million in Series B funding to build infrastructure for studying and improving the safety of large-scale AI models.
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback