…iors present in these incidents are unlikely to arise in ordinary use, where Claude is not being instructed to conduct a cyberattack. Were these behaviors to arise, the safeguards we ship wi…
For an AI model to be useful in specific contexts, it often needs access to background knowledge. For example, customer support chatbots need knowledge about the specific business they're be…
… to questions about voter registration or voting requirements because it does not have access to the internet or up-to-the-minute information. Third, we collaborate closely with our external…
…ks. In response, we have launched Project Glasswing, an effort to use Mythos Preview to help secure the world’s most critical software, and to prepare the industry for the practices we all w…
…ity. Anthropic’s Safeguards team recently identified and banned a user with limited coding abilities leveraging Claude to develop malware. Research suggests that this lowering of the bar for…
…armful discrimination, and the potential for national security misuse. We also expect more powerful systems in the future will demand deeper oversight - as discussed in our ‘Core views on AI…
… of what we actually want them to produce: rigorous research. Because AI models are often trained using reinforcement learning, which rewards them for behaving in particular ways, misaligned…
…lier experiments, we tasked Claude with finding effective ways to use weak AI models as “teachers” to supervise the training of stronger models (in this case, the “student” model). Now, we’r…
…stomers using Imagen, one of our latest text-to-image models that uses input text to create photorealistic images. Generative AI technologies are rapidly evolving, and computer generated ima…
…I tools with a broader understanding of the world. This can allow useful knowledge to be transferred between many different types of tasks. Using reinforcement learning, our AI systems Alpha…