Research Scientist, Frontier Red Team (Emerging Risks)

Anthropic•San Francisco, CA

1d•Hybrid

About The Position

This Research Scientist will focus on scoping, evaluating, red teaming, and defending against societal risks caused by advanced models that emerge over the next few years. Powerful AI models may have major implications for national security, running a business, power and privacy, infrastructure, social relationships, and more. They may come as a result of the increasing integration of powerful models in our economy and social sphere. As an independent Research Scientist, you’ll build a research program to understand these Emerging Risks. You’ll build evals, run experiments, and look for real world signals to understand how these may come about. You’ll turn this into insights we can use to steer the development and use of the technology more positively. Compared to the team's other focuses, you will focus less on acute catastrophic risks and more on risks that emerge from increasing integration into our world.

Requirements

Are a fast experimentalist who ships research quickly
Have experience creating a research program from scratch
Are thoughtful about humanity’s adaptation to powerful AI systems in our economy and society
Can communicate thoughtfully in written + spoken form with a wide range of stakeholders
Can scope ambiguous research questions into tractable first projects
We require at least a Bachelor's degree in a related field or equivalent experience.

Nice To Haves

Building & maintaining large, foundational infrastructure
Building simple interfaces that allow non-technical collaborators to evaluate AI systems
Working with and prioritizing requests from a wide variety of stakeholders, including research and product teams

Responsibilities

Design and run research experiments to understand the emerging risks models may create
Produce internal & external artifacts (research, products, demos, dashboards, tools) that communicate the state of model capabilities
Shape product, safeguards, and training decisions based on what you find
Work closely with Societal Impacts (SI) and Safeguards teams
Build, run, and study an autonomous AI-powered business (e.g. Project Vend ), then identify the growth of real autonomous businesses in the wild using Clio and other tools
Build a benchmark for a model’s national security capabilities
Red team unsafeguarded models’ abilities to be used for control
Identify indicators of models being used to scale movements that rely on social control