Research Scientist, Societal Impacts

Key Skills

Algorithms, AWS, Azure, GPUs, JavaScript, Large Language Models - LLMs, NoSQL, SQL, UI design, UX design

Job Description

As a societal impacts research scientist, you’ll devise new ways to measure and assess Anthropic systems for societally or policy-relevant traits and capabilities, and will discuss what we discover in our research publications and policy campaigns.

Strong candidates will have a track record of running experiments relating to machine learning systems, working in a fast-paced startup environment, and an eagerness to develop their own technical skills so they can best interface with our systems. The ideal candidate will enjoy a mixture of running experiments and doing research, developing new tools and evaluation suites, and evangelizing this and other work to key stakeholders and communities.

About Anthropic

Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems.

Your responsibilities will include:

• Designing, proposing and running experiments on our systems. These will range from developing basic probes for different capabilities in models (such as assessing a given system for fairness traits); to designing comparative studies to understand how well these systems complement humans (for example, by analyzing how interacting with an AI system alters human behavior); to developing comprehensive suites of tests we can run complicated systems through (such as developing ways to evaluate the broad capabilities displayed by modern language models).
• Partnering closely with our other researchers to fully understand, analyze and evaluate state-of-the-art research across a broad range of capabilities and safety research.
• Interfacing with, providing feedback on and publicly communicating about our internal technical infrastructure and tools, with a goal of them better supporting the societal impact analysis of AI systems.
• Developing tools, evaluation suites and tests that enable understanding of AI systems for policymakers, academia, civil society and more.
• Working with the policy and other research teams to share these tools to the above stakeholders and incorporating their feedback and requests.
• Sharing your insights from your work by contributing to Anthropic research publications, giving presentations to external groups, and partnering with our policy team to articulate insights to governments.
• Generating net-new insights about the potential societal impact of systems being developed by Anthropic, and using this understanding to inform Anthropic strategy, research and policy campaigns.

You might be a good fit, if:

• You have experience assessing machine learning models for unknown traits within known capabilities (for instance, assessing the outputs of a generative model to discover what parts of a dataset are being magnified and minimized).
• You have a track record of using technical infrastructure to interface effectively with machine learning models
• You enjoy and are skilled at writing up and communicating your results, even when they’re null.
• You’re comfortable creating your own research agenda and executing against it.
• You find it exciting to partner with colleagues on ‘big science’ projects, where the whole company works to build one AI artefact and then analyze it.
• You have a prior background in data science, or another technical field which involves interfacing with technical artefacts and generating insights about them.
• You’re passionate about communicating the insights from your research to external stakeholders, ranging from academics to policymakers to other research labs.
• You have an interest in AI policy; this role offers the chance to partner with Anthropic policy to turn your research insights into actionable recommendations for governments.

Some examples of our work:

• Predictability and Surprise in Large Generative Models
• Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
• The Capacity for Moral Self-Correction in Large Language Models
• Opportunities and Risks of LLMs for Scalable Deliberation with Polis
• Towards Measuring the Representation of Subjective Global Opinions in Language Models
• Challenges in evaluating AI systems
• Collective Constitutional AI: Aligning a Language Model with Public Input
• Unpredictable Abilities Emerging from Large AI Models [MIT Tech Review]
• Language models might be able to self-correct biases—if you ask them [Wired]
• Collective Constitutional AI [NYTimes]
• Red team data — used to make open source models, e.g., Lama-2 more harmless!
• GlobalOpinionQA - measuring whose global values LMs are aligned with.

Annual Salary (USD)

• The expected salary range for this position is $250k-375k.

For this role, we are open to a 6-month residency at Anthropic (with the opportunity to convert to full-time after).

Compensation and Benefits*

Anthropic’s compensation package consists of three elements: salary, equity, and benefits. We are committed to pay fairness and aim for these three elements collectively to be highly competitive with market rates.

Equity - Equity will be a major component of the total compensation for this position. We aim to offer higher-than-average equity compensation for a company of our size, and communicate equity amounts at the time of offer issuance.

Benefits - Benefits we offer include:

• Optional equity donation matching at a 3:1 ratio, up to 50% of your equity grant.
• Comprehensive health, dental, and vision insurance for you and all your dependents.
• 401(k) plan with 4% matching.
• 21 weeks of paid parental leave.
• Unlimited PTO – most staff take between 4-6 weeks each year, sometimes more!
• Stipends for education, home office improvements, commuting, and wellness.
• Fertility benefits via Carrot.
• Daily lunches and snacks in our office.
• Relocation support for those moving to the Bay Area.

 This compensation and benefits information is based on Anthropic’s good faith estimate for this position, in San Francisco, CA, as of the date of publication and may be modified in the future. The level of pay within the range will depend on a variety of job-related factors, including where you place on our internal performance ladders, which is based on factors including past work experience, relevant education, and performance on our interviews or in a work trial.

Deadline to apply: None. Applications will be reviewed on a rolling basis.

Hybrid policy: For this role, we prefer candidates who are able to be in our office more than 25% of the time, though we encourage you to apply even if you don’t think you will be able to do that.

We encourage you to apply even if you do not believe you meet every single qualification. Not all strong candidates will meet every single qualification as listed. Research shows that people who identify as being from underrepresented groups are more prone to experiencing imposter syndrome and doubting the strength of their candidacy, so we urge you not to exclude yourself prematurely and to submit an application if you're interested in this work. We think AI systems like the ones we're building have enormous social and ethical implications. We think this makes representation even more important, and we strive to include a range of diverse perspectives on our team.

Compensation and Benefits*

Anthropic’s compensation package consists of three elements: salary, equity, and benefits. We are committed to pay fairness and aim for these three elements collectively to be highly competitive with market rates.

Equity - On top of this position's salary (listed above), equity will be a major component of the total compensation. We aim to offer higher-than-average equity compensation for a company of our size, and communicate equity amounts at the time of offer issuance.

Benefits - Benefits we offer include:

• Optional equity donation matching at a 3:1 ratio, up to 50% of your equity grant.
• Comprehensive health, dental, and vision insurance for you and all your dependents.
• 401(k) plan with 4% matching.
• 21 weeks of paid parental leave.
• Unlimited PTO – most staff take between 4-6 weeks each year, sometimes more!
• Stipends for education, home office improvements, commuting, and wellness.
• Fertility benefits via Carrot.
• Daily lunches and snacks in our office.
• Relocation support for those moving to the Bay Area.

This compensation and benefits information is based on Anthropic’s good faith estimate for this position, in San Francisco, CA, as of the date of publication and may be modified in the future. The level of pay within the range will depend on a variety of job-related factors, including where you place on our internal performance ladders, which is based on factors including past work experience, relevant education, and performance on our interviews or in a work trial.

How we're different

We believe that the highest-impact AI research will be big science. At Anthropic we work as a single cohesive team on just a few large-scale research efforts. And we value impact — advancing our long-term goals of steerable, trustworthy AI — rather than work on smaller and more specific puzzles. We view AI research as an empirical science, which has as much in common with physics and biology as with traditional efforts in computer science. We're an extremely collaborative group, and we host frequent research discussions to ensure that we are pursuing the highest-impact work at any given time. As such, we greatly value communication skills. We're trying to build a core of knowledge and intuition about the most robustly effective innovations in AI, and so thoroughly-documented null results are almost as valuable as positive discoveries. We do not have boundaries between engineering and research, and we expect all of our technical staff to contribute to both as needed.

The easiest way to understand our research directions is to read our recent research. This research continues many of the directions our team worked on prior to Anthropic, including: GPT-3, Circuit-Based Interpretability, Multimodal Neurons, Scaling Laws, AI & Compute, Concrete Problems in AI Safety, and Learning from Human Preferences.

Come work with us!

Anthropic is a public benefit corporation based in San Francisco. We offer competitive compensation and benefits, optional equity donation matching, generous vacation and parental leave, flexible working hours, and a lovely office space in which to collaborate with colleagues.