Ethical Red Teamer - NLP

Key Skills

Algorithms, Data science techniques, Design, Effective communication skills, Employee life cycle, Machine learning techniques, Natural Language Processing (NLP), Operations, Planning and organization, Python Programming

Job Description

We’re Salesforce, the Customer Company, inspiring the future of business with AI+ Data +CRM. Leading with our core values, we help companies across every industry blaze new trails and connect with customers in a whole new way. And, we empower you to be a Trailblazer, too — driving your performance and career growth, charting new paths, and improving the state of the world. If you believe in business as the greatest platform for change and in companies doing well and doing good – you’ve come to the right place.

Salesforce’s Office of Ethical and Humane Use is seeking a Technical AI Ethicist with a focus on Machine Learning and Natural Language Processing to play a pivotal role guiding the responsible development of cutting edge artificial intelligence products. In this role, you will help propose, plan, and execute AI testing operations, while working cross-functionally to create and evangelize standard methodologies to educate people on the risks of AI with the goal of making our products the most trusted in the market. And you will bring technical depth to the assessment of AI products, models and applications, in order to identify the best technical mitigations to risk.

The ideal candidate will have experience in artificial intelligence and specifically the field of responsible / ethical AI coupled with a technical and quantitative skills in Machine Learning and Natural Language Processing. 

Responsibilities:

Play an authoritative internal role in AI and ethics. Lead and educate teams on risk mitigation strategies early in the AI and product development process to position customers to succeed in a rapidly shifting AI landscape.

Partner with Einstein and Cloud application product teams to identify potential ethical issues and create solutions in all phases of our lifecycle from feature ideation to deployment. Guide decisions that require assessment and evaluation of bias, harm, fairness, and AI explainability.

Propose, plan, and implement AI testing operations and repeatable patterns for product/app reviews. Identify and champion standard processes at scale.

Help lead testing activities and outcomes; assist in automating testing techniques and creating custom tooling for specific operations.

Report results and align partners on solutions to address the vulnerabilities identified. Institutionalize and embed those repeatable processes to normalize ethical testing and evaluation practices. Partner with Engineering teams to evaluate thresholds for Definition of Done.

Write code to support research and evaluation, usually in Python.

Test for technical vulnerabilities, model vulnerabilities, and harm/abuse:

Technical and model vulnerabilities including but not limited to prompt attacks, training data extraction, backdooring, adversarial examples, data poisoning, and exfiltration

Harm/abuse, including but not limited to bias, toxicity and inaccuracy

Write detailed reports covering the goals and outcomes of testing operations, including substantial observations and recommendations

Collaborate with Security and Trust teams to improve detection and response capabilities, launch and monitor bug bounties, and more.

Collaborate with AI Research team on newest approaches to model safety.

Collaborate with non-technical teams to propose process and policy enhancements and additions

Stay abreast of latest research into standard methodologies and approaches to bias, fairness, and ethics in AI.

Collaborate with Policy team on ensuring models align with Acceptable Use policies

Qualifications:

Bachelor's degree (or foreign degree equivalent) in Computer Science, Engineering, Information Systems, Human-Computer Interaction, Engineering, Data Science, Social or Applied Sciences, or a related field. Advanced degree preferred.

7+ years of relevant experience in AI ethics, AI research, Security, Trust & Safety or similar roles

Experience in applying ML and NLP technologies; solid ML-focused engineering and research skills, particularly around using and training models.

Experience in analyzing complex, large-scale data sets and making decisions based on data

Shown leadership, organizational, and execution skills. Passion for developing ground breaking AI ethics technology and deploying it through a multi-stakeholder approach.

Experience working in a technical environment with a broad, cross functional team to get results, define product requirements, coordinate assets from other groups (design, legal, etc.), and guide the team through key achievements

Experience writing detailed, concrete scenarios of significant risk from AI in a way that can be tested; track record of building evaluation and proposing ideas for remediation

Excellent written and oral communication skills, as well as interpersonal skills, including the ability to articulate technical concepts to both technical and non-technical audiences.

Published research on algorithmic fairness, accountability, and clarity, especially around detecting and mitigating bias or AI safety.

Accommodations

If you require assistance due to a disability applying for open positions please submit a request via this Accommodations Request Form.

Posting Statement

At Salesforce we believe that the business of business is to improve the state of our world. Each of us has a responsibility to drive Equality in our communities and workplaces. We are committed to creating a workforce that reflects society through inclusive programs and initiatives such as equal pay, employee resource groups, inclusive benefits, and more. Learn more about Equality at www.equality.com and explore our company benefits at www.salesforcebenefits.com.

Salesforce is an Equal Employment Opportunity and Affirmative Action Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender perception or identity, national origin, age, marital status, protected veteran status, or disability status. Salesforce does not accept unsolicited headhunter and agency resumes. Salesforce will not pay any third-party agency or company that does not have a signed agreement with Salesforce.

Salesforce welcomes all.

Pursuant to the San Francisco Fair Chance Ordinance and the Los Angeles Fair Chance Initiative for Hiring, Salesforce will consider for employment qualified applicants with arrest and conviction records.

For New York-based roles, the base salary hiring range for this position is $193,200 to $305,600.

For Colorado-based roles, the base salary hiring range for this position is $175,600 to $254,700.

For Washington-based roles, the base salary hiring range for this position is $175,600 to $280,200.

For California-based roles, the base salary hiring range for this position is $193,200 to $305,600.