Protection Scientist Engineer

Our client’s team is responsible for identifying and investigating misuses of their products – especially new types of abuse–and creating systems of defence to address them. Part of this team specialises in the latter, and they need additional support to help them monitor abuse on product launches and design and build scaled detection, review, enforcement, and measurement for uncovered risks. This position is a Full-Time (40 hours per week), contract position.

About The Role:

Protection Science Engineering is an interdisciplinary role mixing data science, machine learning, investigation, and policy/protocol development. As a Protection Scientist Engineer within the client team, you will assist in designing and building systems to proactively identify and enforce on abuse on their AI products. This includes ensuring they have robust abuse monitoring in place for new products, sustaining monitoring for existing products, and prototyping and incubating systems of defense against the highest risk harms. You will also respond to and investigate critical escalations, especially those that are not caught by existing safety systems. This will require developing understanding of the products and data, and working cross-functionally with product, policy, and engineering teams.

You will need a strong ability to use SQL and python to query, transform, and understand data, and to build and improve prototype detection. An investigative mindset is key, with experience in identifying and enforcing on bad actors (in any industry). A background including data science, machine learning and classification basics, AI, and/or threat investigation is a plus.

This is a remote position, based in London, UK. The position includes participation in an on-call rotation that will involve resolving urgent escalations outside of normal work hours, for at least 7 consecutive days per quarter. Some detection and investigations may involve exposure to sensitive and distressing content, including sexual, violent, or otherwise-disturbing material.

In This Role, You Will:

• Scope and implement abuse monitoring requirements for new product launches.
• Improve processes to sustain monitoring operations for existing products, including developing approaches to automate monitoring subtasks.
• Prototype and mature into production systems of detection, review, and enforcement of abuse for major harms.
• Work with Product, Policy, Ops, and Investigative teams to understand key risks and how to identify and address them, and with Engineering teams to ensure we have sufficient data and scaled tooling.

Requirements:

• Ability to work remotely on GMT and must be geographically located in the UK.
• Quantitative and coding background , including statistics/metrics and proficiency in python and SQL.
• Experience in identifying (and ideally enforcing on) bad actors with scaled tooling.
• Ability to be on-call approximately once a quarter that will involve resolving urgent escalations outside of normal work hours, including occasional evenings and weekends
• Ability to rapidly context switch across domains, modalities, and abuse areas, to include high severity areas like violent activities and child safety
• Excited to work in a fast paced, ambiguous, and purposeful space with high impact across users and beyond, and to learn quickly.

Desired Qualities (but not required):

• Background in machine learning and classification, especially on novel or poorly understood behaviours.
• Experience scaling and automating processes, especially with language models.
• Experience working with scaled labellers or reviewers, particularly at scale.
• Familiarity with one of the following topics/areas: AI safety, child safety, abuse by nation state and other malicious actors, digital mental health, fraud abuse, radicalization/persuasion/grooming; hateful activities and groups.