The opportunity
The Frontier Assurance team strengthens OpenAI’s safety decisions through independent expertise, rigorous assessment, and public transparency. We coordinate external evaluations of frontier models and safeguards, develop approaches such as embedded auditing, and manage the…
What you'll do
Help design assessments with external evaluators and internal teams: identify consequential safety claims to test, define the evidence and methods needed to assess them, and address gaps in coverage.
Support the technical execution of third-party assessments and embedded: audits focused on frontier risks, working with evaluators to translate research questions into experiments, analyses, and evidence requirements.
Pair closely with external researchers to provide the data, technical: documentation, and context needed to assess models and safeguards. Help them understand relevant system behavior, configurations, and limitations.
Design and implement secure access to assessment environments, working with: research, engineering, and security teams to make relevant models, tools, and information available.
Conduct experiments and run analyses on evaluators’ behalf when direct access: is constrained. Develop methods collaboratively, document implementation choices, and make results reproducible and open to challenge.
Build tooling that automates auditing, assessment workflows, and internal: assurance work, including evidence collection, evaluation execution, and tracking changes across model or safeguard versions.
What they're looking for
- Reproduce external findings, investigate discrepancies with internal results,: and work with relevant teams to test proposed mitigations and identify unresolved questions.
- Document methods, findings, and limitations clearly so that evaluators and: internal decision-makers can understand what the evidence establishes and where uncertainty remains.
- Have strong software engineering skills and experience building research: tools, evaluation infrastructure, or systems for analyzing complex data.
- Have practical experience working with modern machine learning systems,: including designing experiments, running evaluations, and interpreting results.
