The opportunity
Our mission at Duolingo is to develop the best education in the world and make it universally available. It’s a big mission, and that’s where you come in!
What you'll do
Own the DET proctoring quality standard: maintain rubrics, decision rules, examples, and guidance that define accurate decisions across routine and difficult edge cases.
Lead calibration and adjudication: operate recurring Proctor Certification Test cycles, resolve disputed decisions, identify patterns of disagreement, and ensure outcomes are consistent and auditable.
Build and run an ongoing learning program: publish a forward-looking training calendar, deliver monthly interventions, and launch targeted training in response to quality signals, threats, or workflow changes.
Measure effectiveness: use calibration results, audits, and performance trends to identify learning needs, evaluate training impact, and decide when to clarify standards, retrain, or update guidance.
Guide specialist execution: set expectations for Training Specialists and adjudicators, assign deliverables, review outputs, and operate onboarding and certification that ramp new specialists quickly.
Maintain durable sources of truth: keep calibration, adjudication, and training runbooks current so critical processes are documented and auditable.
What they're looking for
- Evolve quality alongside proctoring: translate new threats and workflow changes into clear standards, training, and quality controls in partnership with Psychometrics, Security, Operations, Product, and Engineering.
- Represent the proctoring craft: bring frontline proctor evidence into product experiments and workflow decisions; assess whether launched changes preserve decision quality.
- Connect human and technology-assisted quality: establish benchmarks for AI-assisted workflows and validate automated capabilities against real sessions before deployment.
- + years in assessment development, scoring/rater quality, certification,: quality assurance, or a field involving complex, consequential judgments; experience aligning reviewers to a shared standard through calibration programs.
