Engineering Manager, EvalsActive

San FranciscoTechnology

The opportunity

Our mission is to automate coding. The first step in our journey is to build the best tool for professional programmers, using a combination of inventive research, design, and engineering.

What they're looking for

  • You can align research, product, data, and infrastructure on what “good”: means—and turn that into durable metrics, processes, and release/training rituals.
  • You have good taste and strong opinions on model and agent behaviors, and you: stay up-to-date on emerging research and industry trends.
  • You have strong data acumen, and can collaborate effectively with data scientists and researchers.
  • You’ve built and operated evaluation or measurement systems (e.g., AI evals,: experimentation platforms, ranking/relevance, search quality, or reliability instrumentation).