The opportunity
The Personalization team makes deciding what to play next easier and more enjoyable for every listener. From Blend to Discover Weekly, we’re behind some of Spotify’s most-loved features.
What you'll do
Own the quality bar for Spotify’s AI voice models, defining what “good: enough” means for different products, audiences, languages and use cases.
Independently design evaluation rubrics and build an evaluation approach that: measures qualities such as naturalness, tone, pacing, pronunciation, accent, audio quality and overall experience.
Plan and carry out evaluation campaigns, turning qualitative and quantitative: findings into clear recommendations for Voice Production, Speak and product teams.
Establish an ongoing evaluation rhythm, including when and how we reassess: model quality as our technology and products evolve.
Benchmark Spotify’s AI voices against third-party models and competitors to: understand where our experiences lead, where they differ and where there are opportunities to improve.
Help determine appropriate quality trade-offs as Spotify expands into: additional languages and regions, considering the product use case and needs of each market.
What they're looking for
- You have experience in product quality, evaluation, data curation or a: closely related field where human judgment is used to assess model or product performance.
- You know how to independently define evaluation criteria and turn subjective: qualities into clear, repeatable rubrics.
- You are comfortable designing and running both qualitative and MOS-style: evaluations and synthesizing the results into actionable insights.
- You have an understanding of voice or audio quality, including qualities such: as naturalness, pacing, tone, accent and pronunciation, or the ability to develop that expertise quickly.
- You have strong judgment and can maintain a consistent quality bar while: balancing different product requirements, markets and stakeholder perspectives.
- You are comfortable working autonomously, identifying what needs to be: evaluated and establishing the processes and cadence to make that evaluation useful over time.
- You communicate clearly across creative, technical and product teams and can: act as a reliable point of contact for internal partners and external vendors.
- You are curious about generative AI, speech technology and the evolving AI: voice landscape and can translate changes in the industry into meaningful benchmarks for Spotify.
