The opportunity
The Platform team creates the technology that enables Spotify to learn quickly and scale easily, enabling rapid growth in our users and our business around the globe. Spanning many disciplines, we work to make the business work; creating the infrastructure, tooling, frameworks,…
What you'll do
Define quality frameworks, evaluation rubrics, thresholds, and methodologies for multilingual AI experiences.
Design and execute structured evaluations for AI-generated, AI-translated,: AI-curated, and recommendation-driven experiences.
Lead multilingual dataset curation, annotation, enrichment, and ground-truth: creation to support AI model development and evaluation.
Analyze evaluation results, identify quality gaps, and provide actionable: recommendations to improve multilingual AI quality.
Support LLM-as-a-judge workflows, evaluator calibration, and human-AI agreement studies.
Partner closely with Product, Engineering, Data Science, Research,: Localization, vendors, and market experts to improve AI quality signals and inform launch decisions.
What they're looking for
- Document best practices and help define quality standards across languages, markets, and AI use cases.
- Contribute to building scalable evaluation capabilities that support the next: generation of AI-powered experiences across Spotify.
- You have experience in multilingual quality evaluation, localization, data: curation, annotation, AI evaluation, or related fields, including text-to-text and text-to-speech experiences.
- You understand language quality, cultural relevance, content quality, and: user experience across multiple languages and markets.
