The opportunity
Anthropic is at the forefront of AI research, dedicated to developing safe, ethical, and powerful artificial intelligence. Our mission is to ensure that transformative AI systems are aligned with human interests.
What you'll do
Design and implement data processing infrastructure for large language model: training (highly performant, reproducible, traceable)
Develop and maintain core processing primitives (e.g., tokenization,: deduplication, chunking) with a focus on scalability
Build robust systems for data quality assurance and validation at scale
Collaborate with research teams to implement novel data processing architectures
Build and operate end-to-end data pipelines that turn raw web-scale corpora into training-ready datasets
