The opportunity
Cohere is the leading security-first enterprise AI company. We build cutting-edge foundation AI models and end-to-end products that are designed to solve real-world business problems.
What you'll do
Design and build scalable inference pipelines that run on large GPU clusters.
Conduct data ablations to assess data quality and experiment with data mixtures to enhance model performance.
Research and implement innovative synthetic data curation methods, leveraging: Cohere’s infrastructure to drive advancements in natural language processing.
Collaborate with cross-functional teams, including researchers and engineers,: to ensure data pipelines meet the demands of cutting-edge language models.
Strong software engineering skills, with proficiency in Python and experience building data pipelines.
Familiarity with data processing frameworks such as Apache Spark, Apache Beam, Pandas, or similar tools.
What they're looking for
- Experience working with LLMs through work projects, open-source contributions or personal experimentation.
- Familiarity with LLM inference frameworks such as vLLM and TensorRT.
- Experience working with large-scale datasets, including web data, code data, and multilingual corpora.
- A passion for bridging research and engineering to solve complex data-related challenges in AI model training.
