The opportunity
Anthropic’s Inference organization serves Claude to millions of users and enterprise customers, across our own products and the world’s largest cloud platforms, with the speed, reliability, and efficiency that frontier AI demands. The stack that makes this possible is deep and…
What you'll do
Scaling and infrastructure. Routing, autoscaling, and capacity management for: the serving fleet, and bringing new capacity online.
Velocity. The toolchains, development environments, and CI/CD that let: inference engineers build, test, and ship across every accelerator platform.
Inference engine. The software between the accelerator kernels and the: routing layer that serves every request.
Performance. Improving throughput, latency, cost, and reliability across the: stack, with correctness treated as part of performance.
Own the programs that scale Claude’s serving fleet across accelerator and: cloud platforms, from early direction through delivery.
Drive new capacity online end to end, working across compute, infrastructure,: security, networking, and inference teams so one person is accountable for the whole path.
What they're looking for
- Direct experience with LLM or ML inference serving at scale, such as model: servers, batching and scheduling, KV caching, quantization, speculative decoding, or multi-model routing.
- Experience with accelerator infrastructure (GPU, TPU, or Trainium), including: cluster bring-up, hardware qualification, or running one workload across more than one accelerator platform.
- Experience managing capacity across multiple clouds or regions, including: demand forecasting, headroom policy, cost modeling, or unit economics for compute-heavy services.
- Experience with developer productivity, release engineering, or CI/CD at: scale, especially for hardware-dependent workloads, and with using engineering metrics to measurably raise velocity.
- Experience running performance or efficiency programs, with enough grounding: in rooflines, kernels, ML compilers (XLA, Triton, Neuron), numerics, or collective communication to reason about trade-offs with the engineers doing the work.
- Familiarity with Kubernetes-based orchestration and job scheduling.
- Experience working with research teams and turning their needs into infrastructure requirements.
- Practical experience using AI tools in your own workflow, including building: automations or agents, and clear ideas about where they help and where they don't.
