The opportunity
The Workload Networking team is responsible for the collective communication stack used in our largest training jobs. Using a combination of C++ and CUDA we work on novel collective communication techniques that enable efficient training of our flagship models on our largest custom built supercomputers.
What you'll do
Collaborate closely with ML researchers to design and implement efficient: collective operations in C++ and CUDA.
Ensure that our largest training jobs take full advantage of the different: network transports used in our supercomputers.
Work on simulations to inform our future supercomputer network designs.
Have written distributed algorithms using RDMA in the past.
Are comfortable writing low level performance sensitive CPU and/or GPU code.
Are familiar with network simulation techniques.
