Software Engineer, Systems ML - PyTorch Compiler, PyTorch Framework, PyTorch Performance

Meta Platforms • Bellevue, WA

133d

About The Position

This role is about developing the core PyTorch 2.0 technologies, innovating and advancing the state-of-the-art of ML compilers, and accelerating PT2 adoption through direct engagements with OSS and industry users. The PyTorch Compiler team is dedicated to making PyTorch run faster and more resource-efficient without sacrificing its flexibility and ease of use. The team is the driving force behind PT2, a step function change in PyTorch's history that brought compiler technologies to the core of PyTorch. PT2 technologies have gained industry-wide recognition since their first release in March 2023. The team is committed to building the PT2 compiler that withstands the test of time while striving to become the #1 ML framework compiler in the industry. The team is highly innovative, passionate about the technologies we build, and love to do technical work. Our work is open source, cutting-edge, and industry leading.

Requirements

• Advanced in Python or C++ programming
• Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience
• 4+ years of research or industry experience in developing compilers, ML systems, ML accelerators, GPU performance, and similar
• Experience in developing PyTorch/PT2, Triton, MLIR, JAX, XLA, TVM is a huge plus
• Knowledge in GPU architecture, ML accelerator performance, and developing high-performance kernels
• Experience in building OSS communities and extensive social media presence in the ML Sys domain
• Experience with training models, end-to-end model optimizations, or applying ML to systems
• Knowledge of communication collectives, PyTorch distributed, and parallelism
• Experience in developing inside other ML frameworks like Caffe2, TensorFlow, ONNX, TensorRT

Responsibilities

• Develop the PT2 compiler (e.g., TorchDynamo, TorchInductor, Export, PyTorch Core)
• Improve PyTorch performance via systematic solutions for the entire community
• Explore the intersection of the PyTorch compiler and PyTorch distributed
• Optimize Generative AI models across the stack (pre-training, fine-tuning, and inference)
• Collaborate with users of PyTorch to enable new use cases of PT2 technologies both inside and outside Meta

What This Job Offers

• Industry: Broadcasting and Content Providers
• Education Level: Bachelor's degree
• Number of Employees: 5,001-10,000 employees