Software Engineer-AI/ML, AWS Neuron Inference

About the position

AWS Neuron is the complete software stack for the AWS Inferentia and Trainium cloud-scale machine learning accelerators. This role is for a senior software engineer in the Machine Learning Inference Applications team. This role is responsible for development and performance optimization of core building blocks of LLM Inference - Attention, MLP, Quantization, Speculative Decoding, Mixture of Experts, etc. The team works side by side with chip architects, compiler engineers and runtime engineers to deliver performance and accuracy on Neuron devices across a range of models such as Llama 3.3 70B, 3.1 405B, DBRX, Mixtral, and so on.

Responsibilities

• Adapting latest research in LLM optimization to Neuron chips to extract best performance from both open source as well as internally developed models.
• Working across teams and organizations.

Requirements

• 3+ years of non-internship professional software development experience.
• 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience.
• Experience programming with at least one software programming language.
• Fundamentals of Machine learning models, their architecture, training and inference lifecycles along with work experience on some optimizations for improving the model performance.

Nice-to-haves

• 3+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience.
• Bachelor's degree in computer science or equivalent.
• Hands-on experience with PyTorch or Jax - preferably involving developing and deploying LLMs in production on GPUs, Neuron, TPU or other AI acceleration hardware.

Benefits

• Equity and sign-on payments may be provided as part of a total compensation package.
• Full range of medical, financial, and/or other benefits.