Staff Software Engineer, Workflow Orchestration

About the position

Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way. The Workflow Orchestration team is responsible for the data workflow orchestration solution, which powers a big range of data use cases from offline data analytics, experiments, to AI/ML model developments, trainings. We are the founding team of Apache Airflow and a team with rich and diverse experience in different orchestration solutions.

Responsibilities

• Work with data infra engineers, data engineers, data scientists and product managers to design and build frameworks to support data development workflows.
• Developing and maintaining automation tools to streamline deployment and management of our platform.
• Maintain and evolve data engineering tooling to ensure high availability, reliability, usability and performance.
• Collaborate with data engineering and analytics teams to understand their needs and requirements to create a paved path for interacting with the data warehouse.
• Design & implement frameworks (applications) for processing data at scale such as a metrics platform, SQL generation framework or a data quality framework.

Requirements

• 10+ years of experience in software engineering, with a focus on data infrastructure or supporting data engineers and/or data scientists.
• Strong programming skills in one or more of the following languages: Java, Python, or Scala.
• Experience building data applications that create and implement higher-level abstractions on top of the lower-level data infrastructure to simplify complex data operations, improve scalability, and enhance overall system performance.
• Experience working with data storage and distributed processing technologies (i.e. Hive, Spark, Trino, Flink or other SQL databases, etc …)
• Experience with data modeling, database design and various SQL dialects.
• Experience with containerization technologies such as Docker and Kubernetes.
• Extensive background in workflow orchestration solutions such as Apache Airflow, Luigi, Azkaban, Oozie, Prefect or Kubeflow.
• Utilize experience in Data Engineering to optimize processes and bring forth new ideas and technologies that enhance our data capabilities.
• Excellent communication skills and the ability to collaborate with cross-functional teams to build empathetic tools and platforms.
• Excellent analytical and problem solving skills.
• BS/MS/PhD in Computer Science, a related field, or equivalent work experience preferred.

Benefits

• Base pay range of $204,000—$260,000 USD.
• Bonus eligibility.
• Equity options.
• Employee Travel Credits.