Principal Platform Engineer – Data Ops Engineer

DIRECTV • Los Angeles, CA

1d • Remote

About The Position

DIRECTV Data Analytics and Operations Team is looking for a curious, talented highly motivated Data Ops Engineer to lead the automation, orchestration, and optimization of our cloud-based data workflows across Snowflake, Databricks, and AWS. The ideal candidate will have deep expertise in Apache Airflow, CI/CD, automation, and performance monitoring, with a passion for building scalable, efficient, and high-performance data operations solutions. In this role, you will work closely with data engineering, DevOps, security, and business teams to design and implement next-generation orchestration, automation, and data deployment strategies that drive efficiency, reliability, and cost-effectiveness. This is the perfect opportunity to become a part of a fast paced and innovative team that solves real world problems and drives business value.

Requirements

• 4 – 5 years required, 10+ years preferred of overall software engineering experience, including 10+ years preferred as a Big Data Architect, focused on end-to-end data infrastructure design using Spark, PySpark, Kafka, Databricks, and Snowflake.
• 4 – 5 years required, 8+ years preferred of hands-on programming experience with Python, PySpark, JavaScript, and Shell scripting, with demonstrated expertise in building reusable and configuration-driven frameworks for Databricks.
• 5+ years of experience designing and implementing configuration-driven frameworks in PySpark on Databricks, enabling scalable and metadata-driven data pipeline orchestration.
• 4 – 5 years required, 8+ years preferred of experience in CI/CD pipeline development and automation using GitLab, Jenkins, and Databricks REST APIs, including infrastructure provisioning and deployment at scale.
• 4 – 5 years required, 8+ years preferred of deep expertise in Snowflake, Databricks, and AWS, including migration, optimization, and orchestration of data workflows, as well as advanced features such as masking, time travel, and Delta Lake.
• 7+ years of experience in performance monitoring and observability using tools like SonarQube, JMeter, Splunk, Datadog, and AWS CloudWatch, with a focus on optimizing pipeline efficiency and reducing cost.
• 7+ years of experience in Tier 2/3 support roles, specializing in root cause analysis, incident resolution, and the creation of troubleshooting runbooks and automation for operational resilience.
• 4+ years of experience with dbt, including the conversion of traditional dimensional models to modular dbt models, integration with CI/CD, and application of testing and documentation best practices.
• Deep expertise in Apache Airflow and orchestration technologies, having led large-scale orchestration implementations across multi-cloud environments.
• Strong analytical and architectural skills to design, optimize, and troubleshoot complex data pipelines, with demonstrated success in delivering performance and cost improvements (e.g., $1M in annual savings through Spark SQL optimization).

Nice To Haves

• Databricks Certified Data Engineer Associate / Professional
• SnowPro Advanced Architect Certification
• AWS Certified DevOps Engineer – Professional
• Apache Airflow Certification
• ITIL 4 Managing Professional (for incident management expertise)
• Certified ScrumMaster (CSM) (for agile collaboration skills)

Responsibilities

• Platform Architecture & Data Orchestration Strategy (30%) Define the long-term orchestration strategy and architectural standards for workflow management across Snowflake, Databricks, and AWS. Lead the design, implementation, and optimization of complex workflows using Apache Airflow and related tools. Mentor teams in best practices for DAG design, error handling, and resilience patterns. Champion cross-platform orchestration that supports data mesh and modern data architecture principles.
• Engineering Excellence & Automation Frameworks (25%) Architect and guide the development of reusable automation frameworks in Python, Spark, and Shell that streamline data workflows and platform operations. Lead automation initiatives across data platform teams, setting coding and modularization standards. Evaluate and introduce emerging technologies and scripting tools to accelerate automation and reduce toil.
• Enterprise CI/CD Governance & DevOps Leadership (20%) Define and maintain enterprise-wide CI/CD standards for data pipelines and platform deployments using Jenkins, GitLab, and AWS CodePipeline. Drive adoption of Infrastructure as Code (IaC) and GitOps practices to enable scalable and consistent environment provisioning. Provide technical leadership for DevOps integration across Data, Security, and Cloud Engineering teams.
• Performance Engineering & Platform Optimization (15%) Lead performance audits and capacity planning efforts across Snowflake, Databricks, and orchestrated workflows. Build frameworks for proactive monitoring, benchmarking, and optimization using Datadog, AWS CloudWatch, and JMeter. Partner with platform teams to implement self-healing systems and auto-scaling capabilities.
• Operational Resilience & Leadership Collaboration (10%) Oversee complex incident resolution, lead post-mortems, and implement systemic preventive measures. Develop standardized runbooks, incident response frameworks, and training programs to elevate Tier 2/3 capabilities. Act as a liaison between engineering leadership, security, and business teams to drive platform roadmaps and risk mitigation.

What This Job Offers

• Job Type: Full-time
• Career Level: Principal