Lead Data Engineer (ADF+SQL)

Everest Technologies • New Albany, OH

1d

About The Position

We are currently seeking a SQL + ADF developer to join our team in Columbus, OH. Job Duties Design, develop, and maintain robust ETL pipelines using Azure Data Factory (ADF) for data integration. Write and optimize complex SQL queries for data extraction, transformation, and loading across large datasets. Implement data workflows and orchestrations in ADF, ensuring scalability and reliability. Collaborate with data architects and analysts to deliver high-quality data solutions aligned with business needs. Monitor and troubleshoot ADF pipelines, resolving failures and optimizing performance. Ensure data security, compliance, and governance standards are met within the Azure environment. Perform performance tuning for SQL queries and ADF activities to meet SLA requirements. Document ETL processes, pipeline configurations, and data flow diagrams for transparency and maintainability. Continuously explore and implement best practices for Azure-based data engineering solutions.

Requirements

• Strong proficiency in SQL, including advanced query optimization and stored procedures.
• Hands-on experience with Azure Data Factory (ADF) for building and managing data pipelines.
• Exposure to Azure ecosystem components such as Data Lake, Synapse, and related services.
• Solid understanding of ETL concepts and data integration best practices.
• Ability to troubleshoot and optimize ADF workflows for performance and reliability.
• Familiarity with version control systems (Git) and CI/CD pipelines for data solutions.
• Knowledge of data governance, security, and compliance within cloud environments.
• Strong analytical and problem-solving skills with attention to detail.

Nice To Haves

• Experience with Microsoft Fabric (Lakehouse, OneLake) is a strong plus.

Responsibilities

• Design, develop, and maintain robust ETL pipelines using Azure Data Factory (ADF) for data integration.
• Write and optimize complex SQL queries for data extraction, transformation, and loading across large datasets.
• Implement data workflows and orchestrations in ADF, ensuring scalability and reliability.
• Collaborate with data architects and analysts to deliver high-quality data solutions aligned with business needs.
• Monitor and troubleshoot ADF pipelines, resolving failures and optimizing performance.
• Ensure data security, compliance, and governance standards are met within the Azure environment.
• Perform performance tuning for SQL queries and ADF activities to meet SLA requirements.
• Document ETL processes, pipeline configurations, and data flow diagrams for transparency and maintainability.
• Continuously explore and implement best practices for Azure-based data engineering solutions.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed