Lead Data Engineer

National Vision•Duluth, GA

About The Position

This role is responsible for the design, development, and maintenance of data pipelines for the enterprise data warehouse to address the data and analytics needs of the business. What Would You Do? The Specifics. Work with the data and analytics team of project managers, business analysts, data architects, and data modelers to understand requirements and data architecture for an ETL/ELT process. Design and develop robust ETL processes using ETL tools and technologies. Understand existing ETL processes and optimize for enhanced performance. Partner with vendors as well as the internal data analytics team to collaborate and deliver on projects; mentor and lead data engineers. Create technical design documents that capture source-to-target mapping, transformation algorithms, and business rules. Collaborate with the business, data and analytics, and cross-functional team to suggest efficient and innovative solutions. Support ongoing maintenance and optimizations of ETL jobs.

Requirements

6+ years experience in ETL Development and Database Warehousing (Required)
Four-year college degree or equivalent experience. BA/BS in relevant discipline (Required)
Good understanding of Agile and SDLC methodologies. (Required)
Knowledge of industry-wide technology trends and leading practices. (Required)
Demonstrated knowledge of Databases (i.e., Microsoft SQL Server, Azure SQL) (Required)
Proficiency in ETL tools (Databricks, Azure synapse pipelines) (Required)
Proficiency in cloud data warehouses (e.g., Azure.) (Required)
Proficiency in SQL; Ability to understand, modify and write complex SQL queries (Required)
Experience in big data technologies such as Spark, Kafka etc. (Required)
Experience in any programming language such as Java, Python, Scala (Required)
Strong verbal/written communication abilities, effective interpersonal skills, and strong attention to detail. (Required)
Excellent organization, planning, judgment, and time management skills. (Required)
Maximizes efforts towards productivity, identifies problem areas, and assists with implementing solutions. (Required)
Excellent critical thinking skills; using logic and reasoning to identify strengths and weaknesses of alternative solutions, conclusions, or approaches to problems. (Required)

Nice To Haves

Microsoft Azure or Databricks Certifications (Preferred)
Proficiency in data lifecycle, data architecture and data modeling concepts (Preferred)

Responsibilities

Work with the data and analytics team of project managers, business analysts, data architects, and data modelers to understand requirements and data architecture for an ETL/ELT process.
Design and develop robust ETL processes using ETL tools and technologies.
Understand existing ETL processes and optimize for enhanced performance.
Partner with vendors as well as the internal data analytics team to collaborate and deliver on projects; mentor and lead data engineers.
Create technical design documents that capture source-to-target mapping, transformation algorithms, and business rules.
Collaborate with the business, data and analytics, and cross-functional team to suggest efficient and innovative solutions.
Support ongoing maintenance and optimizations of ETL jobs.