Data Engineer, Quality

Case IQ

9d • $110 - $130 • Remote

About The Position

Case IQ helps companies protect their employees, culture, and business through world-class software for uncovering, investigating, and preventing fraud, ethics, harassment, discrimination, and security incidents. We’re proud to count a roster of Fortune 500 companies among our customers, which have relied on Case IQ for managing millions of cases over the past 20 years and helping mitigate billions of dollars in financial and brand risk.

As a Data Engineer – Quality on the Continuous Monitoring team, you will play an important role in ensuring the integrity of business-critical data. You will work as an expert Data Engineer with focus on end-to-end data quality. You will manage data testing activities to certify that data is moving smoothly through pipelines. You will work closely with product, engineering, development management, and other stakeholders to promote data quality in user-facing and internal applications. This role is development heavy and requires proficiency in software engineering and programming.

Requirements

• Graduate of a university or college program in computer science, programming, or engineering
• 5+ years of experience in a data engineering role with focus on ensuring data quality
• 3+ years of experience with Apache Airflow
• 3+ years of experience with SQL
• Proficient in Python Programming, SQL queries and frameworks to build full stack applications
• Proficiency in writing software for Apache Airflow DAGs
• Proficiency in solving data problems using Python as a primary language and through Python-based frameworks
• Understanding of end to end system architecture and workflows design
• Experience with Great Expectations or similar data validation tools
• Experienced with CI/CD pipelines, Github workflows, actions, and feature flags
• Strong interpersonal skills and ability to work effectively in a team environment
• Well-organized, result-oriented, self-motivated, collaborative
• Strong ability to prioritize tasks, troubleshoot problems, and resolve issues

Responsibilities

• Develop full stack applications to test and validate data quality
• Ensure the quality (accuracy, completeness, timeliness) of Continuous Monitoring customer data
• Run, monitor, and troubleshoot Apache Airflow data pipelines in business critical environments
• Leverage Apache Spark to design and implement scalable data processing pipelines, ensuring efficient handling of large datasets and high data quality
• Implement and maintain data validation frameworks using Great Expectations and similar frameworks
• Write SQL queries to analyze data for various quality conditions
• Develop and maintain Python-based data ingestion pipelines and automated test scripts
• Debug issues and conduct exploratory testing to identify root cause of issues
• Develop a Quality Strategy and Framework for the Data Engineering team
• Implement test plans and create test scenarios and test cases
• Report defects and issues identified during the test process using a defect tracking system (Jira)
• Attend daily stand-ups to report on testing progress and identify and impediments
• Prioritize and implement multiple activities in parallel
• Engage in peer reviews, requirements, code, design, and tests
• Take ownership of the quality of the product, reproduce customer issues, and perform root cause analysis for test escapes
• Work with developers to capture root causes and supporting artifacts (e.g., logs)

Benefits

• Work remotely within a flexible work environment (our team spans the US and Canada)
• Competitive company-paid benefits plan starting day 1!
• Generous professional development budget
• RRSP/ 401k matching program
• Half-day Fridays in the summer

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level