Sr. Data Engineer (Hybrid)

American Medical Association•Chicago, IL

6d•$115,523 - $150,972•Hybrid

About The Position

Sr. Data Engineer (Hybrid) Chicago, IL The American Medical Association (AMA) is the nation's largest professional Association of physicians and a non-profit organization. We are a unifying voice and powerful ally for America's physicians, the patients they care for, and the promise of a healthier nation. To be part of the AMA is to be part of our Mission to promote the art and science of medicine and the betterment of public health. At AMA, our mission to improve the health of the nation starts with our people. We foster an inclusive, people-first culture where every employee is empowered to perform at their best. Together, we advance meaningful change in health care and the communities we serve. We encourage and support professional development for our employees, and we are dedicated to social responsibility. We invite you to learn more about us and we look forward to getting to know you. We have an opportunity at our corporate offices in Chicago for a Sr. Data Engineer (Hybrid) on our Information Technology team. This is a hybrid position reporting into our Chicago, IL office, requiring 3 days a week in the office. As a Sr. Data Engineer, you will play a key role in implementing and maintaining AMA’s enterprise data platform to support analytics, interoperability, and responsible AI adoption. This role partners closely with platform engineering, data governance, data science, IT security, and business stakeholders to deliver high‑quality, reliable, and secure data products. This role contributes to AMA’s modern lakehouse architecture, optimizing data operations, and embedding governance and quality standards into engineering workflows. This role serves as a senior technical contributor within the team—providing mentorship to junior engineers and implementing engineering best practices within the data platform function, in alignment with architectural direction set by leadership.

Requirements

Bachelor's degree in Computer Science, Engineering, Information Systems, or related field preferred or equivalent work experience and HS diploma/equivalent education required.
5+ years of experience in data engineering within cloud environments
Experience in people management preferred.
Demonstrated hands-on experience with modern data platforms (Databricks preferred).
Proficiency in Python, SQL, and data transformation frameworks.
Experience designing and operationalizing ETL/ELT pipelines, orchestration workflows (Airflow, Databricks Workflows), and CI/CD processes.
Solid understanding of data modeling, structured/unstructured data patterns, and schema design.
Experience implementing governance and quality controls: metadata, lineage, validation, stewardship workflows.
Working knowledge of cloud architecture, IAM, networking, and security best practices.
Demonstrated ability to collaborate across technical and business teams.
Exposure to AI/ML engineering concepts, feature stores, model monitoring, or MLOps patterns.
Experience with infrastructure‑as‑code (Terraform, CloudFormation) or DevOps tooling.

Responsibilities

Data Engineering & AI Enablement Build and maintain scalable data pipelines and ETL/ELT workflows supporting analytics, operational reporting, and AI/ML use cases.
Implement best practice patterns for ingestion, transformation, modeling, and orchestration within a modern lakehouse environment (e.g., Databricks, Delta Lake, Azure Data Lake).
Develop high‑performance data models and curated datasets with strong attention to quality, usability, and interoperability; create reusable engineering components and automation.
Collaborate with the Architecture Team, the Data Platform Lead, and federated IT teams to optimize storage, compute, and architectural patterns for performance and cost‑efficiency.
Build model-ready data sets and feature pipelines to support AI/ ML use cases; serve as a technical coordination point supporting business units’ AI-related infrastructure needs.
Collaborate with data scientists and AI Working Group to operationalize models responsibly and maintain ongoing monitoring signals.
Governance, Quality & Compliance Embed data governance, metadata standards, lineage tracking, and quality controls directly into engineering workflows; ensure technical implementation and alignment within engineering workflows.
Work with the Data Governance Lead and business stakeholders to operationalize stewardship, classification, validation, retention, and access standards.
Implement privacy‑by‑design and security‑by‑design principles, ensuring compliance with internal policies and regulatory obligations.
Maintain documentation for pipelines, datasets, and transformations to support transparency and audit requirements.
Platform Reliability, Observability & Optimization Monitor and troubleshoot pipeline failures, performance bottlenecks, data anomalies, and platform‑level issues.
Implement observability tooling, alerts, logging, and dashboards to ensure end‑to‑end reliability.
Support cost governance by optimizing compute resources, refining job schedules, and advising on efficient architecture.
Collaborate with the Data Platform Lead on scaling, configuration management, CI/CD pipelines, and environment management.
Collaborate with business units to understand data needs, translate them into engineering requirements, and deliver fit-for-purpose data solutions; share and apply best practices and emerging technologies within assigned initiatives.
Work with IT Security and Legal/ Compliance to ensure platform and datasets meet risk and regulatory standards.
Staff Management Lead, mentor, and provide management oversight for staff.
Responsible for setting objectives, evaluating employee performance, and fostering a collaborative team environment.
Responsible for developing staff knowledge and skills to support career development.
May include other responsibilities as assigned