Data Engineer

SGA • New York, NY

13d • $80 - $85 • Hybrid

About The Position

Responsibilities : Build and optimize data pipelines to consume data from various sources, transform & model it, and then distribute it to multiple consumers. Collaborate with technology partners across the Firm to understand and integrate with new data sources and consumers. Work with multiple data sources (including structured and unstructured) to create an optimize data cubes for consumption by analytical dashboards and / or downstream systems. Identify, design, and implement improvements / automations for optimizing data processing, storage, and availability, while maintaining scalability, etc. Develop and support analytics & machine learning projects. Support migration of the data platform from on-premises to cloud infrastructure. Work with various stakeholders to assist with data & analytics related projects and issues. Required Skills : We are looking for a candidate with 7+ years of experience in Data Engineering role, who has attained a bachelor's or master's degree in computer science, computer engineering, information technology/systems or another relevant technical field. Advanced working knowledge and experience with data modelling, data warehousing concepts, database design principles, query authoring (SQL) and performance optimization. A successful history of manipulating, transforming, processing, and extracting value from large, disconnected datasets. Hands-on programming experience with Python, Java, or C#. Hands-on experience with ETL tools. Working knowledge of data transformation, processing, analysis, and machine learning using Python (Pandas, NumPy, etc.) Excellent problem-solving and analytical skills with attention to details. Strong project management and organizational skills. Experience supporting and working with cross-functional teams in a dynamic environment. Excellent communication and collaboration skills. Experience with Relational database systems like Sybase, SQL Server, Postgres, etc. Experience with object-oriented and/or function scripting languages: Python, Java, C++, or Scala, etc. Experience with ETL tools (Informatica, etc.) and frameworks. Experience with machine learning / artificial intelligence techniques like regression, classification, clustering, time series, text mining, large language models, etc. Advanced working knowledge and experience with data modelling, data warehousing concepts, database design principles, query authoring (SQL) and performance optimization. Hands-on programming experience in one language (Python, Java, or C#). Hands-on experience with ETL (Informatica, etc.) tools and frameworks. Hands-on experience of data transformation, processing, analysis, and machine learning using Python (Pandas, NumPy, etc.) Preferred Skills : Experience of visualization tools like Tableau, QlikView, Power BI etc. Experience with Cloud technologies like Snowflake, Microsoft Azure, Databricks, etc Experience with data-science tools / workbench like Dataiku, Jupyter, Rapid Miner, etc. Experience with Big Data tools / technologies: Hadoop, Pig, Hive, Hue, PySpark etc.

Requirements

• 7+ years of experience in Data Engineering role
• attained a bachelor's or master's degree in computer science, computer engineering, information technology/systems or another relevant technical field
• Advanced working knowledge and experience with data modelling, data warehousing concepts, database design principles, query authoring (SQL) and performance optimization
• A successful history of manipulating, transforming, processing, and extracting value from large, disconnected datasets
• Hands-on programming experience with Python, Java, or C#
• Hands-on experience with ETL tools
• Working knowledge of data transformation, processing, analysis, and machine learning using Python (Pandas, NumPy, etc.)
• Excellent problem-solving and analytical skills with attention to details
• Strong project management and organizational skills
• Experience supporting and working with cross-functional teams in a dynamic environment
• Excellent communication and collaboration skills
• Experience with Relational database systems like Sybase, SQL Server, Postgres, etc.
• Experience with object-oriented and/or function scripting languages: Python, Java, C++, or Scala, etc.
• Experience with ETL tools (Informatica, etc.) and frameworks
• Experience with machine learning / artificial intelligence techniques like regression, classification, clustering, time series, text mining, large language models, etc.
• Advanced working knowledge and experience with data modelling, data warehousing concepts, database design principles, query authoring (SQL) and performance optimization
• Hands-on programming experience in one language (Python, Java, or C#)
• Hands-on experience with ETL (Informatica, etc.) tools and frameworks
• Hands-on experience of data transformation, processing, analysis, and machine learning using Python (Pandas, NumPy, etc.)

Nice To Haves

• Experience of visualization tools like Tableau, QlikView, Power BI etc.
• Experience with Cloud technologies like Snowflake, Microsoft Azure, Databricks, etc
• Experience with data-science tools / workbench like Dataiku, Jupyter, Rapid Miner, etc.
• Experience with Big Data tools / technologies: Hadoop, Pig, Hive, Hue, PySpark etc.

Responsibilities

• Build and optimize data pipelines to consume data from various sources, transform & model it, and then distribute it to multiple consumers.
• Collaborate with technology partners across the Firm to understand and integrate with new data sources and consumers.
• Work with multiple data sources (including structured and unstructured) to create an optimize data cubes for consumption by analytical dashboards and / or downstream systems.
• Identify, design, and implement improvements / automations for optimizing data processing, storage, and availability, while maintaining scalability, etc.
• Develop and support analytics & machine learning projects.
• Support migration of the data platform from on-premises to cloud infrastructure.
• Work with various stakeholders to assist with data & analytics related projects and issues.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level