Senior Data Engineer

Senior Data Engineer 

• Create and maintain effective data pipeline architectures
• Assemble large, complex data sets Design and implement infrastructure that enables teams to become more self-served
• Design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies
• Build and/or configure monitoring tools and alerts that assure the integrity of operations
• Work with stakeholders including the Product, Engineering and Design teams to assist with data-related technical issues and support their data infrastructure needs.

Requirements

We have seen that people who successfully fit in this position have:

• 4+ years of experience in a Data Engineer role
• Strong Knowledge of Data Management, Data Warehousing concepts (Dimensional Modeling, Data Marts) and Database Technologies.
• Strong understanding of modern approaches to data engineering and the best approach to solve data problems.
• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases
• ETL/ELT processes (Knowledge of DBT is a plus)
• Working experience in Python.
• Hands-on experience in Production Data Lake architecture in cloud and Big Data implementations in general (designing, constructing, cataloging and optimizing data lake)
• Experience with Cloud Technologies and Serverless Computing (we use AWS).
• Working experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
• Strong analytic skills related to working with unstructured datasets.
• Exposure to DevOps and CI/CD practices.
• Familiarity with Change Data Capture (CDC) is a plus.
• Experience in workflow management tools/job orchestration tools is a plus.