Purpose of the role:
We are looking for a Data Engineer to participate in the design, development, and management of our data warehousing infrastructure. This role involves building and maintaining data pipelines, managing both relational and non-relational databases, optimizing queries, and transforming raw data into structured data for analysis and reporting.
The successful candidate will collaborate closely with the program, MLE, product management, and software engineering teams to understand data requirements and oversee the implementation of suitable solutions to ensure stakeholders’ needs are satisfied.
Embedded within the Digital Health Team, this role will report to the Senior Manager, Data Engineering & Architecture. The ideal candidate will bring extensive expertise in Transactional Databases, Data Warehouses, and BI reporting systems.
Roles And Responsibilities:
- Participate in the design, implementation, and maintenance of data pipelines by performing extraction, transformation, and loading activities from structured and unstructured data sources into a data warehouse.
- Design and Build Data models – star schema, snowflake. Understand common analytical data models like Kimball. Build physical data models and align with best practice and requirements.
- Design, develop, and optimize complex SQL queries to support data discovery, analysis, and reporting. Leverage SQL to ensure accurate data staging and transformation processes that align with business requirements.
- Conduct thorough data discovery to identify and address data quality issues, ensuring that the data is accurate, complete, and accessible for business needs.
- Monitor system logs for errors and performance issues; troubleshoot and resolve issues as they arise. Conduct day-to-day system checks and maintenance tasks to ensure system availability.
- Develop BI technical documentation – data dictionaries, definitions, data flows, database schemas, data model diagrams, Entity Relationship Diagrams (ERDs), etc.
- Collaborate with BI developers and users to understand business rules, capture requirements, develop user stories and write technical/functional specifications based on conceptual design and stated business requirements.
- Assist across internal teams to define excellence in data governance, privacy, and security.
Skills & Competencies:
- 3+ years experience as a Data Engineer.
- Demonstrated experience in implementing data pipelines/ETL into data warehouses and data querying and analysis using cloud-based solutions such as AWS Redshift or Snowflake.
- Proficiency in SQL & Python for advanced querying, data manipulation, and performance optimization.
- Experience with ETL tools like AWS Glue, Airbyte or Talend for building automated data pipelines.
- Strong knowledge of data modeling and transformation using tools like dbt.
- Hands-on experience with workflow automation tools such as Airflow.
- Expertise in managing CouchDB or PostgreSQL databases, including schema management, performance tuning, and complex queries.
- Extensive data warehouse experience, with skills in performance tuning, query optimization, indexing, and data integrity management.
- Proficiency in using BI tools such as Tableau, Power BI, or Supersetfor creating reports and dashboards.
- Strong understanding of data governance, security, and compliance best practices in cloud environments.
- Soft Skills: Teamwork, Collaboration, Problem – solving, Strong communication and presentation skills to effectively translate technical concepts to business stakeholders.
Minimum Qualifications:
- A Bachelor’s degree in computer science, Data Science, Statistics, Mathematics, or a related discipline.
- Professional certifications or equivalent experience in data management, ETL processes, data warehousing, data visualization, and managing large and complex datasets.