Senior Data Quality Engineer (I, II, III) bei Cellares

Cellares is seeking an innovative and highly motivated Senior Data Quality Engineer who will contribute to the development of our advanced cell therapy manufacturing platform.The primary focus of this position is to ensure the accuracy, reliability, and integrity of data within our data platform. The individual will participate on a cross-functional team, design, build, and maintain automated testing frameworks to ensure data integrity at every stage of our data pipelines. The successful candidate should have extensive experience in quality assurance for data platforms, ideally with significant hands-on experience in the Databricks environment. They should be detail-oriented and possess strong analytical and problem-solving skills.Candidates should enjoy working in a fast-paced, mission-driven environment, and be prepared to tackle a broad selection of challenges as the company grows. Candidates should be great team players with the ability to work with minimal supervision.



Responsibilities

Build and maintain automated data validation tests using Databricks notebooks and tools like Pytest

Test data ingestion, transformation, and loading processes within the Databricks Lakehouse, specifically focusing on the Bronze, Silver, and Gold layers of the Medallion architecture

Implement tests for data accuracy, completeness, consistency, timeliness, and uniqueness at different points in the pipeline to catch data issues early

Reconcile data by comparing record counts, schemas, and values between source systems and target tables in Databricks

Implement automated data quality checks within data pipelines to ensure no data regressions occur with new code deployments

Implement automated monitoring and alerting for data quality metrics, identifying anomalies in data freshness, schema evolution, and volume

Work closely with data engineers and product owners to understand data requirements and ensure data quality meets business needs

Ensure compliance with data governance policies by building quality checks that validate data sensitivity, masking, and lineage, leveraging tools like Unity Catalog

Communicate project status and new discoveries in a clear and timely manner during daily stand-ups

Requirements

Bachelor’s or Master’s in Computer Science, Electrical Engineering, or related field and 5+ years of relevant experience

Experience with data pipeline and data quality testing strategy and execution, with significant hands-on experience in the Databricks environment

Strong proficiency in Python for developing and executing data validation scripts

In-depth knowledge of Databricks, Delta Lake, and the Lakehouse architecture. Proficiency in writing complex SQL queries for data validation, reconciliation, and troubleshooting issues

Solid understanding of data warehousing concepts, including dimensional modeling (star/snowflake schemas)

Hands-on experience with Azure, including Azure storage and data services that integrate with Databricks

Ability to process data, interpret testing results and provide feedback to the team

Desire to be part of a rapidly evolving organization, with compelling technology, and taking products and processes to the next level

Self-awareness, integrity, authenticity, and a growth/entrepreneurial mindset

Jetzt bewerben