Data QA - Test Engineer

Role Overview

We are partnering with an established investment firm undertaking a core data migration from eFront Invest FIA into an Azure Databricks Lakehouse .

We are seeking a hands-on Senior Data QA / Data Quality Engineer to lead automated migration testing, source-to-target reconciliation, and pipeline validation. You will be responsible for ensuring that all data flowing from eFront lands in Databricks with 100% mathematical and schema integrity.

Key Responsibilities

• Source-to-Target Reconciliation: Design, build, and execute automated validation queries to compare legacy eFront extracts against target Databricks Delta Lake tables (Bronze, Silver, Gold).
• Business Logic & Metrics Testing: Verify complex private equity calculation logic (cash flows, commitments, valuations, NAV, and return metrics) across data pipelines.
• Automated Data Quality Frameworks: Develop automated test suites using Python (PyTest) , PySpark , and advanced SQL to catch discrepancies, nulls, and duplicate records.
• CI/CD Quality Gates: Integrate automated testing and data validation checks into Azure DevOps (ADO) deployment pipelines.
• Schema Drift & Anomaly Detection: Monitor and resolve schema evolution, edge-case discrepancies, and late-arriving financial records.

Required Experience & Skills

• Experience testing data extracted from or integrated with eFront (eFront Invest / FIA / FrontScript).
• Proven track record as a Data QA Engineer, Data Quality Engineer, or Data Test Automation Engineer on enterprise cloud migrations.
• Hands-on testing and reconciliation experience within Azure Databricks / Delta Lake environments.
• Familiarity with private markets, fund accounting, or private equity financial structures (NAV, cash flows, capital calls, distributions).