IT & Software

Principal Databricks Data Engineer

Astra-North Infoteck Inc.

Toronto · Ontario · Canada

Share this job: X Facebook WhatsApp Save job : Principal Databricks Data Engineer
12–18 years of overall Data Engineering experience

8+ years of experience with enterprise Data Warehouse and Data Lake platforms

5+ years of hands‑on experience with Databricks and Apache Spark at scale

Strong experience modernizing legacy Cloudera platforms including:

Modernize Cloudera platforms to Databricks Lakehouse architecture

Redesign ingestion, transformation, and consumption patterns from HDFS-based architecture to Cloud Object Storage and Delta Lake

Refactor legacy Hive and Impala logic into PySpark and Spark SQL ELT pipelines

Ensure data reconciliation, audit integrity, and consistency during migration

Design and govern enterprise Data Warehouse and Data Lake/Lakehouse architectures

Curated / Conformed Layer

Semantic / Consumption Layer

Modernize traditional Enterprise Data Warehouse platforms into scalable Lakehouse architectures

Strong experience with finance and risk data models including:

Financial Hierarchies

Liquidity Risk

Enable reporting capabilities including:

Build and manage semantic and consumption layers for BI, reporting, and analytics

Define business metrics, dimensions, hierarchies, and KPIs

Experience with:

Develop and optimize large‑scale data pipelines using:

Optimize workloads using:

Data Quality Frameworks

Establish data lineage and metadata management

Ensure data security, access control, and compliance standards

Experience with AWS or Azure cloud platforms

Experience with CI/CD pipelines using:

Familiarity with:

Experience with dbt is an advantage

Act as a technical authority and lead enterprise architecture decisions

Mentor senior engineers and establish engineering standards

Collaborate with finance, risk, analytics, and governance stakeholders

Translate complex data structures into business‑ready insights

Nice‑to‑Have Skills

Experience in BFSI, Capital Markets, or Regulatory Reporting

Oracle Financials

Experience supporting AI/ML workloads

Databricks and Cloud Certifications

Key Responsibilities

Lead Cloudera to Databricks transformation initiatives

Shape enterprise finance and risk data platforms

Support regulatory, management, and analytical reporting systems

#J-18808-Ljbffr

Reference: WJ-3875_16353614 · Posted 17 Nov 2026, 01:25 AM

Find more on GigBlow

This role is listed on GigBlow for discovery and search. Hiring decisions and applications are handled by the employer or their chosen application system.

Databricks Data jobs in other states

Browse jobs