IT & Software

Principal Databricks Data Engineer

Astra-North Infoteck Inc.

Toronto · Ontario · Canada

Share this job: X Facebook WhatsApp Save job : Principal Databricks Data Engineer
  • 12–18 years of overall Data Engineering experience
  • 8+ years of experience with enterprise Data Warehouse and Data Lake platforms
  • 5+ years of hands‑on experience with Databricks and Apache Spark at scale
  • Strong experience modernizing legacy Cloudera platforms including:
    • Modernize Cloudera platforms to Databricks Lakehouse architecture
    • Redesign ingestion, transformation, and consumption patterns from HDFS-based architecture to Cloud Object Storage and Delta Lake
    • Refactor legacy Hive and Impala logic into PySpark and Spark SQL ELT pipelines
    • Ensure data reconciliation, audit integrity, and consistency during migration
    • Design and govern enterprise Data Warehouse and Data Lake/Lakehouse architectures
    • Curated / Conformed Layer
    • Semantic / Consumption Layer
    • Modernize traditional Enterprise Data Warehouse platforms into scalable Lakehouse architectures
  • Strong experience with finance and risk data models including:
    • Financial Hierarchies
    • Liquidity Risk
  • Enable reporting capabilities including:
    • Build and manage semantic and consumption layers for BI, reporting, and analytics
    • Define business metrics, dimensions, hierarchies, and KPIs
  • Experience with:
    • Develop and optimize large‑scale data pipelines using:
    • Optimize workloads using:
    • Data Quality Frameworks
    • Establish data lineage and metadata management
    • Ensure data security, access control, and compliance standards
    • Experience with AWS or Azure cloud platforms
    • Experience with CI/CD pipelines using:
    • Familiarity with:
    • Experience with dbt is an advantage
  • Act as a technical authority and lead enterprise architecture decisions
  • Mentor senior engineers and establish engineering standards
  • Collaborate with finance, risk, analytics, and governance stakeholders
  • Translate complex data structures into business‑ready insights

Nice‑to‑Have Skills

  • Experience in BFSI, Capital Markets, or Regulatory Reporting
  • Oracle Financials
  • Experience supporting AI/ML workloads
  • Databricks and Cloud Certifications

Key Responsibilities

  • Lead Cloudera to Databricks transformation initiatives
  • Shape enterprise finance and risk data platforms
  • Support regulatory, management, and analytical reporting systems
#J-18808-Ljbffr

Reference: WJ-4483_1644133 · Posted 16 Nov 2026, 05:12 PM

Find more on GigBlow

This role is listed on GigBlow for discovery and search. Hiring decisions and applications are handled by the employer or their chosen application system.

Databricks Data jobs in other states

Browse jobs