Lead Data Consultant - AWS / Databricks

In this role you will lead and craft production-grade data platforms and AI applications for client engagements, acting as a senior technical lead and hands-on architect. You’ll collaborate with client teams to design end-to-end data ecosystems, ensure robust engineering standards, and shape reusable assets for Scale Factory’s practice. You’ll also contribute to the technical practice by building reference implementations and accelerators. This is a hands-on, leadership-heavy role that blends architecture, delivery, and advisory work to enable transformation.

• 25 Days of Annual Leave
• Pension Plan
• Health Benefits
• Flexible Work Model
• Training & Development
• Commuter Benefits

• Design end-to-end data platforms (ingestion, storage, transformation, serving, governance) aligned with technical direction
• Make and document architectural decisions and trade-offs (e.g., lakehouse vs. warehouse)
• Translate business requirements into scalable data pipelines and technical roadmaps
• Embed non-functional requirements (security, privacy, lineage, quality, FinOps) into designs
• Act as Tech Lead on client engagements, guiding delivery teams
• Write, review, and maintain production-grade code
• Establish and champion engineering standards (testing, CI/CD, observability)
• Mentor engineers to raise team output
• Facilitate technical discovery and architecture alignment sessions with clients
• Advise on trade-offs, performance, refactoring, risk mitigation
• Communicate architectural decisions to technical and non-technical stakeholders
• Build reusable assets, reference architectures, and playbooks
• Assist with interviewing and technical assessments for data engineers
• Collaborate with cloud and testing practice groups for cross-pollination

• 6+ years in data architecture and delivery with hands-on leadership
• Proven experience delivering modern data platforms end-to-end on AWS and/or Databricks
• Production coding with Python, SQL, Spark, and Databricks or AWS data tools
• Consulting or client-delivery experience with accountability for outcomes
• Strong data modeling (dimensional, NoSQL, lakehouse/medallion)
• Data governance and operations (catalogues, lineage, access controls, quality checks, monitoring)
• Cost optimization and platform monitoring
• SC clearance or eligibility
• Desirable: Databricks and/or AWS certifications (Associate/Professional)
• Desirable: streaming architectures (Kafka/MSK, Kinesis, Spark)
• Desirable: Data Mesh, data products, or federated governance
• Desirable: AI/ML data foundations (feature stores, MLflow, model serving, RAG)
• Desirable: regulated sectors experience

• Pragmatic decision-making
• Clear communication
• Collaborative coaching/mentoring
• AWS and/or Databricks platforms
• Python, SQL, Spark
• Data modeling and governance