Databricks Spark Data Engineer
BeachHead
Databricks and Apache Spark specialists can build scalable data solutions for a six-month contract supporting a top financial client. This hands-on role focuses on enterprise-grade pipelines and large-scale datasets.
The Databricks Developer designs, develops, and optimises data pipelines across distributed computing environments. You will use Auto Loader, Spark Structured Streaming, Unity Catalog, and Lakehouse Federation. The work includes solution design, stakeholder collaboration, and practical technical recommendations. Banking or financial services experience is highly preferred.
Key Responsibilities:
- Design and optimise Databricks and Apache Spark pipelines
- Build batch, Auto Loader, and streaming ETL pipelines
- Maintain catalogs, schemas, tables, views, functions, and volumes
- Implement SCD 1, SCD 2, and change data capture
- Improve performance through partitioning and liquid clustering
Requirements:
- Strong hands-on Databricks and Apache Spark experience
- Experience with Azure Databricks or Delta Lake
- Strong Unity Catalog and Lakehouse Federation experience
- Enterprise experience; banking experience is preferred
- Strong communication and stakeholder management skills
Apply Databricks, Spark, streaming ETL, and governance expertise to enterprise data engineering for a financial client.
#J-18808-LjbffrReference: WJ-4483_1622220 · Posted 14 Nov 2026, 03:05 AM