Senior Data Engineer - Python, Airflow, Pyspark, SQL

Senior Data Engineer - Python, SQL, Airflow, Azure, Pyspark, Kubernetes

Salary: Up to £80,000 per annum

Benefits: Excellent benefits package including annual bonus, free onsite gym, and company pension scheme.

About the Role

We are seeking an experienced Senior Data Engineer to join an innovative SaaS business that is transforming the way commercial data is managed and utilised. In this role, you will lead the development of scalable data pipelines and integrate complex data sources into a modern cloud-based platform.

About the Company

Our client is a leading technology business focused on transforming commercial Asset management through data and innovation. Their SaaS platform consolidates and reconciles data across financial, operational, and leasing functions, providing a single source of truth for asset, fund, and portfolio performance.

Key Responsibilities

• Data Pipeline Development
• Data Platform Management
• Stakeholder Collaboration
• Leadership & Mentorship

Technology Stack

• Data Processing: Apache Spark / PySpark
• Workflow Orchestration: Apache Airflow
• Containerisation: Kubernetes
• Cloud Platform: Microsoft Azure
• Data APIs & Semantic Layer: CubeJS
• Programming Languages: Python, SQLEssential Skills & Experience

Essential Skills & Experience

• Degree in a STEM discipline, ideally Computer Science, Software Engineering, or a related field.
• Minimum of 5 years' experience within Data Engineering, Data Warehousing, or a similar role.
• Strong experience with Python and PySpark.
• Excellent SQL skills.
• Experience with data orchestration tools such as Airflow, Azure Data Factory (ADF), SSIS, or similar.
• Solid understanding of data modelling principles and modern data warehouse architectures.
• Experience designing and building scalable, production-grade data pipelines.

Desirable Experience

• Experience working within cloud-native Azure environments.
• Knowledge of modern analytics platforms and semantic data layers.
• Previous mentoring or technical leadership experience.