3 of 3 Delta Lake Jobs in the North of England

fast-growing organisation. Key Responsibilities: Design, develop, and maintain scalable data pipelines using SQL and Python (PySpark) . Ingest, transform, and curate data from multiple sources into Azure Data Lake and Delta Lake formats. Build and optimize datasets for performance and reliability in Azure Databricks . Collaborate with analysts and business stakeholders to translate data requirements into … Skills & Experience: Strong proficiency in SQL for data transformation and performance tuning. Solid experience with Python , ideally using PySpark in Azure Databricks . Hands-on experience with Azure Data Lake Storage Gen2 . Understanding of data warehouse concepts , dimensional modelling , and data architecture . Experience working with Delta Lake and large-scale data processing. Experience building ETL More ❯

data estate, built primarily on Microsoft Azure and Databricks Key Responsibilities: Design and implement scalable, secure cloud-based data architecture (Azure & Databricks) Develop optimised data models and pipelines using Delta Lake and Azure services Define data standards, policies, and governance practices aligned with compliance Enable real-time analytics and machine learning use cases across business functions Ensure data … engineering and architecture Collaborate with internal stakeholders and third-party vendors Key Skills & Experience: Proven background designing and delivering enterprise-scale data platforms Strong knowledge of Microsoft Azure, Databricks, Delta Lake, and data warehousing Advanced data modelling and ETL/ELT optimisation experience Familiarity with regulatory frameworks such as IFRS 17 , BCBS 239 , and UK Data Protection Excellent More ❯

being phased out by late 2025. This project involves migrating pipelines and platform infrastructure running on PVC to the Databricks Enterprise Edition (E2) using EC2-based compute, which enables Delta Lake, advanced security, and scalable architecture. The project also includes decomposing and migrating core components of a centralized automation framework, a multi-repository CI/CD system that … cloud-native environments. Strong experience with AWS services including EC2, IAM, ECR, S3, and Autoscaling. Deep knowledge of Databricks, including workspace setup, instance profiles, notebook CI/CD, and Delta Lake capabilities. Hands-on with Terraform for infrastructure provisioning and GitLab/Jenkins for pipeline orchestration. Experience with Docker, container image lifecycle, and secure CI/CD container More ❯