26 to 50 of 132 Delta Lake Jobs in England

Lead Architect, Data & AI Platforms

Location
Greater London, England, United Kingdom
other teams can pick up and run, rather than as documents. Azure-First Data Platforms : Deep architecture and build experience on Microsoft Azure (Data Lake/Lakehouse patterns, ADLS Gen2, Synapse Analytics, Azure Databricks, Delta Lake), with enough hands-on AWS or Google Cloud Platform (GCP) experience … Lead Azure-First Platform Architecture : Architect and implement Azure data platforms for clients, including landing zones, networking, identity, and data platform services (Azure Data Lake Storage, Synapse, Databricks), extending patterns to AWS or GCP where a client estate requires it. Design & Build Data Pipelines : Design, Implement key data ingress ...

Senior Data Engineer

Location
Manchester, England, United Kingdom
scalable, maintainable, and aligned with organisational standards and governance requirements. Design, build, and optimise data pipelines on Databricks using PySpark and Spark SQL, applying Delta Lake and a medallion/Lakehouse architecture. Implement data validation and reconciliation checks, and automated testing across pipelines to ensure trusted, accurate outputs. ...

Lead Data Engineer

Location
Southampton, England, United Kingdom
make sound engineering decisions. Strong communication skills, with the ability to explain complex technical concepts clearly to both technical and non‐technical audiences. Databricks Delta Lake, Unity Catalog, Workflows, Asset Bundles, Notebooks, Compute Management, Delta Live Tables, Spark Declarative Pipelines and platform optimisation. Microsoft Fabric Lakehouse, Data ...

Senior Data Engineer

Location
City Of London, England, United Kingdom
Unifeye. What You'll Do Design and build scalable batch and streaming data pipelines using Spark and Databricks. Develop modern Lakehouse architectures using Delta Lake and Medallion (Bronze, Silver and Gold) design principles. Build robust ingestion frameworks from APIs, files and streaming data sources. Ensure data quality, testing … data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. Good understanding of Lakehouse architecture and Delta Lake. Experience working with AWS, Azure or GCP. Experience implementing CI/CD, version control and deployment automation. Strong communication skills and experience working ...

Data Engineer

Location
Greater London, England, United Kingdom
Unifeye. What You'll Do Design and build scalable batch and streaming data pipelines using Spark and Databricks. Develop modern Lakehouse architectures using Delta Lake and Medallion (Bronze, Silver and Gold) design principles. Build robust ingestion frameworks from APIs, files and streaming data sources. Ensure data quality, testing … data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. Good understanding of Lakehouse architecture and Delta Lake. Experience working with AWS, Azure or GCP. Experience implementing CI/CD, version control and deployment automation. Strong communication skills and experience working ...

Data Technical Lead

Location
City of Westminster, England, United Kingdom
Snowflake and cloud-native data services Data architecture:Lakehouse, medallion, datameshand domain-oriented patterns Data processing:Apache Spark,SQLand distributed processing technologies Data storage:Delta Lake, ApacheIcebergand open table formats Data pipelines:Enterprise-scale ETL/ELT,batchand streaming patterns Transformation and quality:dbt, Great Expectations and equivalent ...

Enterprise Data Architect

Location
Greater London, England, United Kingdom
architecture, and environment strategy.* Design and govern integration patterns for sharing data with external clients and third parties, including secure, scalable approaches using Databricks Delta Sharing, APIs, and file-based exchange where appropriate.* Lead the design and adoption of Master Data Management (MDM), including entity definitions, golden-record strategy … match-and-merge rules, and stewardship processes.* Establish standards and practices for optimising Databricks performance and cost — cluster and compute strategy, Delta table, caching, and workload right-sizing.* Define data modelling standards (conceptual, logical, physical) across analytical and operational domains.* Work with data governance colleagues to embed data quality ...

Lead Data Tester

Hiring Organisation
Quilter
Location
London, England, United Kingdom
their business impact.• Clear communication of test progress, coverage, defects and residual risks to technical and non- technical stakeholders.• Databricks: Practical testing skills across Delta Lake, Unity Catalog, Workflows, Asset Bundles, Notebooks, Compute Management, Delta Live Tables and Spark Declarative Pipelines, including data integrity and performance validation. ...

Lead Data Tester

Location
Southampton, England, United Kingdom
their business impact. Clear communication of test progress, coverage, defects and residual risks to technical and non‐technical stakeholders. Databricks: Practical testing skills across Delta Lake, Unity Catalog, Workflows, Asset Bundles, Notebooks, Compute Management, Delta Live Tables and Spark Declarative Pipelines, including data integrity and performance validation. ...

Senior Data Engineer

Location
Greater London, England, United Kingdom
ongoing enhancement and modernisation of enterprise data platforms Databricks & Data Platform Engineering Develop and support Databricks-based data solutions and Lakehouse architectures Utilise Delta Lake and Spark technologies to deliver scalable data processing capabilities Optimise distributed data processing workloads for performance and reliability Implement data quality controls ...

Senior Data Engineer

Location
Greater London, England, United Kingdom
tested code for production data workloads. Practical experience building and optimising distributed data processing workloads using Spark and Databricks, or comparable platforms. Experience with Delta Lake or comparable lakehouse technologies, including notebooks, workflows and reliable production deployment. Experience delivering and operating data solutions on at least one major ...

Principal AI Data Engineer - GenAI & Agentic AI

Location
Greater London, England, United Kingdom
projectsDeep understanding of large language model families, including GPT, Llama, Claude and Mistral, and their reasoning capabilitiesStrong hands-on experience with Databricks, including Delta Live Tables (DLT), Delta Lake concepts and Unity Catalog governanceSolid understanding of streaming technologies, including Spark Structured Streaming and Auto LoaderStrong programming skills ...

Lead Data Engineer

Location
Manchester, England, United Kingdom
Apache Kafka or Amazon MSK, including stream processing with Spark Structured Streaming. Hands‐on experience with open table formats, such as Apache Iceberg or Delta Lake, including partitioning, schema evolution and table maintenance at scale. Experience with pipeline orchestration and infrastructure‐as‐code using Terraform. Experience acting ...

Senior Solution Architect, Data & AI

Location
City Of London, England, United Kingdom
batch, streaming, change data capture (CDC), ETL/ELT and orchestration. A good understanding of relational and non-relational databases, open table formats (e.g. Delta Lake, Apache Iceberg), and data storage and lifecycle design. Practical knowledge of data governance, including cataloguing, lineage, data quality, ownership, classification and access ...

Data Architect

Location
Brentford, England, United Kingdom
least 3 years hands-on with Azure Databricks in production environments Proven experience designing and implementing medallion architecture (Bronze/Silver/Gold) using Delta Lake on Azure Databricks Strong hands-on experience with Unity Catalog — metastore setup, catalog and schema design, access controls, and governance policies Experience … configuring Azure Data Lake Storage Gen2 — containers, access control lists, lifecycle policies, and integration with Databricks Hands-on experience with Microsoft Fabric — specifically Mirrored Azure Databricks Unity Catalog and Power BI connectivity to Databricks SQL Warehouses Experience with enterprise data source integration — MS SQL Server (on-premises and Azure ...

Data Architect

Location
Manchester, England, United Kingdom
least 3 years hands-on with Azure Databricks in production environments Proven experience designing and implementing medallion architecture (Bronze/Silver/Gold) using Delta Lake on Azure Databricks Strong hands-on experience with Unity Catalog — metastore setup, catalog and schema design, access controls, and governance policies Experience … configuring Azure Data Lake Storage Gen2 — containers, access control lists, lifecycle policies, and integration with Databricks Hands-on experience with Microsoft Fabric — specifically Mirrored Azure Databricks Unity Catalog and Power BI connectivity to Databricks SQL Warehouses Experience with enterprise data source integration — MS SQL Server (on-premises and Azure ...

Senior Data Engineer

Hiring Organisation
Hireful
Location
Leeds, Yorkshire, United Kingdom
Employment Type
Permanent Contract, On-site
Salary
£89,999
extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise data architecture, balancing … growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ELT solutions. ...

Senior Data Engineer

Hiring Organisation
Hireful
Location
Bristol, South West, United Kingdom
Employment Type
Permanent Contract, On-site
Salary
£89,999
extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise data architecture, balancing … growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ELT solutions. ...

Senior Data Engineer

Hiring Organisation
Hireful
Location
Central London, South East, United Kingdom
Employment Type
Permanent Contract, On-site
Salary
£89,999
extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise data architecture, balancing … growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ELT solutions. ...

Senior Data Engineer - Databricks

Hiring Organisation
hireful
Location
London, United Kingdom
Employment Type
Permanent
Salary
£80000 - £90000/annum £80K - £90K Basic + 10% + Extensive
extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise data architecture, balancing … growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ELT solutions. ...

Data Engineer

Hiring Organisation
UK Tote Group
Location
London, England, United Kingdom
further detail at interview.What You’ll Be Doing:Designing, building, and optimising streaming and batch pipelines in Databricks Develop robust bronze, silver, and gold delta tables using the Medallion Architecture to serve downstream analytics and APIsImplement and manage integrations from AWS MSK (Kafka), S3, and external APIs into … whether in PySpark, including Structured Streaming.You should have experience with Kafka (MSK) and real-time data ingestion, as well as a deep understanding of Delta Lake, Declarative Pipelines (DLT) and the Medallion Architecture. A strong AWS background is important, particularly with services such as S3.You’ll need ...

Data Engineer

Location
Wigan, England, United Kingdom
detail at interview. What You’ll Be Doing: Designing, building, and optimising streaming and batch pipelines in Databricks Develop robust bronze, silver, and gold delta tables using the Medallion Architecture to serve downstream analytics and APIs Implement and manage integrations from AWS MSK (Kafka), S3, and external APIs into … PySpark, including Structured Streaming. You should have experience with Kafka (MSK) and real-time data ingestion, as well as a deep understanding of Delta Lake, Declarative Pipelines (DLT) and the Medallion Architecture. A strong AWS background is important, particularly with services such as S3. You’ll need ...

Senior Azure Data Engineer

Location
Newcastle upon Tyne, England, United Kingdom
growth for all data, analytic and AI services. Your involvement will span the entire data lifecycle: from orchestrating data ingestion into our Azure Data Lake, designing and developing sophisticated data models for analytics, to deploying Azure Logic Apps and Function Apps for process automation. With tools like Databricks, Data … environment Develop and implement data ingestion solutions on the Azure platform, ensuring scalability, reliability, and performance. Leveraging technologies such as Azure Data Factory, Databricks, Delta Lake, and Python Design and maintain robust ETL/ELT processes to integrate data from various sources into Azure data services Help ...

Data Engineer

Location
Manchester, England, United Kingdom
meet deadlines. Experience with source control and CI/CD (GitHub, Azure DevOps, DABs) Data architecting, solutions architecting and experience optimising pipelines and Delta Lake Databricks Data Engineer Professional certified #J-18808-Ljbffr ...

Senior Full Stack Software Engineer

Location
Brentford, England, United Kingdom
enhancement, rendering or tagging applications. Integrating AI, Machine Learning or computer vision products into customer-facing software. GIS applications or geo-referenced imagery. Data-lake or lakehouse technologies such as Delta Lake, Apache Iceberg, Apache Hudi, Databricks, Snowflake or AWS Lake Formation. Event-driven architectures using ...