17 of 17 Delta Lake Jobs in the South East

platforms such as Azure, AWS or GCP* Define lakehouse standards, data modelling approaches and governance frameworks* Lead design of batch and streaming platforms using Delta Lake, Spark and related technologies* Establish security, access control, data quality and compliance standards* Optimise performance, scalability and cloud cost management* Lead … experience* Strong experience as a senior or lead data engineer building and scaling production data platforms* Deep hands on expertise in Databricks including Spark, Delta Lake and Databricks Workflows* Python and SQL proficiency* Infrastructure as code experience using Terraform, Bicep or CloudFormation* Cloud data platform experience, ideally Azure ...

platforms such as Azure, AWS or GCP * Define lakehouse standards, data modelling approaches and governance frameworks * Lead design of batch and streaming platforms using Delta Lake, Spark and related technologies * Establish security, access control, data quality and compliance standards * Optimise performance, scalability and cloud cost management * Lead … experience * Strong experience as a senior or lead data engineer building and scaling production data platforms * Deep hands on expertise in Databricks including Spark, Delta Lake and Databricks Workflows * Python and SQL proficiency * Infrastructure as code experience using Terraform, Bicep or CloudFormation * Cloud data platform experience, ideally Azure ...

Data Engineer and partnering with data science, analytics and product teams. Key responsibilities * Build and maintain scalable data pipelines using Databricks, Apache Spark and Delta Lake * Develop and optimise batch and streaming workflows for reliability and performance * Contribute to lakehouse architecture and medallion pattern implementation * Implement data quality … opportunities * Translate business requirements into technical solutions * Contribute to documentation and continuous improvement Required experience * Strong experience delivering production grade Databricks pipelines * Strong Spark, Delta Lake and Databricks Workflows experience * Python and SQL proficiency * Experience on Azure data platforms such as Azure Data Factory, ADLS Gen2, Azure DevOps ...

Senior Data Engineer (Databricks) - London (Hybrid) - Up to £85,000 Databricks - Apache Spark - Delta Lake - Python - SQL - Azure We're partnering with a fast-growing technology business that's investing heavily in its data platform and engineering capability. This is an opportunity to join a highly collaborative team … growing technology business. Key Experience Required Proven experience as a Data Engineer within production environments. Strong hands-on experience with Databricks, Apache Spark, Delta Lake, Python, and SQL. Experience building and supporting scalable ETL/ELT pipelines. Strong understanding of distributed data processing and data modelling. Cloud platform ...

Lead Data Engineer (Databricks) - London (Hybrid) - Up to £110,000 Databricks - Apache Spark - Delta Lake - Python - SQL - Azure We're partnering with a growing technology business investing heavily in its cloud data platform and AI capabilities. This is a senior, hands-on leadership role where … machine learning and large-scale data initiatives. Key Experience Required Proven experience as a Senior or Lead Data Engineer. Deep expertise in Databricks, Spark, Delta Lake, Python and SQL. Experience designing and scaling cloud-based data platforms. Strong knowledge of ETL/ELT, data modelling and distributed processing. ...

Skills Strong experience designing and delivering modern data platforms using Microsoft Fabric, Azure Databricks, Azure Data Factory, Azure Synapse Analytics, Azure SQL and Data Lake technologies . Proven experience implementing Lakehouse, Data Warehouse and Medallion Architecture patterns, including data modelling, governance and performance optimization . Expertise in building scalable … Python, Spark and PySpark for data engineering, analytics and automation workloads. Experience with Azure Databricks and/or Microsoft Fabric engineering workloads, including notebooks, Delta Lake, data transformation, optimization and operationalization of data solutions. An understanding of data governance, security, lineage and data quality, using tools and frameworks ...

communicate complex concepts in a clear, concise manner Languages: Terraform, Python, SQL, PowerShell. CI/CD: Azure DevOps pipelines for data platform deployments. Databricks: Delta Lake, Unity Catalog, Workflows, Notebooks, Clusters, MLflow. Azure: ADLS Gen2, Azure Data Factory, Event Hubs/Kafka, Key Vault, Functions, Synapse. AI Foundry … Service Plan Deployment and Configuration, Integration with Azure bot Services MS Fabric: Tenant Management, Workspace Private Links, Deployment, Capacity Management, Workspace Management, One Lake, Data Mirroring, Power BI AI Foundry : Model management, Hosted Agents Deployment and Management , Project Configuration (Connections and Governance ) and lifecycle management. Inclusion & Diversity We value ...

term platform innovation Required Experience Strong experience designing enterprise data platforms within Microsoft Azure environments Deep expertise with Azure Databricks, Spark (PySpark/Scala), Delta Lake, and Lakehouse architecture Strong hands-on experience with Snowflake architecture and optimisation Experience with Azure Data Factory, ADLS Gen2, Synapse Analytics ...

enable timely, reliable and well-governed data across the organisation.You’ll:Design and develop scalable batch and streaming data pipelines using Databricks, Spark and Delta LakeImplement medallion architecture (bronze/silver/gold) to ensure data quality, lineage and efficient consumptionOptimise Spark jobs and workflows to meet near real … time monitoring SLAsBuild and maintain streaming pipelines using Structured Streaming and Delta Live TablesEnsure strong data governance, including schema enforcement, validation rules and ACID complianceOrchestrate and automate pipelines using Databricks Jobs, APIs or cloud-native schedulersCollaborate with data scientists, analysts and environmental specialists to deliver high-quality, well-documented ...

improving security, supportability and delivery speed. 4. Data Engineering and Integration Design and build reliable ingestion and transformation pipelines using Python, PySpark, SQL and Delta Lake. Implement full and incremental ingestion, CDC where appropriate, schema evolution, reconciliation, error handling and data quality controls. Design reusable integration patterns for REST … Azure Databricks implementation and troubleshooting Databricks Serverless and compute/workload optimisation Azure identity, networking, security, secrets, monitoring and private connectivity Databricks SQL, Delta Lake and performance optimisation Python, PySpark and SQL Jobs/workflows, incremental processing, CDC and data quality REST/API and external data integration ...

workflows and automated testing. Designing evaluation frameworks to benchmark, test and improve GenAI system performance and reliability. The Individual Strong expertise with Databricks, including Delta Lake, Unity Catalog, DLT and MLflow. Deep knowledge of Large Language Models including GPT, Llama, Claude and Mistral. Experience building and deploying GenAI ...

workflows and automated testing. Designing evaluation frameworks to benchmark, test and improve GenAI system performance and reliability. The Individual Strong expertise with Databricks, including Delta Lake, Unity Catalog, DLT and MLflow. Deep knowledge of Large Language Models including GPT, Llama, Claude and Mistral. Experience building and deploying GenAI ...

continuous improvement initiatives. Key Skills & Experience Proven experience as a Data Engineer within production environments. Strong hands-on experience with Databricks, Apache Spark, Delta Lake, Python and SQL. Experience building and supporting scalable ETL/ELT pipelines. Strong understanding of distributed data processing and data modelling. Cloud platform ...

while remaining hands-on with delivery. Experience working across Architecture, Engineering, Product, and Business teams. Data Engineering & Platforms Strong hands-on experience with: Databricks Delta Lake Unity Catalog dbt Spark/PySpark Data pipeline development and optimisation Lakehouse architectures ETL/ELT frameworks Enterprise data platform implementation Cloud ...

peers on safe and effective usage within team practices. Preferred Qualifications, Capabilities, and Skills: Experience building data products on Databricks using lakehouse patterns, including Delta Lake and medallion architecture Familiarity with enterprise data platforms and data mesh principles, including domain-aligned datasets and well-defined data contracts Exposure ...

Python, Data Modelling, Data Warehousing.Strong background in incident resolution, requests, changes and problem‐solving within SLAs.Hands‐on experience with Spark, Data Architecture, SQL and Delta Lake.Cloud development experience (Azure/Fabric).Working knowledge of Medallion Architecture.Demonstrated capability in implementing and supporting pipelines in demanding environments.Strong communication skills and confidence ...

Modelling, Data Warehousing. Strong background in incident resolution, requests, changes and problem-solving within SLAs. Hands-on experience with Spark, Data Architecture, SQL and Delta Lake. Cloud development experience (Azure/Fabric). Working knowledge of Medallion Architecture. Demonstrated capability in implementing and supporting pipelines in demanding environments. Strong ...