1 to 25 of 189 Delta Lake Jobs in the UK

business. As part of this journey, we're consolidating our data capabilities onto a single Azure Databricks platform, leveraging technologies including Azure Data Factory, Delta Lake, Unity Catalog and Power BI to unlock new insights and drive innovation. Joining our Enterprise Data Team , you'll play … driven initiatives across the organisation. You'll be responsible for building and maintaining data pipelines using Azure Databricks, Azure Data Factory and Delta Lake, supporting the modernisation of legacy platforms while ensuring data quality, reliability and performance. The role will also involve implementing data governance and security controls ...

business. As part of this journey, we're consolidating our data capabilities onto a single Azure Databricks platform, leveraging technologies including Azure Data Factory, Delta Lake, Unity Catalog and Power BI to unlock new insights and drive innovation. Joining our Enterprise Data Team , you'll play … driven initiatives across the organisation. You'll be responsible for building and maintaining data pipelines using Azure Databricks, Azure Data Factory and Delta Lake, supporting the modernisation of legacy platforms while ensuring data quality, reliability and performance. The role will also involve implementing data governance and security controls ...

Responsibilities Design, develop and maintain scalable data pipelines using Python, PySpark and Databricks Build and optimise ETL/ELT workflows using Spark and Delta Lake Engineer and transform data supporting Counterparty Credit Risk, Credit Risk and wider risk functions Work with datasets relating to areas such as counterparties … hands-on Python development experience Strong PySpark/Apache Spark engineering experience Extensive hands-on experience with Databricks , including notebooks, jobs/workflows and Delta Lake Advanced SQL and relational database experience Strong track record designing and building scalable ETL/ELT data pipelines Experience with Delta ...

optimize enterprise-scale data platforms using the Databricks Lakehouse Platform . The ideal candidate will have deep expertise in Databricks architecture, Apache Spark, Delta Lake, data engineering, cloud platforms, and enterprise data solutions. Key Responsibilities Design and architect Databricks Lakehouse solutions for large-scaleenterprise data platforms. Lead … deployment, and optimization. Define Databricks workspace architecture, clusterstrategies, security models, and operational standards. Design and developscalable data pipelinesusing Databricks, Apache Spark, PySpark, SQL, andDelta Lake . Implement data ingestionframeworks supporting batchand real-timedata processing. Design Delta Lake architectures including: Delta tables Medallion architecture (Bronze/Silver ...

intersection of consulting, technology, innovation, and strategic alliances and you will bring deep expertise in the Databricks Data Intelligence Platform, Lakehouse architecture, Delta Lake, Apache Spark, Unity Catalog, MLflow, Mosaic AI, data engineering, machine learning, GenAI and cloud-native delivery. As a key leader within our AI & Data … value pools, translate business needs into Databricks-led solution designs, build prototypes and proof points. Shape end-to-end solutions using Databricks Intelligence Platform, Delta Lake, Apache Spark, Unity Catalog, Databricks Workflows, Delta Live Tables/Lakeflow-style pipelines, MLflow, Feature Store, Model Serving and Mosaic ...

intersection of consulting, technology, innovation, and strategic alliances and you will bring deep expertise in the Databricks Data Intelligence Platform, Lakehouse architecture, Delta Lake, Apache Spark, Unity Catalog, MLflow, Mosaic AI, data engineering, machine learning, GenAI and cloud-native delivery. As a key leader within our AI & Data … value pools, translate business needs into Databricks-led solution designs, build prototypes and proof points. Shape end-to-end solutions using Databricks Intelligence Platform, Delta Lake, Apache Spark, Unity Catalog, Databricks Workflows, Delta Live Tables/Lakeflow-style pipelines, MLflow, Feature Store, Model Serving and Mosaic ...

intersection of consulting, technology, innovation, and strategic alliances and you will bring deep expertise in the Databricks Data Intelligence Platform, Lakehouse architecture, Delta Lake, Apache Spark, Unity Catalog, MLflow, Mosaic AI, data engineering, machine learning, GenAI and cloud-native delivery. As a key leader within our AI & Data … value pools, translate business needs into Databricks-led solution designs, build prototypes and proof points. • Shape end-to-end solutions using Databricks Intelligence Platform, Delta Lake, Apache Spark, Unity Catalog, Databricks Workflows, Delta Live Tables/Lakeflow-style pipelines, MLflow, Feature Store, Model Serving and Mosaic ...

growing data platform team. You'll play a key role in designing, building and optimising production-grade data pipelines across Databricks, Spark and Delta Lake, while helping shape the engineering standards and architecture behind the wider data platform. What you'll be doing: Build and optimise scalable Databricks …/Spark data pipelines across batch and streaming workloads. Develop Lakehouse solutions using Delta Lake and medallion architecture. Implement data quality, monitoring, governance and security across the platform. Work closely with Data Scientists, Analysts and Product teams to deliver high-quality data products. Contribute to engineering standards, code ...

build of large-scale data platforms using Azure Databricks and modern cloud technologies Implement and optimize ETL workflows and streaming pipelines with PySpark and Delta Live Tables following Lakehouse principles Enhance performance, manage cloud costs and ensure platform reliability for structured streaming workloads Define data governance, security and quality … optimization, data skew mitigation and query tuning Strong programming skills in Python with knowledge of modern software design principles Practical experience with structured streaming, Delta Lake and Delta Live Tables Proven experience in Lakehouse migration and modernization using open table formats such as Delta Lake ...

GitHub Actions and Terraform setup for deployments Implement, run and monitor dbt tests to ensure data quality and consistent dimensional models Extend and maintain Delta Lake fact and dimension tables for analytics use cases, following a bronze/silver/gold-style data architecture Work closely with analysts … they bring clear value, including integrating AI tools for analysis and AI coding agents into day-to-day workflows Tech stack Azure Databricks (Spark, Delta Lake) SQL Server, Confluent Kafka Scala, SQL, dbt Git, GitHub Actions, Terraform Sigma (BI) Must-have experience Commercial experience as a Data Engineer ...

delivery of advanced Databricks solutions within the Azure ecosystem. Responsibilities Design, build, and optimise end-to-end data pipelines using Azure Databricks, including Delta Live Tables. Collaborate with stakeholders to define technical requirements and propose Databricks-based solutions. Drive best practices for data engineering. Help clients realise the potential … work estimation. Skills & Experiences Proven experience designing and implementing data solutions in Azure using Databricks as a core platform. Hands-on expertise in Delta Lake, Delta Live Tables and Databricks Workflows. Strong coding skills in Python and SQL, with experience in developing modular, reusable code in Databricks. ...

organisation. Key Responsibilities: Design, build and maintain data pipelines from APIs, databases, SaaS applications, files and spreadsheets into Databricks. Develop scalable data models and Delta Lake architecture for reporting, analytics and future AI use cases. Manage Databricks Workflows, Lakeflow or Delta Live Tables, SQL Warehouses and related … where specialist support is required. Qualifications: Significant experience as a Data Engineer, including ownership of production data pipelines. Strong hands-on experience with Databricks, Delta Lake, Python, SQL and PySpark. Experience integrating data from APIs, business applications, databases and files. Experience with Unity Catalog, access management, data governance ...

Data Engineer and partnering with data science, analytics and product teams. Key responsibilities * Build and maintain scalable data pipelines using Databricks, Apache Spark and Delta Lake * Develop and optimise batch and streaming workflows for reliability and performance * Contribute to lakehouse architecture and medallion pattern implementation * Implement data quality … opportunities * Translate business requirements into technical solutions * Contribute to documentation and continuous improvement Required experience * Strong experience delivering production grade Databricks pipelines * Strong Spark, Delta Lake and Databricks Workflows experience * Python and SQL proficiency * Experience on Azure data platforms such as Azure Data Factory, ADLS Gen2, Azure DevOps ...

considered for future opportunities that match your expertisePrimary Responsibilities Client Engagement & Delivery Data Pipeline Development (Batch and Streaming) Databricks & Lakehouse Architectures Data Modelling & Optimisation (Delta Lake, Medallion architecture) Collaboration & Best Practices Quality, Governance & SecurityBusiness Relationships Solution Architects Data Engineers, Developers, ML Engineers, and Analysts Client stakeholders … mentoring teams of engineers to deliver high-quality scalable data solutions. Technical Expertise Deep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow). Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalent. Strong SQL and Python ...

maintain enterprise-grade ETL/ELT pipelines supporting ingestion, transformation, CDC, validation, and data delivery across multiple business functions. Develop and support Lakeflow pipelines, Delta Lake implementations, and modern data engineering frameworks. Drive best practices across performance optimisation, security, governance, monitoring, and platform reliability. Collaborate with business, data … development skills. Expertise building and supporting large-scale ETL/ELT pipelines and cloud-based data platforms. Strong understanding of Delta Lake, Unity Catalog, data governance principles, and medallion architecture. Experience integrating REST APIs, JSON data sources, Azure Data Factory, ADLS, Key Vault, and other Azure services. Hands ...

building scalable ingestion pipelines from APIs, relational databases and financial data providers into Azure Databricks Writing complex transformations in PySpark and Spark SQL, with Delta Lake as the storage foundation Tuning Databricks workloads for performance and cost, and embedding quality checks and validation into every pipeline Implementing governance … with Azure Databricks Strong Python and PySpark, backed by advanced SQL Deep grounding in data warehousing, data modelling and integration patterns Unity Catalog and Delta Lake in a production environment Data quality and governance experience, ideally including Azure Purview Git, Agile delivery and a collaborative approach to code ...

grow, belong and thrive.Your day at NTT DATAPrimary Responsibilities• Client Engagement & Delivery• Data Pipeline Development (Batch and Streaming)• Databricks & Lakehouse Architectures• Data Modelling & Optimisation (Delta Lake, Medallion architecture)• Collaboration & Best Practices• Quality, Governance & SecurityBusiness Relationships• Solution Architects• Data Engineers, Developers, ML Engineers, and Analysts• Client stakeholders … mentoring teams of engineers to deliver high-quality scalable data solutions.Technical Expertise• Deep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow).• Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalent.• Strong SQL and Python (or equivalent ...

belong and thrive. Your day at NTT DATAPrimary Responsibilities Client Engagement & Delivery Data Pipeline Development (Batch and Streaming) Databricks & Lakehouse Architectures Data Modelling & Optimisation (Delta Lake, Medallion architecture) Collaboration & Best Practices Quality, Governance & SecurityBusiness Relationships Solution Architects Data Engineers, Developers, ML Engineers, and Analysts Client stakeholders … mentoring teams of engineers to deliver high-quality scalable data solutions. Technical Expertise Deep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow). Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalent. Strong SQL and Python ...

thrive. Your day at NTT DATA Primary Responsibilities Client Engagement & Delivery Data Pipeline Development (Batch and Streaming) Databricks & Lakehouse Architectures Data Modelling & Optimisation (Delta Lake, Medallion architecture) Collaboration & Best Practices Quality, Governance & Security Business Relationships Solution Architects Data Engineers, Developers, ML Engineers, and Analysts Client stakeholders … mentoring teams of engineers to deliver high-quality scalable data solutions. Technical Expertise Deep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow). Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalent. Strong SQL and Python ...

thrive. Your day at NTT DATA Primary Responsibilities Client Engagement & Delivery Data Pipeline Development (Batch and Streaming) Databricks & Lakehouse Architectures Data Modelling & Optimisation (Delta Lake, Medallion architecture) Collaboration & Best Practices Quality, Governance & Security Business Relationships Solution Architects Data Engineers, Developers, ML Engineers, and Analysts Client stakeholders … mentoring teams of engineers to deliver high-quality scalable data solutions. Technical Expertise Deep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow). Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalent. Strong SQL and Python ...

enterprise environments 4+ years of hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform Strong expertise in Apache Spark, SQL, Python, Delta Lake, and modern data engineering practices Experience building and optimizing Extract, Transform, Load and Extract, Load, Transform data pipelines for scalable analytics … Platform. Design and implement scalable data and analytics solutions leveraging Databricks technologies Build, maintain, and optimize data pipelines using Apache Spark, SQL, Python, and Delta Lake Develop governed datasets and data products that support reporting, analytics, and artificial intelligence initiatives Partner with business stakeholders and technical teams ...

PySpark. Develop ETL/ELT and data transformation workflows using Python and SQL. Create and maintain Databricks notebooks, jobs, workflows and clusters. Work with Delta Lake and lakehouse architectures. Monitor, troubleshoot and optimise pipeline performance. Maintain high standards of data quality and reliability. Automate processes and create reusable … years’ Data Engineering experience. Strong hands‐on Databricks, PySpark, Python and SQL skills. Experience building and optimising ETL/ELT data pipelines. Knowledge of Delta Lake and data warehousing. Experience with at least one of AWS, Azure or GCP. Strong troubleshooting and problem-solving skills. A Computer Science ...

PySpark. Develop ETL/ELT and data transformation workflows using Python and SQL. Create and maintain Databricks notebooks, jobs, workflows and clusters. Work with Delta Lake and lakehouse architectures. Monitor, troubleshoot and optimise pipeline performance. Maintain high standards of data quality and reliability. Work across AWS, Azure … years Data Engineering experience. Strong hands-on Databricks, PySpark, Python and SQL skills. Experience building and optimising ETL/ELT data pipelines. Knowledge of Delta Lake and data warehousing. Experience with at least one of AWS, Azure or GCP. Strong troubleshooting and problem-solving skills. A Computer Science ...

build and enhance a secure, scalable and reliable Databricks platform Develop and maintain batch and real-time data pipelines using Databricks, Apache Spark and Delta Lake Contribute to platform architecture, engineering standards and best practices across the data estate Work with Product, Analytics and Data Science teams … across the engineering team Job Requirements: Significant experience building and supporting scalable, production-grade data platforms Strong hands-on expertise with Databricks, Apache Spark, Delta Lake, Python and SQL Good understanding of cloud-based data engineering, ideally within Microsoft Azure environments Experience with Infrastructure as Code tools such ...

build and enhance a secure, scalable and reliable Databricks platform Develop and maintain batch and real-time data pipelines using Databricks, Apache Spark and Delta Lake Contribute to platform architecture, engineering standards and best practices across the data estate Work with Product, Analytics and Data Science teams … across the engineering team Job Requirements: Significant experience building and supporting scalable, production-grade data platforms Strong hands-on expertise with Databricks, Apache Spark, Delta Lake, Python and SQL Good understanding of cloud-based data engineering, ideally within Microsoft Azure environments Experience with Infrastructure as Code tools such ...