1 to 25 of 115 Delta Lake Jobs in the UK

available The Role As a Data Engineer, you will focus on designing, developing, and optimising data ingestion, transformation, and serving layers across Azure Databricks, Delta Lake, and Databricks SQL. Your work will enable reliable data products and insights for stakeholders, and you\'ll collaborate with data architects, analysts … Build and maintain metadata-driven pipelines using Azure Databricks to ingest and transform batch and streaming data, ensuring data quality and scalable processing. Develop Delta Lake layers following bronze/silver/gold patterns, and model data optimised for BI and analytics, including Power BI integration through Databricks ...

scalable data pipelines using Dataflow Gen2 and Fabric Notebooks • Develop and maintain Lakehouse solutions using Medallion architecture • Work with SQL, Python, Spark SQL and Delta Lake • Integrate data from APIs, databases, CRM, ERP and third party platforms • Support both batch and real time data processing • Develop semantic models … Notebooks and Lakehouse architecture • Strong SQL and Python skills • Experience with Spark and Spark SQL • Proven experience designing scalable ETL and ELT pipelines • Data Lake and Delta Lake experience • Experience integrating data from multiple structured and unstructured sources • Good understanding of CI/CD, DevOps and modern ...

develop and manage STACK's enterprise data platform. This is a production-focused engineering role requiring practical expertise in Databricks, PySpark, Python, SQL and Delta Lake. The role will design reliable pipelines integrating Finance, Delivery, Operations, HR and PMO systems, establish governed and high-quality datasets, and support secure … production-grade ETL/ELT pipelines using Databricks, PySpark, Python and SQL, integrating APIs, databases, SaaS applications, files and spreadsheets. Develop and optimize scalable Delta Lake data models and lakehouse architecture for reporting, analytics and future AI use cases. Build, schedule, monitor and troubleshoot data processing jobs using ...

helping clients make appropriate architecture and technology decisions. Databricks & Modern Data Platforms Design enterprise Databricks and Lakehouse solutions incorporating technologies and patterns such as Delta Lake, medallion architecture, Unity Catalog and Databricks SQL. Apply strong knowledge of modern cloud data ecosystems, including Azure and/or AWS, with … delivering enterprise cloud data architectures within complex client or business environments. Deep expertise in Databricks and modern Lakehouse architecture, including practical knowledge of Delta Lake and Unity Catalog. Strong experience with Microsoft Azure and/or AWS and modern enterprise data platforms, including Databricks, Microsoft Fabric or another ...

GitHub Actions and Terraform setup for deployments Implement, run and monitor dbt tests to ensure data quality and consistent dimensional models Extend and maintain Delta Lake fact and dimension tables for analytics use cases, following a bronze/silver/gold-style data architecture Work closely with analysts … they bring clear value, including integrating AI tools for analysis and AI coding agents into day-to-day workflows Tech stack Azure Databricks (Spark, Delta Lake) SQL Server, Confluent Kafka Scala, SQL, dbt Git, GitHub Actions, Terraform Sigma (BI) Must-have experience Commercial experience as a Data Engineer ...

data from internal systems and third-party platforms including Salesforce, Sopra Banking Services, CACI and Experian. Engineer Modern Data Architectures - Design Lakehouse, Warehouse and Delta Lake solutions using medallion architectures to support enterprise analytics and reporting. Deliver Analytics-Ready Data Assets - Build trusted, well-modelled datasets and semantic … successfully built, enhanced or operated cloud‐based data platforms, ideally within Azure environments. Data Architecture Knowledge - You understand data modelling, ELT/ETL design, Delta Lake principles, data quality frameworks and modern engineering best practices. Cloud & DevOps Capability - You have experience with Azure services, automation, CI/ ...

enterprise environments 4+ years of hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform Strong expertise in Apache Spark, SQL, Python, Delta Lake, and modern data engineering practices Experience building and optimizing Extract, Transform, Load and Extract, Load, Transform data pipelines for scalable analytics … Platform. Design and implement scalable data and analytics solutions leveraging Databricks technologies Build, maintain, and optimize data pipelines using Apache Spark, SQL, Python, and Delta Lake Develop governed datasets and data products that support reporting, analytics, and artificial intelligence initiatives Partner with business stakeholders and technical teams ...

thrive. Your day at NTT DATA Primary Responsibilities Client Engagement & Delivery Data Pipeline Development (Batch and Streaming) Databricks & Lakehouse Architectures Data Modelling & Optimisation (Delta Lake, Medallion architecture) Collaboration & Best Practices Quality, Governance & Security Business Relationships Solution Architects Data Engineers, Developers, ML Engineers, and Analysts Client stakeholders … mentoring teams of engineers to deliver high-quality scalable data solutions. Technical Expertise Deep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow). Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalent. Strong SQL and Python ...

thrive. Your day at NTT DATA Primary Responsibilities Client Engagement & Delivery Data Pipeline Development (Batch and Streaming) Databricks & Lakehouse Architectures Data Modelling & Optimisation (Delta Lake, Medallion architecture) Collaboration & Best Practices Quality, Governance & Security Business Relationships Solution Architects Data Engineers, Developers, ML Engineers, and Analysts Client stakeholders … mentoring teams of engineers to deliver high-quality scalable data solutions. Technical Expertise Deep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow). Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalent. Strong SQL and Python ...

build and enhance a secure, scalable and reliable Databricks platform Develop and maintain batch and real-time data pipelines using Databricks, Apache Spark and Delta Lake Contribute to platform architecture, engineering standards and best practices across the data estate Work with Product, Analytics and Data Science teams … across the engineering team Job Requirements: Significant experience building and supporting scalable, production-grade data platforms Strong hands-on expertise with Databricks, Apache Spark, Delta Lake, Python and SQL Good understanding of cloud-based data engineering, ideally within Microsoft Azure environments Experience with Infrastructure as Code tools such ...

frameworks and model benchmarking methodologiesAdvanced programming skills in Python and SQL with strong knowledge of data modeling and ETL/ELT processesSolid understanding of Delta Lake, Delta Live Tables, Unity Catalog, Spark Structured Streaming and large-scale data processingExperience with Azure Data Factory (ADF), Qlik Replicate ...

implement scalable Databricks Lakehouse platforms on AWS and/or Azure aligned to client requirements Architect end-to-end data platforms including ingestion, storage (Delta Lake), processing, and consumption layers Build and configure cloud infrastructure using infrastructure-as-code (e.g. Terraform & Declarative Automation Bundles(DAB's)) Establish secure … pipelines, and release processes for Databricks deployments Implement monitoring, logging, cost optimisation, and performance tuning across the platform Design and implement data pipelines using Delta Live Tables, Auto Loader, and Databricks Workflows for both batch and streaming workloads Client Delivery & Enablement responsibilities: Work directly with clients to translate business ...

practices, and guiding engineering teams to deliver scalable, secure, and cost optimized data platforms. \* Design end to end data platforms leveraging Databricks Lakehouse architecture, Delta Lake, Unity Catalog, and ML/AI workloads. \* Define data ingestion, transformation, governance, and consumption frameworks aligned to enterprise standards. \* Evaluate and recommend … emerging technologies. \* Develop reference architectures, solution blueprints, and best-practice playbooks for Databricks implementations. \* Architect scalable ETL/ELT pipelines using PySpark, Spark SQL, Delta Live Tables, and Databricks Workflows. \* Implement performance optimization strategies for notebooks, clusters, job orchestration, and data storage. \* Guide teams in building curated, governed data ...

supporting colleagues. What you'll do ·Build and optimise pipelines using Azure Databricks, PySpark, SQL, ADF and ADLS Gen2. ·Develop governed lakehouse solutions using Delta Lake, medallion architecture and Unity Catalog. ·Own Power BI semantic models, DAX measures and row-level security, ensuring reporting is accurate, secure …/CD. ·Mentor colleagues and share good practice across the team. What you'll bring ·Strong production experience with Databricks, PySpark, Delta Lake and Unity Catalog. ·Hands-on Power BI expertise, including data modelling, DAX, RLS and performance optimisation. ·Strong SQL skills and experience with Azure data services. ...

shared data assets Technical Leadership & Standards Serve as a senior technical authority for data product architecture across the program Define architectural best practices for Delta Lake design, metadata management, product lifecycle governance, and data quality controls Provide architecture assurance and design reviews for data products developed by internal … data contracts, service models, and consumer engagement frameworks Excellent stakeholder management, communication, and consulting skills Technical Skills Deep expertise in Databricks, including Lakehouse Architecture, Delta Lake, Unity Catalog, and Databricks SQL Strong understanding of data modelling, schema design, metadata management, and lineage tracking Experience implementing self-service data ...

Required Skills & Experience 3+ years of experience within Data Engineering, Analytics Engineering, or Data Analytics. Strong hands-on experience with: Databricks SQL Python PySpark Delta Lake Experience designing ETL/ELT pipelines and data transformation frameworks. Strong understanding of data modelling, performance optimisation, and data quality practices. Experience … cutting-edge AI-assisted development practices. Challenging, business-critical projects with significant impact. Collaborative and innovative working environment. Keywords: Databricks, Python, SQL, PySpark, Delta Lake, Data Engineer, Analytics Engineer, Data Analytics, ETL, ELT, AI Engineer, Generative AI, GitHub Copilot, ChatGPT, Claude Code, Cursor, Prompt Engineering, Context Engineering, Data ...

managing and reporting control issues with transparency. Skills & Experience Essential Proven in-depth commercial experience in the following key areas: Databricks – Lakehouse architecture, Spark, Delta Lake, workflows and performance tuning. Apache Spark & Scala – High-performance distributed data processing and optimisation. Cloud Data Engineering (AWS) – Cloud-native data platforms … core business functions Demonstrated leadership, project management, and development skills Relationship and consensus building skills Preferred Kafka/Event-Driven Architecture Apache Iceberg/Delta Lake Python AI/ML & LLM integration Financial Services/Capital Markets Education Bachelor’s degree/University degree or equivalent experience Master ...

risks, exceptions, and technical dependencies raised through governance processes Enterprise & Solution Architecture Provide architecture assurance for Databricks Lakehouse implementations, including data ingestion, medallion architecture, Delta Lake, Unity Catalog, and compute strategies Validate integration patterns across Databricks, cloud platforms, enterprise applications, and data ecosystems Define and maintain reference architectures … delivery requirements Exceptional stakeholder engagement and influencing skills across technical and business audiences Technical Skills Strong hands-on knowledge of Databricks including Lakehouse Architecture, Delta Lake, Unity Catalog, and platform governance Experience with cloud platforms supporting Databricks environments, including AWS and/or Azure Solid understanding of enterprise ...

build and optimise data pipelines across Pricing, Underwriting and Claims. You’ll work on a modern Azure-based data platform, consolidating capabilities on Databricks, Delta Lake and Unity Catalog to support analytics and AI initiatives. You’ll collaborate with Data Architects and Analysts, delivering scalable, secure data solutions ...

Platform Engineering team, you'll own data pipelines from ingestion through transformation, governance and consumption. You'll work with technologies including Databricks, Spark and Delta Lake to make complex data reliable, secure and ready for use across OQC's customer-facing applications. You’ll collaborate across Software, Platform … across engineering teams to deliver reliable data foundations for customer-facing products. What We're Looking For Strong hands-on experience with Databricks, Spark, Delta Lake and SQL. Experience building and operating streaming and batch data pipelines. Strong understanding of data quality, lineage, access control and dataset governance. ...

Must Have: At least 7 years as an Engineer in a Data Field Strong Azure data skills, including: Azure Data Factory V2 Azure Data Lake Storage V2 Azure Databricks Azure Function Apps& Logic Apps Azure Stream Analytics Azure Resource Manager skills (Terraform, Azure Portal, Az CLI and Az PowerShell … Strong PySpark, Delta Lake, Unity Catalog and Python skills. Includes ability to write unit and integration tests in Python with unittest, pytest, etc. Strong understanding of software development practices, such as SOLID principals, structuring code, testing, IOC, etc. Strong repositories, CI/CD skills. Strong knowledge of Kimball ...

extensive experience with workflow orchestration tools (e.g., Apache Airflow, Dagster). Hands-on expertise with cloud data warehouses (e.g., Snowflake, BigQuery, Redshift) and data lake architectures (e.g., Databricks, Delta Lake, Apache Iceberg). Advanced SQL skills and proficiency in Python. Strong background in modern software development practices ...

scalable, maintainable, and aligned with organisational standards and governance requirements. Design, build, and optimise data pipelines on Databricks using PySpark and Spark SQL, applying Delta Lake and a medallion/Lakehouse architecture. Implement data validation and reconciliation checks, and automated testing across pipelines to ensure trusted, accurate outputs. ...

systems design within a modern cloud‐based environment Advanced experience working and understanding the tradeoffs of at least one of the following Data Lake table/file formats: Delta Lake, Parquet, Iceberg, Hudi Experience working with lakehouse/medallion architectures in Databricks Experience with experimentation support tooling ...

cloud platforms (AWS, Azure, or GCP) including Snowflake deployment, networking (Private Link), and cloud storage integration (S3, ADLS, GCS); familiarity with lakehouse stacks (Databricks, Delta Lake, Apache Iceberg) is a plus.Mastery of Snowflake security and governance: RBAC, tag-based governance, dynamic data masking, row access policies, column-level ...