1 to 25 of 26 Permanent Delta Lake Jobs in England

pipelines within a Databricks Lakehouse environment. You’ll work with both streaming and batch data pipelines using technologies such as Apache Spark Structured Streaming, Delta Live Tables, and Kafka, ensuring data flows efficiently and reliably across the organisation. Working closely with BI, product, and engineering teams, you will help … Experience building streaming and batch pipelines using Spark Structured Streaming Hands-on experience with Kafka (MSK) and real-time data ingestion Strong understanding of Delta Lake, Delta Live Tables, and Medallion Architecture Proficiency in Python and SQL for data engineering and analytics TO BE CONSIDERED... Please either ...

modelling for Finance and Enterprise data products, working closely with architecture where required. Implement and maintain data pipelines and ETL workflows in Databricks (PySpark, Delta Lake). Contribute to CI/CD pipelines for data applications using Azure DevOps and infrastructure-as-code (Terraform) in line with established … within the team and communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous). Experience contributing ...

with a genuine passion for software engineering in a data context. You'll work across the modern Azure data stack - ADF, ADLS Gen2, PySpark, Delta Lake - with increasing exposure to Microsoft Fabric as the platform matures. You'll collaborate closely with customers and internal teams to ensure data … streaming platforms (e.g. Event Hubs, Kafka), and legacy databases into ADLS Gen2 and OneLake Develop structured data models aligned to Lakehouse, Medallion Architecture, and Delta Lake patterns Continuously profile, debug, and optimise Spark jobs, SQL queries, and Python scripts for maximum performance and cost-efficiency at scale Champion ...

work at home 100% of the time. Your Role: Design, develop, and optimise large-scale cloud data platforms using Azure Data Factory, Data Lake, Key Vault, Azure Functions, Databricks, Delta Lake, PySpark, and Unity Catalog. Collaborate with underwriting, actuarial, Delegated Authority, bordereaux, exposure management, reinsurance, finance … continuous improvement. Your Skills: Expert-level hands-on experience with Azure Data Services and Databricks in enterprise, highly regulated environments. Deep understanding of Delta Lake, Medallion architecture, Lakehouse patterns, distributed compute, data modelling, and performance optimisation. Strong programming skills in Python, PySpark, and Spark SQL, with experience implementing ...

elevating engineering capability across engagements. What you'll do Lead the engineering, optimisation and governance of Azure-based data platforms, spanning ADF, Data Lake, Azure Functions, Key Vault, Databricks, Delta Lake, PySpark and Unity Catalog . Collaborate closely with insurance domain teams- Underwriting, Actuarial, Delegated Authority, Bordereaux … efficient operation. What you'll bring Expert-level hands-on experience with Azure data services and Databricks in enterprise-scale environments. Strong knowledge of Delta Lake , Medallion architecture, distributed compute and Lakehouse engineering patterns. Advanced Python, PySpark and Spark SQL skills, including deploying data workloads through Azure DevOps ...

hands-on role. What You’ll Be Doing: • Designing and building end-to-end data pipelines using Databricks (PySpark/SQL/Delta Lake) • Developing and managing robust, scalable Lakehouse architectures • Implementing Delta Live Tables (DLT) for reliable, testable, production-grade pipelines • Optimising performance using: • Working with … Platform teams to improve developer experience and platform usability Key Skills & Experience: • Strong experience building pipelines with PySpark/Spark SQL • Deep understanding of Delta Lake • Databricks Platform Expertise • Hands-on experience with: Unity Catalog (governance & security), Spark Declarative Pipelines, Databricks Workflows/Jobs • Strong understanding of Lakehouse ...

Responsibilities Design and implement data architectures on the Databricks Lakehouse Platform Build and optimize ETL/ELT pipelines using Apache Spark and Databricks Implement Delta Lake and medallion architecture (Bronze, Silver, Gold) Ensure data governance, security, and performance optimization Integrate Databricks with cloud platforms and enterprise data sources … deliver scalable data solutions Required Skills & Qualifications Databricks Certification (e.g., Databricks Certified Data Engineer Associate/Professional) Strong experience with Databricks, Apache Spark, and Delta Lake Proficiency in Python, SQL, and data pipeline development Experience with cloud platforms (Azure, AWS, or GCP) Solid understanding of modern data architecture ...

Databricks, including target architecture, data models, security, and governance frameworks. Analyze existing Foundry pipelines, datasets, and ontologies, and redesign them into scalable Databricks (Delta Lake) ETL/ELT workflows using PySpark and SQL. Implement validation frameworks to ensure complete data accuracy, consistency, and lineage throughout the migration process. … business stakeholders to minimize migration risk, ensure smooth cutover, and maintain production stability. Skills Must have Advanced Databricks Experience. Deep knowledge of Databricks architecture, Delta Lake, job orchestration, cluster management, and performance tuning. Proven Migration Experience (Palantir → Databricks). Demonstrated experience leading or executing platform migrations, including pipeline ...

energy market data using Python, SQL and Spark. Support data flows ensuring generation and storage assets deliver the correct contracted schedules. Design and manage Delta Lake tables and SQL databases, applying strong data modelling and optimisation strategies. Develop monitoring dashboards and alerting to ensure transparency around pipeline health … similar production-focused role Strong Python (Pandas) and SQL skills Experience building and maintaining data pipelines Hands-on experience with Spark, Databricks or Delta Lake/Lakehouse architectures Familiarity with cloud data platforms, particularly within Azure environments Good understanding of data modelling, storage formats and performance optimisation Experience ...

DSFS data model and existing Netezza structures. Produce the EDW logical and physical data models, dimensional models for the Semantic Layer, and Data Lake zone architecture (Bronze/Silver/Gold). Define Enterprise Semantic Layer design, including pre-computed aggregations, dimensional models, and Tableau connectivity specifications. Lead source … data architecture, with at least 3 years leading enterprise data model design. Experience designing Lakehouse architecture: medallion architecture (Bronze/Silver/Gold), Delta Lake, or equivalent. Proficiency in dimensional modelling (Kimball methodology) and enterprise data modelling. Experience with Databricks (Unity Catalog, Delta Lake) or Snowflake ...

reliability Collaborate with analysts and AI engineers to deliver robust and reusable data products Manage data lakes and warehouses using formats such as Delta Lake and Parquet Implement best practices for data governance, performance, and security Continuously evaluate and adopt new technologies to evolve the organisation's data … platform Provide technical guidance to junior engineers and contribute to team capability building Technical Stack Core: Azure Data Factory Azure Synapse Analytics Azure Data Lake Storage Gen2 SQL Server Databricks Enhancements: Python (PySpark, Pandas) CI/CD (Azure DevOps) Infrastructure as Code (Terraform, Bicep) REST APIs GitHub Actions Desirable ...

reliability Collaborate with analysts and AI engineers to deliver robust and reusable data products Manage data lakes and warehouses using formats such as Delta Lake and Parquet Implement best practices for data governance, performance, and security Continuously evaluate and adopt new technologies to evolve the organisation’s data … platform Provide technical guidance to junior engineers and contribute to team capability building Technical Stack Core: Azure Data Factory Azure Synapse Analytics Azure Data Lake Storage Gen2 SQL Server Databricks Enhancements: Python (PySpark, Pandas) CI/CD (Azure DevOps) Infrastructure as Code (Terraform, Bicep) REST APIs GitHub Actions Desirable ...

Kimball, Data Vault). Experience exposing Lakehouse architectures to tools such as Power BI and Tableau. Hands-on experience in Databricks engineering: Spark, Delta Lake, PySpark. Skilled in building scalable ETL pipelines using Workflows and Delta Live Tables. Proficiency in Python, Scala, or SQL. Knowledge of Databricks ...

Kimball, Data Vault). Experience exposing Lakehouse architectures to tools such as Power BI and Tableau. Hands-on experience in Databricks engineering: Spark, Delta Lake, PySpark. Skilled in building scalable ETL pipelines using Workflows and Delta Live Tables. Proficiency in Python, Scala, or SQL. Knowledge of Databricks ...

Proven experience delivering enterprise-scale Databricks solutions end-to-end.* Strong background in Lakehouse Architecture, including structured and unstructured data.* Expertise in Spark, PySpark, Delta Lake, and Databricks workflows.* Experience building scalable ETL/ELT pipelines, including Delta Live Tables.* Strong programming skills in Python, Scala ...

transformation Hybrid working with flexibility and autonomy Key Experience Deep end-to-end experience delivering enterprise Databricks analytics solutions Expertise in Lakehouse Architecture, Delta Lake, Spark, PySpark Strong data modelling skills (3NF, Kimball, Data Vault) Experience exposing Delta Lakehouse to Power BI or Tableau Skilled in Python … Scala or SQL for data engineering Experience building scalable ETL pipelines (Workflows, Delta Live Tables) Hands-on experience with Azure, AWS or GCP (storage, compute, networking) Knowledge of Databricks security: RBAC, IAM, encryption Excellent analytical skills and the confidence to challenge thinking at all levels Bonus: Databricks certifications (Data ...

journeys. What You'll Bring Extensive experience designing and delivering end-to-end data platforms using Databricks , with a strong focus on Lakehouse Architecture , Delta Lake, and scalable data engineering patterns. A solid background in data analytics, consulting, or a similar technical discipline. Strong knowledge of structured … Kimball , and Data Vault . Hands-on experience exposing Lakehouse solutions to BI tools (e.g. Power BI , Tableau ). Proficiency in Databricks Workflows , Delta Live Tables , and Spark for batch and streaming pipelines. Strong programming capability in Python , Scala , and/or SQL . Practical experience with at least ...

including solution shaping, cost estimation, effort sizing, and client presentations Support technical deal qualification and proposal development Deliver scalable data pipelines using Databricks, Spark, Delta Lake, and PySpark Own end-to-end architecture decisions across delivery programmes Work closely with consulting, commercial, and engineering teams Required Experience Strong ...

e.g., star schema, slowly changing dimensions), with the ability to guide others. Hands-on experience with Microsoft Fabric or Azure Databricks, including exposure to Delta Lake and lakehouse architectures. Familiarity with CI/CD practices and Git-based version control. Nice to haves: Prior experience in Asset Management ...

industries A supportive, down-to-earth team What they're looking for: Solid experience with Databricks (Azure or AWS) Good understanding of tools like Delta Lake, Unity Catalog, MLflow Strong SQL and Spark (Python or Scala) Experience working with data pipelines and modern data platforms A good grasp ...

across analytical and operational datasets. Proven experience delivering complex end-to-end data platforms in Azure or Fabric environments. Strong understanding of medallion architectures, Delta Lake, distributed compute and modern data engineering principles. Experience defining data engineering standards, governance and security frameworks. Ability to engage confidently with business ...

streamlined data analysis, product builds and reduced time to market Competencies and Behaviours 1 -3 years analytics/data science experience Practical knowledge of; Delta Lake architecture and versioned datasets, Data pipelines, orchestration and scheduling concepts Proficiency in analytical programming language such as python and/ ...

Modelling, Data Warehousing. Strong background in incident resolution, requests, changes and problem-solving within SLAs. Hands-on experience with Spark, Data Architecture, SQL and Delta Lake. Cloud development experience (AWS, GCP or Azure). Working knowledge of Medallion Architecture. Demonstrated capability in implementing and supporting pipelines in demanding environments. ...

experience with Microsoft Purview • Strong Azure Data Platform experience: ADLS Gen2, Azure Databricks, Data Factory• Hands-on data engineering experience (Spark/PySpark, Delta Lake)• Knowledge of security and compliance controls, GDPR, PII/DLP, RBAC/ABAC• Experience leading Agile teams and managing risks, dependencies, and senior ...

based development workflows Experience delivering features end-to-end in distributed teams Confident troubleshooting production issues in real time Desirable Skills Apache Spark (EMR) & Delta Lake PostgreSQL (Aurora) CI/CD using GitHub Actions IaC (Helm, CloudFormation) Monitoring tools (Sentry) Role Responsibilities You'll be hands ...