1 to 25 of 111 Permanent Delta Lake Jobs in England

/CD, Git and automated deployment Monitoring, logging and operational alerting Data-quality and reconciliation controls API/file/SFTP integration patterns Delta Lake/Delta MERGE Databricks Asset Bundles Azure DevOps MLOps/model deployment experience SFMC or Comarch integration Customer decisioning/personalisation platforms … Nice To Have Delta Lake/Delta MERGE Databricks Asset Bundles Azure DevOps MLOps/model deployment experience SFMC or Comarch integration Customer decisioning/personalisation platforms MLOps & Integration Data Engineer Role Overview BNA Production-Readiness, MLOps & Integration Engineer: Automated end-to-end BNA orchestration Reliable retry ...

scalable data pipelines using Dataflow Gen2 and Fabric Notebooks • Develop and maintain Lakehouse solutions using Medallion architecture • Work with SQL, Python, Spark SQL and Delta Lake • Integrate data from APIs, databases, CRM, ERP and third party platforms • Support both batch and real time data processing • Develop semantic models … Notebooks and Lakehouse architecture • Strong SQL and Python skills • Experience with Spark and Spark SQL • Proven experience designing scalable ETL and ELT pipelines • Data Lake and Delta Lake experience • Experience integrating data from multiple structured and unstructured sources • Good understanding of CI/CD, DevOps and modern ...

raise risks early As part of your duties, you will be responsible for: Architecture and modelling Design the lakehouse architecture on Azure Databricks using Delta Lake and the Medallion pattern (bronze, silver and gold layers), organising data by quality and purpose to keep the platform scalable and maintainable. … reporting and self‐service analytics. Select and design for the right storage and exchange formats for each workload, including row‐based and columnar storage, Delta and Iceberg table formats, and file and message formats (e.g. Parquet, CSV, Excel, XML, JSON, Protobuf). Optimise platform performance and cost, including partitioning ...

develop and manage STACK's enterprise data platform. This is a production-focused engineering role requiring practical expertise in Databricks, PySpark, Python, SQL and Delta Lake. The role will design reliable pipelines integrating Finance, Delivery, Operations, HR and PMO systems, establish governed and high-quality datasets, and support secure … production-grade ETL/ELT pipelines using Databricks, PySpark, Python and SQL, integrating APIs, databases, SaaS applications, files and spreadsheets. Develop and optimize scalable Delta Lake data models and lakehouse architecture for reporting, analytics and future AI use cases. Build, schedule, monitor and troubleshoot data processing jobs using ...

helping clients make appropriate architecture and technology decisions. Databricks & Modern Data Platforms Design enterprise Databricks and Lakehouse solutions incorporating technologies and patterns such as Delta Lake, medallion architecture, Unity Catalog and Databricks SQL. Apply strong knowledge of modern cloud data ecosystems, including Azure and/or AWS, with … delivering enterprise cloud data architectures within complex client or business environments. Deep expertise in Databricks and modern Lakehouse architecture, including practical knowledge of Delta Lake and Unity Catalog. Strong experience with Microsoft Azure and/or AWS and modern enterprise data platforms, including Databricks, Microsoft Fabric or another ...

GitHub Actions and Terraform setup for deployments Implement, run and monitor dbt tests to ensure data quality and consistent dimensional models Extend and maintain Delta Lake fact and dimension tables for analytics use cases, following a bronze/silver/gold-style data architecture Work closely with analysts … they bring clear value, including integrating AI tools for analysis and AI coding agents into day-to-day workflows Tech stack Azure Databricks (Spark, Delta Lake) SQL Server, Confluent Kafka Scala, SQL, dbt Git, GitHub Actions, Terraform Sigma (BI) Must-have experience Commercial experience as a Data Engineer ...

data from internal systems and third-party platforms including Salesforce, Sopra Banking Services, CACI and Experian. Engineer Modern Data Architectures - Design Lakehouse, Warehouse and Delta Lake solutions using medallion architectures to support enterprise analytics and reporting. Deliver Analytics-Ready Data Assets - Build trusted, well-modelled datasets and semantic … successfully built, enhanced or operated cloud‐based data platforms, ideally within Azure environments. Data Architecture Knowledge - You understand data modelling, ELT/ETL design, Delta Lake principles, data quality frameworks and modern engineering best practices. Cloud & DevOps Capability - You have experience with Azure services, automation, CI/ ...

enterprise environments 4+ years of hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform Strong expertise in Apache Spark, SQL, Python, Delta Lake, and modern data engineering practices Experience building and optimizing Extract, Transform, Load and Extract, Load, Transform data pipelines for scalable analytics … Platform. Design and implement scalable data and analytics solutions leveraging Databricks technologies Build, maintain, and optimize data pipelines using Apache Spark, SQL, Python, and Delta Lake Develop governed datasets and data products that support reporting, analytics, and artificial intelligence initiatives Partner with business stakeholders and technical teams ...

frameworks and model benchmarking methodologiesAdvanced programming skills in Python and SQL with strong knowledge of data modeling and ETL/ELT processesSolid understanding of Delta Lake, Delta Live Tables, Unity Catalog, Spark Structured Streaming and large-scale data processingExperience with Azure Data Factory (ADF), Qlik Replicate ...

implement scalable Databricks Lakehouse platforms on AWS and/or Azure aligned to client requirements Architect end-to-end data platforms including ingestion, storage (Delta Lake), processing, and consumption layers Build and configure cloud infrastructure using infrastructure-as-code (e.g. Terraform & Declarative Automation Bundles(DAB's)) Establish secure … pipelines, and release processes for Databricks deployments Implement monitoring, logging, cost optimisation, and performance tuning across the platform Design and implement data pipelines using Delta Live Tables, Auto Loader, and Databricks Workflows for both batch and streaming workloads Client Delivery & Enablement responsibilities: Work directly with clients to translate business ...

doing:·Design, develop and optimise data pipelines using Azure Databricks, PySpark, SQL, ADF and ADLS Gen2.·Build and maintain governed lakehouse solutions using Delta Lake, medallion architecture and Unity Catalog.·Develop and enhance Power BI semantic models, DAX measures and row-level security.·Work with business stakeholders …/CD.·Share technical knowledge, mentor colleagues and contribute to engineering standards.What we're looking for:·Proven production experience with Azure Databricks, PySpark and Delta Lake, including hands-on Unity Catalog governance, access controls and permissions management.·Strong Power BI development skills, including semantic modelling ...

supporting colleagues. What you'll do ·Build and optimise pipelines using Azure Databricks, PySpark, SQL, ADF and ADLS Gen2. ·Develop governed lakehouse solutions using Delta Lake, medallion architecture and Unity Catalog. ·Own Power BI semantic models, DAX measures and row-level security, ensuring reporting is accurate, secure …/CD. ·Mentor colleagues and share good practice across the team. What you'll bring ·Strong production experience with Databricks, PySpark, Delta Lake and Unity Catalog. ·Hands-on Power BI expertise, including data modelling, DAX, RLS and performance optimisation. ·Strong SQL skills and experience with Azure data services. ...

Required Skills & Experience 3+ years of experience within Data Engineering, Analytics Engineering, or Data Analytics. Strong hands-on experience with: Databricks SQL Python PySpark Delta Lake Experience designing ETL/ELT pipelines and data transformation frameworks. Strong understanding of data modelling, performance optimisation, and data quality practices. Experience … cutting-edge AI-assisted development practices. Challenging, business-critical projects with significant impact. Collaborative and innovative working environment. Keywords: Databricks, Python, SQL, PySpark, Delta Lake, Data Engineer, Analytics Engineer, Data Analytics, ETL, ELT, AI Engineer, Generative AI, GitHub Copilot, ChatGPT, Claude Code, Cursor, Prompt Engineering, Context Engineering, Data ...

schema integrity. Key Responsibilities Source-to-Target Reconciliation: Design, build, and execute automated validation queries to compare legacy eFront extracts against target Databricks Delta Lake tables (Bronze, Silver, Gold). Business Logic & Metrics Testing: Verify complex private equity calculation logic (cash flows, commitments, valuations, NAV, and return metrics … Engineer, Data Quality Engineer, or Data Test Automation Engineer on enterprise cloud migrations. Hands-on testing and reconciliation experience within Azure Databricks/Delta Lake environments. Familiarity with private markets, fund accounting, or private equity financial structures (NAV, cash flows, capital calls, distributions). #J-18808-Ljbffr ...

risks, exceptions, and technical dependencies raised through governance processes Enterprise & Solution Architecture Provide architecture assurance for Databricks Lakehouse implementations, including data ingestion, medallion architecture, Delta Lake, Unity Catalog, and compute strategies Validate integration patterns across Databricks, cloud platforms, enterprise applications, and data ecosystems Define and maintain reference architectures … delivery requirements Exceptional stakeholder engagement and influencing skills across technical and business audiences Technical Skills Strong hands-on knowledge of Databricks including Lakehouse Architecture, Delta Lake, Unity Catalog, and platform governance Experience with cloud platforms supporting Databricks environments, including AWS and/or Azure Solid understanding of enterprise ...

scope to drive data democratisation, governance maturity and AI-readiness across the firm.## **What you will be doing** * Design and maintain scalable, secure data lake and lakehouse solutions using Azure Data Lake, Databricks and Delta Lake — built to support real-time insight, advanced analytics … validation, logging and monitoring to catch issues early and keep the platform reliable. **Key Requirements** * Experience in data engineering, including designing and delivering data lake or lakehouse architectures in an enterprise cloud environment, ideally Microsoft Azure.* Hands-on expertise with Databricks, Spark, PySpark, SQL and core Azure Data Services ...

deployment Worked across monitoring, logging and operational alerting Completed data-quality and reconciliation controls API/file/SFTP integration patterns Nice to have: Delta Lake/Delta MERGE Databricks Asset Bundles Azure DevOps MLOps/model deployment work history SFMC or Comarch integration Customer decisioning/ ...

Must Have: At least 7 years as an Engineer in a Data Field Strong Azure data skills, including: Azure Data Factory V2 Azure Data Lake Storage V2 Azure Databricks Azure Function Apps& Logic Apps Azure Stream Analytics Azure Resource Manager skills (Terraform, Azure Portal, Az CLI and Az PowerShell … Strong PySpark, Delta Lake, Unity Catalog and Python skills. Includes ability to write unit and integration tests in Python with unittest, pytest, etc. Strong understanding of software development practices, such as SOLID principals, structuring code, testing, IOC, etc. Strong repositories, CI/CD skills. Strong knowledge of Kimball ...

extensive experience with workflow orchestration tools (e.g., Apache Airflow, Dagster). Hands-on expertise with cloud data warehouses (e.g., Snowflake, BigQuery, Redshift) and data lake architectures (e.g., Databricks, Delta Lake, Apache Iceberg). Advanced SQL skills and proficiency in Python. Strong background in modern software development practices ...

other teams can pick up and run, rather than as documents. Azure-First Data Platforms: Deep architecture and build experience on Microsoft Azure (Data Lake/Lakehouse patterns, ADLS Gen2, Synapse Analytics, Azure Databricks, Delta Lake), with enough hands-on AWS or Google Cloud Platform (GCP) experience … Lead Azure-First Platform Architecture: Architect and implement Azure data platforms for clients, including landing zones, networking, identity, and data platform services (Azure Data Lake Storage, Synapse, Databricks), extending patterns to AWS or GCP where a client estate requires it. Design & Build Data Pipelines: Design, Implement key data ingress ...

scalable, maintainable, and aligned with organisational standards and governance requirements. Design, build, and optimise data pipelines on Databricks using PySpark and Spark SQL, applying Delta Lake and a medallion/Lakehouse architecture. Implement data validation and reconciliation checks, and automated testing across pipelines to ensure trusted, accurate outputs. ...

Unifeye. What You'll Do Design and build scalable batch and streaming data pipelines using Spark and Databricks. Develop modern Lakehouse architectures using Delta Lake and Medallion (Bronze, Silver and Gold) design principles. Build robust ingestion frameworks from APIs, files and streaming data sources. Ensure data quality, testing … data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. Good understanding of Lakehouse architecture and Delta Lake. Experience working with AWS, Azure or GCP. Experience implementing CI/CD, version control and deployment automation. Strong communication skills and experience working ...

Unifeye. What You'll Do Design and build scalable batch and streaming data pipelines using Spark and Databricks. Develop modern Lakehouse architectures using Delta Lake and Medallion (Bronze, Silver and Gold) design principles. Build robust ingestion frameworks from APIs, files and streaming data sources. Ensure data quality, testing … data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. Good understanding of Lakehouse architecture and Delta Lake. Experience working with AWS, Azure or GCP. Experience implementing CI/CD, version control and deployment automation. Strong communication skills and experience working ...

such as Kafka or Kinesis* (Desirable) Experience with Airflow or other workflow orchestration tools* (Desirable) Experience with Docker and Kubernetes* (Desirable) Exposure to Data Lake and Lakehouse architecture* (Desirable) Experience in Agile/Scrum development environments* (Desirable) Experience with Apache Kafka, Apache Airflow, Terraform, Databricks, Delta Lake ...

Snowflake and cloud-native data services Data architecture:Lakehouse, medallion, datameshand domain-oriented patterns Data processing:Apache Spark,SQLand distributed processing technologies Data storage:Delta Lake, ApacheIcebergand open table formats Data pipelines:Enterprise-scale ETL/ELT,batchand streaming patterns Transformation and quality:dbt, Great Expectations and equivalent ...