24 of 24 Remote/Hybrid Delta Lake Jobs in England

considering: We are looking for an experienced AWS Databricks Engineer with strong hands-on expertise in Databricks, AWS cloud services, PySpark, Spark SQL, Delta Lake, Python, SQL, and data engineering. The candidate will be responsible for designing, developing, optimizing, and supporting scalable data pipelines and lakehouse solutions … Your Role: Design, develop, and maintain scalable data pipelines using Databricks on AWS. Build and optimize data processing frameworks using PySpark, Spark SQL, Python, Delta Lake, and Databricks notebooks. Develop end-to-end ETL/ELT pipelines for ingestion, transformation, validation, and consumption. Work with AWS services such ...

considering: We are looking for an experienced Data Architect with strong expertise in Azure Databricks, Azure Data Factory, Azure Data Lake, Delta Lake, data modelling, data governance, and cloud data architecture, along with domain experience in London Market Insurance. The candidate will be responsible for defining … time. Your Role: Define end-to-end data architecture for Azure-based data platforms using Azure Databricks, Azure Data Factory, ADLS Gen2, Delta Lake, Azure Synapse, and Power BI. Design scalable and secure lakehouse architecture using bronze, silver, and gold data layers. Lead architecture and design for data ...

OneLake and Databricks Unity Catalog. Build optimised Lakehouse and warehouse structures using Databricks SQL and Fabric SQL endpoints. Establish data pipelines using Databricks Workflows, Delta Live Tables (DLT), and Fabric Data Factory. Formulate cross‐platform data strategies utilising shared Delta Lake formats to eliminate data duplication. Implement … scalable lakehouse and warehouse solutions (Databricks SQL, Fabric SQL). Strong data pipeline development using Databricks Workflows, DLT, and Fabric Data Factory. Knowledge of Delta Lake and cross‐platform data strategies to reduce duplication. Data governance expertise, including Unity Catalog, RLS, and data lineage. Experience managing compute across ...

solutions. Key Responsibilities Design, develop, and maintain scalable data pipelines using Python and Databricks Build and optimise ETL/ELT workflows leveraging Databricks (Spark, Delta Lake) Develop robust data models and data architectures on Lakehouse platforms Implement and manage data workflows within the Databricks ecosystem Ensure high data … drive engineering best practices Skills & Experience Extensive experience in data engineering Strong programming expertise in Python Hands-on experience with Databricks (workflows, notebooks, jobs, Delta Lake) Strong experience with Apache Spark/PySpark Experience building scalable ETL pipelines Proficiency in SQL and relational databases Experience with cloud platforms ...

solutions. Key Responsibilities Design, develop, and maintain scalable data pipelines using Python and Databricks Build and optimise ETL/ELT workflows leveraging Databricks (Spark, Delta Lake) Develop robust data models and data architectures on Lakehouse platforms Implement and manage data workflows within the Databricks ecosystem Ensure high data … drive engineering best practices Skills & Experience Extensive experience in data engineering Strong programming expertise in Python Hands-on experience with Databricks (workflows, notebooks, jobs, Delta Lake) Strong experience with Apache Spark/PySpark Experience building scalable ETL pipelines Proficiency in SQL and relational databases Experience with cloud platforms ...

pipelines within a Databricks Lakehouse environment. You'll work with both streaming and batch data pipelines using technologies such as Apache Spark Structured Streaming, Delta Live Tables, and Kafka, ensuring data flows efficiently and reliably across the organisation. Working closely with BI, product, and engineering teams, you will help … Experience building streaming and batch pipelines using Spark Structured Streaming Hands-on experience with Kafka (MSK) and real-time data ingestion Strong understanding of Delta Lake, Delta Live Tables, and Medallion Architecture Proficiency in Python and SQL for data engineering and analytics TO BE CONSIDERED Please either ...

their own workflows. We are in the process of a major technology migration, moving our core data pipelines from generating HDF5 files into generating Delta Tables. This will improve efficiency, ensure future scalability and modernise the architecture. We are looking for an experienced lead to head up a small … cutting‐edge FinTech company with the potential to grow your technical, finance and cloud‐ & data‐engineering skills. Responsibilities Technical leadership and management of the delta lake migration team, organising work into 2‐week sprints, upwards reporting on progress and blockers. Development, testing and migration of some ...

into the Bronze layer, working in partnership with the Development and Application Management team. Design and optimise ETL/ELT pipeline frameworks using Databricks, Delta Lake and Azure Data Factory. Ensure Silver and Gold layer data products are fit for purpose for analytics, pricing, AI and ML model … scalability. Experience in regulated financial services, ensuring compliance with industry standards. Deep expertise in data modelling — conceptual, logical and physical. Data warehousing and data lake architecture for high‐performance analytics. ETL/ELT pipeline development and optimisation to support large‐scale data processing. Data integration across structured and unstructured ...

into the Bronze layer, working in partnership with the Development and Application Management team. Design and optimise ETL/ELT pipeline frameworks using Databricks, Delta Lake and Azure Data Factory. Ensure Silver and Gold layer data products are fit for purpose for analytics, pricing, AI and ML model … scalability. Experience in regulated financial services, ensuring compliance with industry standards. Deep expertise in data modelling — conceptual, logical and physical. Data warehousing and data lake architecture for high‐performance analytics. ETL/ELT pipeline development and optimisation to support large‐scale data processing. Data integration across structured and unstructured ...

Microsoft Azure and Databricks Key Responsibilities Design and implement scalable, secure cloud-based data architecture (Azure & Databricks) Develop optimised data models and pipelines using Delta Lake and Azure services Define data standards, policies, and governance practices aligned with compliance Enable real-time analytics and machine learning use cases … internal stakeholders and third-party vendors Key Skills & Experience Proven background designing and delivering enterprise-scale data platforms Strong knowledge of Microsoft Azure, Databricks, Delta Lake, and data warehousing Advanced data modelling and ETL/ELT optimisation experience Familiarity with regulatory frameworks such as IFRS 17 , BCBS ...

Microsoft Azure and Databricks Key Responsibilities: Design and implement scalable, secure cloud-based data architecture (Azure & Databricks) Develop optimised data models and pipelines using Delta Lake and Azure services Define data standards, policies, and governance practices aligned with compliance Enable real-time analytics and machine learning use cases … internal stakeholders and third-party vendors Key Skills & Experience: Proven background designing and delivering enterprise-scale data platforms Strong knowledge of Microsoft Azure, Databricks, Delta Lake, and data warehousing Advanced data modelling and ETL/ELT optimisation experience Familiarity with regulatory frameworks such as IFRS 17 , BCBS ...

transformation Hybrid working with flexibility and autonomy Key Experience Deep end-to-end experience delivering enterprise Databricks analytics solutions Expertise in Lakehouse Architecture, Delta Lake, Spark, PySpark Strong data modelling skills (3NF, Kimball, Data Vault) Experience exposing Delta Lakehouse to Power BI or Tableau Skilled in Python … Scala or SQL for data engineering Experience building scalable ETL pipelines (Workflows, Delta Live Tables) Hands-on experience with Azure, AWS or GCP (storage, compute, networking) Knowledge of Databricks security: RBAC, IAM, encryption Excellent analytical skills and the confidence to challenge thinking at all levels Bonus: Databricks certifications (Data ...

Engineering/Data Science, and Real-Time Analytics). Core Data Stack: High proficiency in advanced SQL , Python/PySpark, and data lakehouse architectures (Delta Lake). Cloud Infrastructure: Strong understanding of Microsoft Azure data services (Azure Data Factory, Azure Synapse, Azure SQL, Key Vault) as they relate ...

with OIDC federation) and/or Azure DevOps, supporting trunk-based development practices. Must Have Databricks: hands-on experience building and optimizing pipelines, managing Delta Lake, and administering workspaces (cluster policies, Unity Catalog, Secret Scopes, Workflows). Python/PySpark: strong programming skills for data processing, automation ...

Validation. AWS Cloud Platform, S3, IAM, CI/CD concepts, DevOps principles, Agile Delivery. Advantageous Skills: Databricks, Palantir Foundry, Foundry Pipeline Builder, Ontology concepts, Delta Lake, Secure Data Environment (SDE) experience, NHS/Healthcare data experience. Government/(url removed)/Public Sector domain experience strongly preferred. Must ...

solutions end-to-end, with a focus on Lakehouse Architecture and ML/AI-driven analytics. Design scalable data models and pipelines using Spark, Delta Lake, and PySpark. Lead client engagements, shaping data strategies and best‐practice architectures. Win business through presenting solutions and advising potential clients. Optimise ...

connectivity issues across Foundry, Databricks and AWS. Desirable Skills Experience with Foundry Virtual Tables, Ontology, Workshop, Pipeline Builder or Code Repositories. Experience with Delta Lake, Parquet, Iceberg, CSV and JSON. Data governance, lineage, secure data-sharing and access approval experience. Python, PySpark or SQL development experience. CI/ ...

Technical Lead, Lead Data Engineer, or Technical Delivery Lead position. Strong hands-on expertise with Python. Extensive experience with Databricks, including Workflows, Notebooks, and Delta Lake. Strong experience with Apache Spark/PySpark. Proven track record building and optimising large-scale ETL/ELT pipelines. Strong understanding of modern ...

multi-site business environment Strong proficiency in Python and SQL, with hands-on experience in Spark/PySpark and modern table formats (e.g. Delta Lake, Iceberg, Hudi) Expertise in AWS data services (e.g. S3, Glue, Redshift, Lambda, Step Functions, Kinesis) and infrastructure-as-code (Terraform or CDK) Experience ...

have the following skills and experience: Strong commercial experience in data architecture or data engineering within modern cloud environments. Deep expertise in Databricks, including Delta Lake, Unity Catalog, and SQL Warehousing. Advanced SQL skills with a focus on performance tuning and optimization. Experience migrating legacy data systems into ...

have the following skills and experience: Strong commercial experience in data architecture or data engineering within modern cloud environments. Deep expertise in Databricks, including Delta Lake, Unity Catalog, and SQL Warehousing. Advanced SQL skills with a focus on performance tuning and optimization. Experience migrating legacy data systems into ...

have the following skills and experience: Strong commercial experience in data architecture or data engineering within modern cloud environments. Deep expertise in Databricks, including Delta Lake, Unity Catalog, and SQL Warehousing. Advanced SQL skills with a focus on performance tuning and optimization. Experience migrating legacy data systems into ...

BMLL Technologies is seeking a Lead Developer to oversee a pivotal technology migration. The role involves hands-on development primarily in C++ and Python, along with managing a small team in Agile sprints. You will ...

000. The successful candidate will engage with clients, focusing on architecture and data solutions while leveraging expertise in Spark, PySpark, and Delta Lake. You'll lead workshops, translate complex requirements, and contribute to modernizing enterprise data platforms. #J-18808-Ljbffr ...