101 to 125 of 150 Delta Lake Jobs in the UK

Skills: Experience with agent frameworks (e.g., Google ADK, LangGraph) and agent protocols such as MCP, A2A, or AG-UI Experience with Databricks, MLflow, and Delta Lake or Apache Iceberg Exposure to model serving on Kubernetes and to model monitoring (drift, latency, accuracy) Familiarity with CRM platforms such ...

discussions, including vendor conversations and the technical sections of proposals Desirable Experience with NHS or other routine healthcare data at patient level Unity Catalog, Delta Lake and medallion structure, bronze to silver to gold Exposure to dbt and dimensional modelling Experience contributing to vendor or partner relationships, such ...

Data Warehousing . Strong background in incident resolution, requests, changes and problem‐solving within SLAs. Hands‐on experience with Spark, Data Architecture, SQL and Delta Lake. Cloud development experience (AWS, GCP or Azure). Working knowledge of Medallion Architecture. Demonstrated capability in implementing and supporting pipelines in demanding environments. ...

from employment, education, an apprenticeship or personal projects. Experience with Microsoft Fabric or another cloud data platform. Familiarity with data warehouses, Lakehouses , PySpark or Delta Lake. Awareness of testing and CI/CD practices. Candidates do not need to meet every desirable criterion to apply. We are proud ...

ensuring compliance with security, governance, and operational standards. Key Responsibilities: Design, develop, and optimise scalable data pipelines using Azure Databricks, PySpark, Spark SQL, and Delta Lake. Build robust ETL/ELT solutions and data ingestion frameworks supporting batch and streaming workloads. Integrate data from multiple sources, including Azure … Database, Azure Data Lake Storage Gen2, Microsoft Fabric, APIs, SaaS platforms, and on-premise systems. Implement data quality, validation, monitoring, and governance controls across the data lifecycle. Support platform engineering activities, including CI/CD implementation, environment promotion, performance optimisation, and operational support. Skills we can’t do without ...

ensuring compliance with security, governance, and operational standards. Key Responsibilities: Design, develop, and optimise scalable data pipelines using Azure Databricks, PySpark, Spark SQL, and Delta Lake. Build robust ETL/ELT solutions and data ingestion frameworks supporting batch and streaming workloads. Integrate data from multiple sources, including Azure … Database, Azure Data Lake Storage Gen2, Microsoft Fabric, APIs, SaaS platforms, and on-premise systems. Implement data quality, validation, monitoring, and governance controls across the data lifecycle. Support platform engineering activities, including CI/CD implementation, environment promotion, performance optimisation, and operational support. Skills we can’t do without ...

product, owning the full supply chain from instrument and edge ingestion, through modelling and governance, to exposure for AI, workflow, analytics and customer data lake consumers. You will drive a critical transformation: moving IDBS from a traditional electronic lab notebook capture tool to an enterprise Data Product Provider. That … product metrics. Deep understanding of modern lakehouse architecture, including medallion data tiers, transactional storage engines and change data capture patterns (for example Databricks and Delta Lake). Strong grasp of scientific data modelling, including unstructured to structured transformation, data contracts, schema evolution, ontologies and provenance standards (for example ...

Requirements 7+ years experience in data engineering or data platform development Strong experience with Databricks, Spark and distributed data processing Experience with Unity Catalog, Delta Lake and streaming architectures Advanced Python and Microsoft SQL Server development Proven experience leading engineering teams or workstreams Strong stakeholder engagement and consulting ...

development Strong experience with modern cloud data platforms, Spark and distributed data processing Experience with data cataloguing and governance tooling, open table formats (e.g. Delta Lake, Iceberg) and streaming architectures Advanced Python and Microsoft SQL Server development Proven experience leading engineering teams or workstreams Strong stakeholder engagement ...

Your code comes with tests, and you leave things tidier than you found them. Depth in at least one of SQL, Spark, Databricks/Delta Lake, Dagster/Airflow/Temporal or dbt. Bonus if you’ve shaped any of them into something a model can query, through ...

Your code comes with tests, and you leave things tidier than you found them. Depth in at least one of SQL, Spark, Databricks/Delta Lake, Dagster/Airflow/Temporal or dbt. Bonus if you've shaped any of them into something a model can query, through ...

Your code comes with tests, and you leave things tidier than you found them. Depth in at least one of SQL, Spark, Databricks/Delta Lake, Dagster/Airflow/Temporal or dbt. Bonus if you've shaped any of them into something a model can query, through ...

Must Haves (Hard Skills) Data Lakehouse Mastery: Proven experience building data infrastructure using Trino/Presto and open table formats like Apache Iceberg or Delta Lake. Advanced Data Modeling: Deep expertise in Medallion Architecture patterns, Star Schemas, and processing semi-structured financial data logs (JSON/Avro). Robust ...

seek engineers with multi-language skills and deep experience in Linux-based high-throughput environments, plus exposure to streaming platforms like Kafka and data lake technologies such as Delta Lake or Apache Iceberg. #J-18808-Ljbffr ...

only. Multi-cloud platform depth (AWS and GCP): hands-on experience with data services across one or both of AWS (Redshift, Glue, S3/Lake Formation, MSK, RDS) and GCP (BigQuery, Dataflow, Pub/Sub, Cloud Composer). Data flow mapping and optimisation: demonstrable experience producing current-state data … serving, and GenAI/RAG architectures grounded in proprietary content. Lakehouse and modern data platform design: strong knowledge of open table formats (Apache Iceberg, Delta Lake), zone architecture (landing, cleansed, curated), streaming and batch pipeline patterns, and data quality frameworks. Project engagement and design assurance: able to engage ...

medallion architecture) Leading the design of cloud data platforms (AWS) - including services like Redshift, S3, Glue, Athena Evaluating and introducing new technologies (e.g. Iceberg, Delta Lake, observability tools) Driving data governance, standards, and best practice Acting as a trusted advisor to both technical and business stakeholders What ...

Strong hands-on experience with AWS data services covering data ingestion, storage, processing, orchestration, security, and monitoring. Advanced knowledge of Databricks , including Lakehouse architecture, Delta Lake, and large-scale data engineering practices. Expertise in conceptual, logical, and physical data modelling, including dimensional modelling, relational modelling, and modern warehouse ...

Databricks experience who can operate at both architectural and engineering level. You should have strong experience across areas such as: Databricks Databricks Lakehouse Delta Lake Medallion Architecture Data Architecture Data Engineering Semantic Modelling/Semantic Layers Databricks Genie/AI/BI Conversational BI Unity Catalog Enterprise data ...

fixing performance bottlenecks in live systems through profiling and measurement. Nice to Have: Experience with PostgreSQL/pgvector, OpenSearch (or Elasticsearch), Spark/Delta Lake, AWS, NoSQL databases, or Rust/C++ is useful. The Founders You'll partner with a founding team of AI PhDs and elite ...

creation of data domains that align with the banks underlying capabilities Data Engineering & Platforms: Design data product schemas across Snowflake and Databricks, including Delta Lake structures and access patterns Define and enforce architectural guardrails for data product design, including performance, scalability, security, and cost efficiency Set design standards ...

Disclosure and Barring Service)Mandatory SkillsAzure Databricks Platform EngineeringHands-on implementation and troubleshooting of Azure DatabricksDatabricks Serverless configuration and workload optimisationDatabricks SQL and Delta Lake developmentCluster management, compute policies, autoscaling, and workload monitoringData EngineeringStrong Python, PySpark, and SQL development skillsDesign and build ingestion and transformation pipelinesFull and incremental ...

services using the below technologies: Python, pandas and SQL in Jupyter notebooks for analysis, and for scripting dbt and Databricks data platform with Delta Lake Lightdash as our BI tool and semantic layer Streamlit for interactive data applications Airflow/Orchestra for orchestration Kubernetes for application deployment ...

Data Platform team in Newcastle upon Tyne. You will build and optimize data pipelines on the Azure cloud using Databricks, Data Factory and Delta Lake, delivering scalable solutions for analytics and AI initiatives. You will mentor colleagues, design data models, and partner with data scientists and business stakeholders ...

STACK Infrastructure EMEA is seeking a Senior Databricks Data Engineer to build and manage the enterprise data platform, applying PySpark, Python, SQL and Delta Lake in production. You will design reliable pipelines across Finance, Delivery, Operations, HR and PMO, implement governance with Unity Catalog, and optimise performance ...

Luxoft is seeking an experienced data engineering specialist to design and implement a scalable enterprise Lakehouse on Azure, leveraging ADF, Delta Lake, ADLS Gen2 and Databricks. The role focuses on cloud-native ELT pipelines, batch and near real-time workloads, and reusable data frameworks with strong governance. ...