51 to 75 of 95 Delta Lake Jobs in London

## **Mandatory Skills****Azure Databricks Platform Engineering*** Hands-on implementation and troubleshooting of Azure Databricks* Databricks Serverless configuration and workload optimisation* Databricks SQL and Delta Lake development* Cluster management, compute policies, autoscaling, and workload monitoring**Data Engineering*** Strong Python, PySpark, and SQL development skills* Design and build ingestion ...

cloud databases Proven track record with complex data migration projects (terabyte+ datasets, multiple legacy source systems, structures and unstructured data) Proficiency with Parquet/Delta Lake or other modern data storage formats Experience with streaming architectures using Kafka, Event Hubs, or Kinesis for real-time data processing Knowledge ...

architect Deep Unity Catalog experience: you have configured it, not just used it, including RBAC on sensitive data Strong PySpark, Spark SQL and Delta Lake, including performance tuning at scale Production pipeline experience with Lakeflow/DLT, Workflows and DABs Desirable: Databricks Certified Data Engineer Professional, Solutions Architect … Champion, or Databricks Champion status Databricks Summit, user group or community speaking MLflow, Model Serving, Mosaic AI or Genie delivery experience Lakebase, Delta Sharing or Lakehouse Federation experience Knowledge of GDS Service Standard, NCSC cloud security principles and government data classifications DV clearance or willingness to undergo DV vetting ...

secure handling of inputs/outputs, and adherence to resiliency and security expectations Preferred qualifications, capabilities, and skills Experience with data lakehouse architectures using Delta Lake, Apache Spark, or similar open‐source frameworks Familiarity with AWS‐native data services such as Glue, Lake Formation, Redshift, or Athena ...

designing and implementing data solutions on at least one public cloud platform (AWS, Azure, or GCP)Working knowledge of distributed data systems: Apache Spark, Delta Lake, or equivalent (Hadoop, Kafka, Flink)Experience leading technical customer conversations — discovery, whiteboarding, architecture reviewsFamiliarity with one or more: data engineering (ETL/ ...

designing and implementing data solutions on at least one public cloud platform (AWS, Azure, or GCP)Working knowledge of distributed data systems: Apache Spark, Delta Lake, or equivalent (Hadoop, Kafka, Flink)Experience leading technical customer conversations — discovery, whiteboarding, architecture reviewsFamiliarity with one or more: data engineering (ETL/ ...

designing and implementing data solutions on at least one public cloud platform (AWS, Azure, or GCP) Working knowledge of distributed data systems: Apache Spark, Delta Lake, or equivalent (Hadoop, Kafka, Flink) Experience leading technical customer conversations - discovery, whiteboarding, architecture reviews Familiarity with one or more: data engineering ...

data fidelity. Collaborate with cross-functional teams to align data architecture with financial regulations (e.g., GDPR, Basel III, MiFID II). Drive adoption of Delta Lake , Apache Spark , and MLflow for advanced analytics and machine learning use cases. Provide technical leadership and mentoring to data engineers and analysts ...

data science and AI projectsDeep understanding of large language models (GPT, Llama, Claude, Mistral) and their reasoning capabilitiesStrong experience with Databricks including DLT, Delta Lake and Unity Catalog governanceSolid knowledge of streaming technologies such as Spark Structured Streaming and AutoloaderProficiency in programming with Python, SQL or ScalaExpertise ...

including performance tuning (partitioning, shuffle optimization, caching), troubleshooting, and designing scalable batch/stream processing patterns. Deep lakehouse and data platform expertise(e.g., Delta Lake concepts, schema evolution, data quality frameworks, lineage/metadata, and governance patterns) applied in production environments. Strong DevSecOps and platform engineering capability, including ...

Skills: Experience with agent frameworks (e.g., Google ADK, LangGraph) and agent protocols such as MCP, A2A, or AG-UIExperience with Databricks, MLflow, and Delta Lake or Apache IcebergExposure to model serving on Kubernetes and to model monitoring (drift, latency, accuracy)Familiarity with CRM platforms such as Salesforce ...

clearly articulate what modern features they used recently and why (platform evolution awareness is essential for this role). Should Have Strong command of Delta Lake and performance patterns: MERGE/CDC strategies, partitioning/file sizing, OPTIMIZE/ZORDER, VACUUM, table maintenance automation. Production architecture practices: Environment ...

including performance tuning (partitioning, shuffle optimization, caching), troubleshooting, and designing scalable batch/stream processing patterns. Deep lakehouse and data platform expertise (e.g., Delta Lake concepts, schema evolution, data quality frameworks, lineage/metadata, and governance patterns) applied in production environments. Strong DevSecOps and platform engineering capability, including ...

discussions, including vendor conversations and the technical sections of proposals Desirable Experience with NHS or other routine healthcare data at patient level Unity Catalog, Delta Lake and medallion structure, bronze to silver to gold Exposure to dbt and dimensional modelling Experience contributing to vendor or partner relationships, such ...

structure data What we are looking for: Proven Data Engineering experience. Experience building production-grade pipelines at scale Proven experience with Databricks, including Delta Lake and Unity Catalog Strong Python and SQL Strong experience with Azure cloud data services This is an exciting opportunity for an experienced Data ...

Skills: Experience with agent frameworks (e.g., Google ADK, LangGraph) and agent protocols such as MCP, A2A, or AG-UI Experience with Databricks, MLflow, and Delta Lake or Apache Iceberg Exposure to model serving on Kubernetes and to model monitoring (drift, latency, accuracy) Familiarity with CRM platforms such ...

performance, cost and standards-compliance metrics, using them to guide roadmap priorities and architecture governance decisions. PERSONAL PROFILE Deep expertise in Databricks (Unity Catalog, Delta Lake, Spark, Databricks SQL) or equivalent Lakehouse platforms, including administration, cost management, and performance optimisation. Strong understanding of cloud-native architecture principles, cost ...

Disclosure and Barring Service)Mandatory SkillsAzure Databricks Platform EngineeringHands-on implementation and troubleshooting of Azure DatabricksDatabricks Serverless configuration and workload optimisationDatabricks SQL and Delta Lake developmentCluster management, compute policies, autoscaling, and workload monitoringData EngineeringStrong Python, PySpark, and SQL development skillsDesign and build ingestion and transformation pipelinesFull and incremental ...

Experience owning delivery roadmaps, OKRs, and budgets for an architecture or engineering function. Strong technical foundation across the data platform stack: Databricks lakehouse architecture (Delta Lake, medallion patterns), Azure Synapse (star schemas, dedicated pools), and data pipeline orchestration (ADF or Databricks Workflows). Experience with data quality frameworks ...

Cognita is seeking a hands-on Data Engineer to build and maintain a modern cloud data platform in Azure Databricks. You will design robust data pipelines from CRM, MIS, Finance and HR data sources, with ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn, and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits and perks ...

creation of data domains that align with the banks underlying capabilities Data Engineering & Platforms Design data product schemas across Snowflake and Databricks, including Delta Lake structures and access patterns Define and enforce architectural guardrails for data product design, including performance, scalability, security, and cost efficiency Set design standards ...

control the creation of data domains that align with the banks underlying capabilitiesData Engineering & PlatformsDesign data product schemas across Snowflake and Databricks, including Delta Lake structures and access patternsDefine and enforce architectural guardrails for data product design, including performance, scalability, security, and cost efficiencySet design standards and review ...

covering the full lifecycle, discovery, design and build, through to production release and support Deep hands‐on experience of the core platform: Unity Catalog, Delta Lake, workflows and jobs, declarative pipelines, SQL warehouses, cluster and compute configuration Migrated data at scale off on‐prem or legacy platforms ...

models and datasets is highly desirable. Broader experience with cloud data platforms such as Azure, and familiarity with Databricks features beyond dbt, such as Delta Lake or Unity Catalogue. Familiarity with data visualisation tools and BI tools and their data requirements. Previous involvement in large‐scale data migration … transformation on Databricks SQL (Databricks SQL/Spark SQL) VS Code UI for SQL and dbt coding Git/GitHub Databricks Workflows/Delta Live Tables for orchestration Modern data visualisation and self‐serve/conversational analytics tooling Benefits: A relaxed working environment with regular socials including a summer ...