76 to 95 of 95 Delta Lake Jobs in London

seeking a Data Architect for a permanent hybrid role in the UK. The candidate will lead data architecture with extensive Azure Databricks experience, including Delta Lake medallion patterns and Unity Catalog governance. You will collaborate with senior stakeholders and ensure secure data lake implementations in a Microsoft ...

only. Multi-cloud platform depth (AWS and GCP): hands-on experience with data services across one or both of AWS (Redshift, Glue, S3/Lake Formation, MSK, RDS) and GCP (BigQuery, Dataflow, Pub/Sub, Cloud Composer). Data flow mapping and optimisation: demonstrable experience producing current-state data … serving, and GenAI/RAG architectures grounded in proprietary content. Lakehouse and modern data platform design: strong knowledge of open table formats (Apache Iceberg, Delta Lake), zone architecture (landing, cleansed, curated), streaming and batch pipeline patterns, and data quality frameworks. Project engagement and design assurance: able to engage ...

track. Required Skills & Experience Technical Skills Strong experience with Python and SQL. Hands-on experience with Apache Spark. Experience with Databricks (Unity Catalog/Delta Lake). Experience orchestrating workflows with Apache Airflow. Familiarity with GCP services (GCS, BigQuery). Experience with dbt and ads/payments APIs ...

spanning legacy, cloud and hybrid environments Comprehensive, hands-on experience architecting enterprise-scale solutions on the Databricks Lakehouse Platform, including workspace design, cluster strategy, Delta Lake and Unity Catalog Deep expertise in medallion architecture, data ontologies, data products and data contracts, with strong command of data quality frameworks ...

fixing performance bottlenecks in live systems through profiling and measurement. Nice to Have: Experience with PostgreSQL/pgvector, OpenSearch (or Elasticsearch), Spark/Delta Lake, AWS, NoSQL databases, or Rust/C++ is useful. The Founders You'll partner with a founding team of AI PhDs and elite ...

regularly observe cloud hardware, network, and operating system faults, and our software must gracefully shield our customers from any of the above. The Delta DML team owns the core write-path operations for Delta Lake, the open-source storage layer behind the Databricks Lakehouse. Our mission … flowing through our platform. We drive performance innovations like Low Shuffle Merge and Deletion Vectors and actively contribute to open source efforts to unify Delta and Iceberg formats. We are seeking a highly skilled and experienced Senior Staff Software Engineer to join our backend team. In this role ...

engineering standards, ensuring high-quality testing, documentation, code quality, reproducibility, and operational reliability. Your skills and experience: Experience with ML platforms including Databricks, MLflow, Delta Lake, and cloud environments. Proficient in Python, PySpark, and SQL, following production coding best practices. Understanding of data, distributed ML pipelines, and model ...

workflow tools. Kafka, Pulsar, Redis Streams, RabbitMQ or other event-driven technologies. Triton Inference Server, Ray Serve, TorchServe, BentoML or Seldon Core. DVC, LakeFS, Delta Lake, Feast or other data-versioning and feature-store technologies. Terraform, Pulumi, CloudFormation or Ansible. GPU cluster management or high-performance computing. Aerial ...

analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the creators of Lakehouse, Apache Spark, Delta Lake and MLflow. Benefits Databricks strives to provide comprehensive benefits and perks. For specific details in your region, refer to the regional benefits ...

Preferred Qualifications, Capabilities, and Skills: Experience with agent frameworks and protocols (Google ADK, LangGraph, MCP, A2A, AG-UI) Experience with Databricks, MLflow, Spark, and Delta Lake or Apache Iceberg Experience serving models on Kubernetes, including GPU workloads and fine-tuned small language models Experience with optimization or pricing ...

Experience building enterprise platforms and data architectures, including multi-tenant systems, fine-grained authorization, identity and access management, API-first architectures, data lakehouse patterns, Delta Lake, vector databases, and RAG pipelines- Strong understanding of model-agnostic AI architectures and tool integration, including multi-provider LLM integration, tool calling ...

with the Go Lang eco-system tools and librariesExperience with Pandas and PySpark libraries would be a plusTechnical experience with Databricks or any similar Delta Lake platforms. Experience working with large datasetsExperience with AWS cloud deployments and APIExperience with Docker and KubernetesFamiliarity with financial statements (e.g. balance sheets ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits and perks ...

Driven Design (DDD) Exceptional command of the English language Nice-to-Have We’d also love experience with: Version control for data and artefacts: DeltaLake DVC MLFlow Designing and operating cloud-based systems AWS and/or Databricks Messaging platforms Infrastructure as Code tools such as: Terraform CloudFormation Pulumi Kubernetes ...

consolidation. Ensure efficient scheduling and orchestration of jobs using Databricks Workflows Collaborate with data scientists and platform teams to improve query performance on Delta Lake. Develop automated monitoring and alerting mechanisms using Databricks REST APIs, Prometheus, or Azure Monitor. Maintain best practices for data engineering performance, testing, and documentation. … Technical Skills Required Strong proficiency with Databricks, Apache Spark (SQL, PySpark, Scala), and Delta Lake. Deep understanding of Spark execution plans, shuffle optimization, and data partitioning strategies. Experience tuning job configurations (e.g., memory management, parallelism, caching, adaptive query execution). Familiarity with orchestration tools: Databricks Workflows, Airflow, or Azure ...

design and implement Lakehouse architectures, govern data access, and lead delivery with a focus on scale and governance. You will work with Unity Catalog, Delta Lake, and SQL/Python/Spark, collaborating with sellers and engineering teams to drive pre-sales and delivery. Hybrid work ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. Benefits At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our employees. ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake and MLflow. Benefits We strive to provide comprehensive benefits and perks that meet the needs of all our employees. Our Commitment to Diversity ...

London, the Data Engineer will own ingestion, transformation and governance across cloud, on-premise and datacentre environments. You will work with Databricks, Spark and Delta Lake to deliver reliable, traceable datasets and enable customer-facing applications while maintaining strict data security and lineage across teams. #J-18808-Ljbffr ...

efficiency. Design, build and operationalise scalable ETL/ELT data pipelines using Databricks, Python and SQL. Develop and optimise solutions using Databricks Lakehouse and Delta Lake. Implement data governance, lineage, controls and audit frameworks within a regulated environment. Lead, mentor and upskill a team of Data Engineers, establishing technical … financial services. Strong experience within Insurance, Banking, Asset Management or FinTech. Advanced Python, PySpark and SQL skills. Deep knowledge of Databricks Lakehouse architecture and Delta Lake. Experience with at least one major cloud platform: AWS, Azure or GCP. Proven experience leading or mentoring Data Engineering teams. Strong stakeholder management ...