20 of 20 Permanent Delta Lake Jobs in London

modelling for Finance and Enterprise data products, working closely with architecture where required. Implement and maintain data pipelines and ETL workflows in Databricks (PySpark, Delta Lake). Contribute to CI/CD pipelines for data applications using Azure DevOps and infrastructure-as-code (Terraform) in line with established … within the team and communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous). Experience contributing ...

decisions, and working with people who care about building things properly, you’ll fit right in. You’ll Work With Azure Data Factory, Data Lake, Key Vault, Azure Functions Databricks, Delta Lake, Unity Catalog, PySpark, Spark SQL Medallion architecture, Lakehouse patterns, distributed compute CI/CD pipelines … home You’ll thrive in this role if you have: Strong hands‐on experience with Azure Data Services and Databricks Solid understanding of Delta Lake, Medallion architecture and Lakehouse patterns Proficiency in Python, PySpark and Spark SQL Experience building CI/CD pipelines using Azure DevOps Knowledge ...

must already hold active SC clearance to be considered. Responsibilities Design, build and maintain Databricks data pipelines using Apache Spark (PySpark & Spark SQL) Implement Delta Lake/Lakehouse architectures (Bronze, Silver, Gold) Develop scalable ELT/ETL workflows for batch and streaming data Optimise performance, reliability and cost … Experience Proven commercial experience as a Databricks Data Engineer Strong PySpark and SQL capability Hands-on delivery within AWS, Azure or GCP Experience with Delta Lake , data lakes and data warehouses Strong understanding of data modelling and scalable pipeline design Experience with Git , CI/CD and agile ...

pipelines using Azure Data Factory and Databricks Develop ETL/ELT processes to transform raw data into analytics-ready datasets Architect and maintain data lake solutions using Azure Data Lake Storage Implement Databricks solutions using Delta Lake and medallion architecture (bronze, silver, gold) Optimise performance, data ...

Doing Designing and building end-to-end data pipelines using Databricks on Azure or AWS Developing modern lakehouse architectures using: Databricks (Spark, SQL, Delta Lake) Azure Data Lake Storage or Amazon S3 Writing high-quality PySpark, Python, and SQL for ingestion, transformation, and modelling Implementing Bronze/ ...

enriching, and aggregating data. Ensure data consistency and accuracy throughout the data lifecycle. Azure Databricks Implementation: Work extensively with Azure Databricks Unity Catalog, including Delta Lake, Spark SQL, and other relevant services. Implement best practices for Databricks development and deployment. Optimise Databricks workloads for performance and cost. Need ...

latest technology, ensuring you can be at the forefront of your field. What You’ll Work With Azure Data Services: Data Factory, Data Lake, SQL Databricks: Spark, Delta Lake Power BI: Advanced dashboards ETL & Data Modelling: T-SQL, metadata-driven pipelines What you’ll do Design ...

equivalent; reliability-first mindset including backfill, retry, and exactly-once semantics * Azure data platform - Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Data Lake Storage; infrastructure as code for data workloads (Terraform or Bicep) * Databricks - Delta Lake, Unity Catalog, job cluster vs interactive cluster trade-offs ...

exposure to mParticle, Tealium, Segment or Salesforce CDP will translate. Production-grade PySpark/Databricks experience — building scalable data pipelines, optimising performance, and operating Delta Lake/Lakehouse architectures in a real production environment. Solid DevOps craft — CI/CD pipelines, Git-based workflows, infrastructure-as-code (Terraform ...

target mapping, data quality rule setup for the target data model’s critical data elements (CDE). Database Optimization: Optimize Cloud databases such as Delta Lake or One Lake for efficiency, scalability, and faster access using techniques like indexing and partitioning. Data Governance: Work with Federated Governance ...

scale data platforms (Databricks, Snowflake, Synapse, BigQuery) Ensure robust data engineering and governance practices across a distributed global data estate Drive scalable lakehouse architectures (Delta Lake, Iceberg, Hudi) to support AI and analytics workloads Cloud, Infrastructure & MLOps Champion multi-cloud engineering across Azure, AWS, and GCP within ...

cloud data strategy, define engineering best practice and guide delivery across cross-functional teams. Responsibilities Build and optimise Azure data platforms using ADF, Data Lake, Azure Functions and Databricks Deliver reliable, high-quality data pipelines Work with stakeholders to ensure solutions meet Lloyd's of London regulatory, audit … Data Services and Databricks in enterprise environments. Proven delivery experience within the Lloyd's of London or London Market insurance sector Deep understanding of Delta Lake, Medallion architecture, Lakehouse patterns and data modelling. Strong Python, PySpark and Spark SQL capability Solid understanding of data governance Proven track record ...

skills, experience working with data pipelines, and an understanding of cloud-based storage such as Amazon S3. Experience with tools like Databricks, Delta Lake, or BI platforms such as Power BI or Tableau would be advantageous. This organisation operates globally and is continuing to invest heavily ...

vector stores, and LLM evaluation methods. Experience working with structured and unstructured datasets (documents, logs, text, images). Familiarity with scalable data storage solutions (Delta Lake, Parquet, Redshift, DynamoDB). Understanding model optimization techniques (quantization, distillation, inference optimization). Strong capability to debug, tune, and optimize distributed systems ...

data platform and services using the below technologies: Python, pandas and SQL in Jupyter notebooks for analysis dbt and Databricks data platform with Delta Lake Lightdash as our BI tool and semantic layer Streamlit for interactive data applications Airflow for orchestration Kubernetes for application deployment CI/ ...

vector stores, and LLM evaluation methods. Experience working with structured and unstructured datasets (documents, logs, text, images). Familiarity with scalable data storage solutions (Delta Lake, Parquet, Redshift, DynamoDB). Understanding model optimization techniques (quantization, distillation, inference optimization). Strong capability to debug, tune, and optimize distributed systems ...

some of the organisation’s most critical functions. Responsibilities: Design, build and deploy scalable, secure data solutions using Azure Databricks, Data Factory and Data Lake Storage. Develop and optimise advanced data pipelines with Python, SQL, Spark/PySpark and Delta Lake. Champion strong data quality, governance and observability … architecture, analytics and business teams to align engineering solutions with organisational needs. Minimum Criteria Extensive experience with Azure services including Azure Databricks, Azure Data Lake Storage, and Azure Data Factory. Advanced proficiency in SQL, Python, and Spark (PySpark), with a strong focus on performance optimization and distributed processing. Proven ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits and perks ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits and perks ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits and perks ...