12 of 12 Delta Lake Jobs in the City of London

understand requirements, design and build robust data pipelines, and deliver end-to-end analytics and ML/AI capabilities. Key Responsibilities Design, build, and maintain scalable data pipelines and Delta Lake architectures in Databricks on AWS. Develop and enhance the Front Office data warehouse to ensure performance, reliability, and data quality for trading analytics. Partner with data scientists … equivalent technical discipline. 8+ years of hands-on experience building large-scale distributed data pipelines and architectures. Expert-level knowledge in Apache Spark, PySpark, and Databricks—including experience with Delta Lake, Unity Catalog, MLflow, and Databricks Workflows. Deep proficiency in Python and SQL, with proven experience building modular, testable, reusable pipeline components. Strong experience with AWS cloud services More ❯

understand requirements, design and build robust data pipelines, and deliver end-to-end analytics and ML/AI capabilities. Key Responsibilities Design, build, and maintain scalable data pipelines and Delta Lake architectures in Databricks on AWS. Develop and enhance the Front Office data warehouse to ensure performance, reliability, and data quality for trading analytics. Partner with data scientists … equivalent technical discipline. 8+ years of hands-on experience building large-scale distributed data pipelines and architectures. Expert-level knowledge in Apache Spark, PySpark, and Databricks—including experience with Delta Lake, Unity Catalog, MLflow, and Databricks Workflows. Deep proficiency in Python and SQL, with proven experience building modular, testable, reusable pipeline components. Strong experience with AWS cloud services More ❯

Strong experience with Azure data services (Data Factory, Synapse, Blob Storage, etc.). Proficiency in SQL for data manipulation, transformation, and performance optimisation. Hands-on experience with Databricks (Spark, Delta Lake, notebooks). Solid understanding of data architecture principles and cloud-native design. Experience working in consultancy or client-facing roles is highly desirable. Familiarity with CI/ More ❯

Strong experience with Azure data services (Data Factory, Synapse, Blob Storage, etc.). Proficiency in SQL for data manipulation, transformation, and performance optimisation. Hands-on experience with Databricks (Spark, Delta Lake, notebooks). Solid understanding of data architecture principles and cloud-native design. Experience working in consultancy or client-facing roles is highly desirable. Familiarity with CI/ More ❯

data processing logic for cleaning, enriching, and aggregating data. Ensure data consistency and accuracy throughout the data lifecycle. Azure Databricks Implementation: Work extensively with Azure Databricks Unity Catalog, including Delta Lake, Spark SQL, and other relevant services. Implement best practices for Databricks development and deployment. Optimise Databricks workloads for performance and cost. Need to program using the languages More ❯

data processing logic for cleaning, enriching, and aggregating data. Ensure data consistency and accuracy throughout the data lifecycle. Azure Databricks Implementation: Work extensively with Azure Databricks Unity Catalog, including Delta Lake, Spark SQL, and other relevant services. Implement best practices for Databricks development and deployment. Optimise Databricks workloads for performance and cost. Need to program using the languages More ❯

Azure with a core focus on Databricks Provide technical leadership on Databricks Design and implement data pipelines in Azure Databricks, optimizing for performance, cost, and scalability Advise of data lake migration, data modelling and pipeline optimisation Refactor and modernize existing data models to integrate with Power BI Coordinate phased migrations to ensure zero disruption to live operations Replace legacy … a strong focus on building scalable data pipelines Expertise in Azure Databricks (7+years) including building and managing ETL pipelines using PySpark or Scala (essential) Solid understanding of Apache Spark, Delta Lake, and distributed data processing concepts Hands-on experience with Azure Data Lake Storage Gen2, Azure Data Factory, and Azure Synapse Analytics Proficiency in SQL and Python More ❯

machine learning use cases. Support the migration of legacy reporting tools into Databricks and modern BI solutions. Key Skills & Experience Essential: Strong hands-on experience with Databricks (SQL, PySpark, Delta Lake). Solid knowledge of BI and data visualisation tools (e.g., Power BI, Tableau, Qlik). Strong SQL and data modelling skills. Experience working with large, complex financial More ❯

machine learning use cases. Support the migration of legacy reporting tools into Databricks and modern BI solutions. Key Skills & Experience Essential: Strong hands-on experience with Databricks (SQL, PySpark, Delta Lake). Solid knowledge of BI and data visualisation tools (e.g., Power BI, Tableau, Qlik). Strong SQL and data modelling skills. Experience working with large, complex financial More ❯

and regulatory compliance. Provide technical leadership to engineering teams and contribute to architecture standards. Skills & Experience Proven experience as a Data/Solution Architect with strong expertise in Databricks (Delta Lake, MLflow, Spark) . Deep understanding of commercial insurance processes (underwriting, claims, personal/specialty lines). Strong background in Azure (Synapse, Data Factory, Fabric) or similar cloud More ❯

and regulatory compliance. Provide technical leadership to engineering teams and contribute to architecture standards. Skills & Experience Proven experience as a Data/Solution Architect with strong expertise in Databricks (Delta Lake, MLflow, Spark) . Deep understanding of commercial insurance processes (underwriting, claims, personal/specialty lines). Strong background in Azure (Synapse, Data Factory, Fabric) or similar cloud More ❯

maintain real-time streaming data systems (Kafka, Kinesis, or Flink) Build robust feature pipelines using Airflow, Prefect, or Dagster Manage and optimise data storage solutions (Snowflake, BigQuery, Redshift, or Delta Lake) Automate and scale model training pipelines in close partnership with ML engineers Deploy, observe, and improve pipelines using Docker, Kubernetes, Terraform, or dbt Champion data reliability, scalability More ❯