9 of 9 Contract Delta Lake Jobs in London

/ELT solutions and enterprise data pipelines. Experience with event-driven and messaging architectures. Data Platforms & Cloud Technologies Hands-on experience with: Azure Data Lake Databricks Snowflake Delta Lake Azure Synapse Analytics/Microsoft Fabric Spark/PySpark Strong understanding of: Data Lake and Lakehouse architecture ...

patterns, batch and streaming ingestion, event driven design Databricks and/or Microsoft Fabric at architecture level Azure Data Factory, Synapse, Azure SQL, Data Lake Storage Gen2 Advanced SQL, Python or PySpark, Spark optimisation, Delta Lake, Unity Catalog Azure DevOps or GitHub Actions, Terraform or Bicep Entra ...

Senior Data Engineer, you'll take ownership of the design, development and optimisation of production-grade data solutions using Databricks, Apache Spark and Delta Lake. Your responsibilities will include: Designing, building and optimising scalable ETL/ELT data pipelines using Databricks, PySpark and Spark SQL Implementing Medallion Architecture patterns … Strong commercial experience with Databricks Excellent PySpark, Apache Spark and Spark SQL skills Experience building production-grade ETL/ELT pipelines Strong knowledge of Delta Lake and Lakehouse architecture Experience implementing Medallion/Bronze-Silver-Gold architectures Strong experience with Azure and/or AWS data platforms Knowledge ...

improving security, supportability and delivery speed. 4. Data Engineering and Integration Design and build reliable ingestion and transformation pipelines using Python, PySpark, SQL and Delta Lake. Implement full and incremental ingestion, CDC where appropriate, schema evolution, reconciliation, error handling and data quality controls. Design reusable integration patterns for REST … Azure Databricks implementation and troubleshooting Databricks Serverless and compute/workload optimisation Azure identity, networking, security, secrets, monitoring and private connectivity Databricks SQL, Delta Lake and performance optimisation Python, PySpark and SQL Jobs/workflows, incremental processing, CDC and data quality REST/API and external data integration ...

implementation of scalable data architectures on the Databricks Lakehouse platform. Oversee the development of efficient ETL/ELT pipelines using PySpark, SQL, and Delta Lake. Mentor and guide junior data engineers, promoting best practices in code quality, CI/CD, and performance tuning. Collaborate with data architects, analysts … London office for 3 days per week for collaborative working and stakeholder engagement. Essential Requirements & Skills Databricks Expertise: Extensive hands-on experience with Databricks, Delta Lake, Unity Catalog, and Spark optimization. Programming Languages: Advanced proficiency in Python/PySpark and SQL. Cloud & Infrastructure: Strong background in cloud data ...

data access. Ensure performance, scalability, and security of backend systems. Databricks & Data Engineering Develop ETL pipelines and workflows using PySpark on Databricks. Work with Delta Lake, Unity Catalog, and Databricks REST APIs. Optimize queries and pipelines for cost efficiency in cloud environments (Azure/AWS). SQL Database ...

structure data What we are looking for: Proven Data Engineering experience. Experience building production-grade pipelines at scale Proven experience with Databricks, including Delta Lake and Unity Catalog Strong Python and SQL Strong experience with Azure cloud data services This is an exciting opportunity for an experienced Data ...

observability Knowledge of Auto Loader, Lakeflow Connect and batch, streaming or incremental ingestion patterns Production-level PySpark and SQL development skills Strong knowledge of Delta Lake, including MERGE, schema evolution, OPTIMIZE, ZORDER and VACUUM Experience with Databricks Workflows, orchestration and CI/CD Understanding of monitoring, alerting, replay ...

environment you build will become the firm's golden data source. What We're Looking For Proven, hands-on delivery of Databricks/Delta Lake builds - ACID transactions, time travel, schema evolution, and performance tuning (OPTIMIZE, Z-ordering, VACUUM) Strong command of medallion architecture and PySpark, lazy evaluation ...