6 of 6 Contract Delta Lake Jobs in London

/ELT solutions and enterprise data pipelines. Experience with event-driven and messaging architectures. Data Platforms & Cloud Technologies Hands-on experience with: Azure Data Lake Databricks Snowflake Delta Lake Azure Synapse Analytics/Microsoft Fabric Spark/PySpark Strong understanding of: Data Lake and Lakehouse architecture ...

improving security, supportability and delivery speed. 4. Data Engineering and Integration Design and build reliable ingestion and transformation pipelines using Python, PySpark, SQL and Delta Lake. Implement full and incremental ingestion, CDC where appropriate, schema evolution, reconciliation, error handling and data quality controls. Design reusable integration patterns for REST … Azure Databricks implementation and troubleshooting Databricks Serverless and compute/workload optimisation Azure identity, networking, security, secrets, monitoring and private connectivity Databricks SQL, Delta Lake and performance optimisation Python, PySpark and SQL Jobs/workflows, incremental processing, CDC and data quality REST/API and external data integration ...

implementation of scalable data architectures on the Databricks Lakehouse platform. Oversee the development of efficient ETL/ELT pipelines using PySpark, SQL, and Delta Lake. Mentor and guide junior data engineers, promoting best practices in code quality, CI/CD, and performance tuning. Collaborate with data architects, analysts … London office for 3 days per week for collaborative working and stakeholder engagement. Essential Requirements & Skills Databricks Expertise: Extensive hands-on experience with Databricks, Delta Lake, Unity Catalog, and Spark optimization. Programming Languages: Advanced proficiency in Python/PySpark and SQL. Cloud & Infrastructure: Strong background in cloud data ...

data access. Ensure performance, scalability, and security of backend systems. Databricks & Data Engineering Develop ETL pipelines and workflows using PySpark on Databricks. Work with Delta Lake, Unity Catalog, and Databricks REST APIs. Optimize queries and pipelines for cost efficiency in cloud environments (Azure/AWS). SQL Database ...

cross-cloud data engineering initiative, building scalable pipelines between Azure and Google Cloud. Key skills: Strong Python & SQL experience Hands-on Azure Databricks, Delta Lake & Unity Catalog Strong GCP/BigQuery/GCS experience Apache Airflow/Cloud Composer for pipeline orchestration Experience building ELT/data pipelines ...

observability Knowledge of Auto Loader, Lakeflow Connect and batch, streaming or incremental ingestion patterns Production-level PySpark and SQL development skills Strong knowledge of Delta Lake, including MERGE, schema evolution, OPTIMIZE, ZORDER and VACUUM Experience with Databricks Workflows, orchestration and CI/CD Understanding of monitoring, alerting, replay ...