18 of 18 Delta Lake Jobs in London

considering: We are looking for an experienced Data Architect with strong expertise in Azure Databricks, Azure Data Factory, Azure Data Lake, Delta Lake, data modelling, data governance, and cloud data architecture, along with domain experience in London Market Insurance. The candidate will be responsible for defining … time. Key Responsibilities: Define end-to-end data architecture for Azure-based data platforms using Azure Databricks, Azure Data Factory, ADLS Gen2, Delta Lake, Azure Synapse, and Power BI. Design scalable and secure lakehouse architecture using bronze, silver, and gold data layers. Lead architecture and design for data ...

solutions. Key Responsibilities Design, develop, and maintain scalable data pipelines using Python and Databricks Build and optimise ETL/ELT workflows leveraging Databricks (Spark, Delta Lake) Develop robust data models and data architectures on Lakehouse platforms Implement and manage data workflows within the Databricks ecosystem Ensure high data … drive engineering best practices Skills & Experience Extensive experience in data engineering Strong programming expertise in Python Hands-on experience with Databricks (workflows, notebooks, jobs, Delta Lake) Strong experience with Apache Spark/PySpark Experience building scalable ETL pipelines Proficiency in SQL and relational databases Experience with cloud platforms ...

implement scalable Databricks Lakehouse platforms on AWS and/or Azure aligned to client requirements Architect end-to-end data platforms including ingestion, storage (Delta Lake), processing, and consumption layers Build and configure cloud infrastructure using infrastructure-as-code (e.g. Terraform & Declarative Automation Bundles(DAB's)) Establish secure … pipelines, and release processes for Databricks deployments Implement monitoring, logging, cost optimisation, and performance tuning across the platform Design and implement data pipelines using Delta Live Tables, Auto Loader, and Databricks Workflows for both batch and streaming workloads Client Delivery & Enablement responsibilities: Work directly with clients to translate business ...

someone who wants to execute a defined plan. You will be working across two fronts simultaneously: building the core platform capabilities — the virtualized data lake, Common Data Layer, and data contract framework — that give WTW a unified, AI-ready data foundation; and embedding directly within product domains to deliver … access requirements. Background in financial services, insurance, or broking is a plus but not required. Technical Skills Deep expertise in Databricks, including Unity Catalog, Delta Lake, Delta Sharing, and the full Databricks data engineering and ML stack. Strong command of DBT for modular, testable, and well-documented ...

modern cloud data engineering. Hands-on experience with Microsoft Fabric, including Lakehouse, Warehouse, Pipelines, Dataflows, Notebooks and Power BI. Good understanding of data lake, lakehouse and warehouse architecture. Experience building data ingestion and transformation pipelines from multiple source systems. Strong SQL skills and experience working with relational data models. … PySpark for data transformation and automation. Understanding of Power BI, semantic models, DAX and reporting requirements. Experience working with APIs, JSON, CSV, Parquet, Delta tables and common enterprise data formats. Knowledge of data governance, access control, data quality, metadata and lineage. Good documentation skills and the ability to produce ...

Engineering/Data Science, and Real-Time Analytics). Core Data Stack: High proficiency in advanced SQL , Python/PySpark, and data lakehouse architectures (Delta Lake). Cloud Infrastructure: Strong understanding of Microsoft Azure data services (Azure Data Factory, Azure Synapse, Azure SQL, Key Vault) as they relate ...

transformation Hybrid working with flexibility and autonomy Key Experience Deep end-to-end experience delivering enterprise Databricks analytics solutions Expertise in Lakehouse Architecture, Delta Lake, Spark, PySpark Strong data modelling skills (3NF, Kimball, Data Vault) Experience exposing Delta Lakehouse to Power BI or Tableau Skilled in Python … Scala or SQL for data engineering Experience building scalable ETL pipelines (Workflows, Delta Live Tables) Hands-on experience with Azure, AWS or GCP (storage, compute, networking) Knowledge of Databricks security: RBAC, IAM, encryption Excellent analytical skills and the confidence to challenge thinking at all levels Bonus: Databricks certifications (Data ...

/Databricks/distributed computing Experience in REST APIs and microservices architecture Cloud & Tools Cloud: Azure (preferred), AWS, or GCP Data platforms: Snowflake, Databricks, Delta Lake DevOps: Git, CI/CD tools, Terraform (optional) Containerization: Docker, Kubernetes Financial Services Experience Proven experience in: Banking/Capital Markets/ ...

Validation. AWS Cloud Platform, S3, IAM, CI/CD concepts, DevOps principles, Agile Delivery. Advantageous Skills: Databricks, Palantir Foundry, Foundry Pipeline Builder, Ontology concepts, Delta Lake, Secure Data Environment (SDE) experience, NHS/Healthcare data experience. Government/GOV.UK/Public Sector domain experience strongly preferred. Must ...

Validation. AWS Cloud Platform, S3, IAM, CI/CD concepts, DevOps principles, Agile Delivery. Advantageous Skills: Databricks, Palantir Foundry, Foundry Pipeline Builder, Ontology concepts, Delta Lake, Secure Data Environment (SDE) experience, NHS/Healthcare data experience. Government/(url removed)/Public Sector domain experience strongly preferred. Must ...

data engineering.· 2+ years experience as data engineer on AI/ML project with Python· Strong experience using Databricks, including Jobs, Asset Bundles, Delta Lake, and MLflow, as well as Azure data engineering tools such as Azure Data Factory and Azure Data Lake Storage (ADLS).· Solid ...

data access. Ensure performance, scalability, and security of backend systems. Databricks & Data Engineering Develop ETL pipelines and workflows using PySpark on Databricks. Work with Delta Lake, Unity Catalog, and Databricks REST APIs. Optimize queries and pipelines for cost efficiency in cloud environments (Azure/AWS). SQL Database ...

Technical Lead, Lead Data Engineer, or Technical Delivery Lead position. Strong hands-on expertise with Python. Extensive experience with Databricks, including Workflows, Notebooks, and Delta Lake. Strong experience with Apache Spark/PySpark. Proven track record building and optimising large-scale ETL/ELT pipelines. Strong understanding of modern ...

data governance frameworks, metadata management, and regulatory compliance It would be a bonus if you have: Familiarity with emerging technologies such as Apache Iceberg, Delta Lake, or data observability tooling Experience leading architecture workshops with both technical and non-technical stakeholders A background providing technical mentorship ...

reporting. Core requirements: Strong experience delivering end-to-end data engineering solutions in production Deep hands-on expertise with Databricks (jobs, notebooks, workflows, Delta Lake) Experience designing and operating CI/CD pipelines for data platforms Strong Python & SQL skills for scalable data processing Solid understanding of lakehouse ...

have the following skills and experience: Strong commercial experience in data architecture or data engineering within modern cloud environments. Deep expertise in Databricks, including Delta Lake, Unity Catalog, and SQL Warehousing. Advanced SQL skills with a focus on performance tuning and optimization. Experience migrating legacy data systems into ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits and perks ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits and perks ...