Data Engineer

Data Engineer

Azure | Databricks | PySpark | Hybrid Cloud | Fabric

Location: London (Hybrid)
Salary: £85,000 + Bonus + Benefits
Type: Permanent

A Data Engineer is required for a fast-evolving (re)insurance business at the heart of the Lloyd's market, currently undergoing a major data transformation. With a strong foundation in the industry and a clear vision for the future, they're investing heavily in their data capabilities to build a best-in-class platform that supports smarter, faster decision-making across the business.

As part of this journey, they're looking for a Data Engineer to join their growing team. This is a hands-on role focused on building scalable data pipelines and enhancing a modern Lakehouse architecture using Databricks, PySpark, and Azure. The environment is currently hybrid cloud and on-prem, with a strategic move towards Microsoft Fabric-so experience across both is highly valued.

What you'll be doing:

• Building and maintaining robust data pipelines using Databricks, PySpark, and Azure Data Factory.
• Enhance and maintain a Lakehouse architecture using Medallion principles
• Working across both cloud and on-prem environments, supporting the transition to Microsoft Fabric.
• Collaborating with stakeholders across Underwriting, Actuarial, and Finance to deliver high-impact data solutions.
• Support DevOps practices and CI/CD workflows in Azure

Technical Requirements:

• Strong hands-on experience with:
• Databricks (including pipeline development and orchestration)
• PySpark and Python for data transformation and processing
• Azure Data Factory for data integration
• Solid understanding of:
• Medallion architecture and Lakehouse design principles
• Hybrid cloud/on-prem environments
• Microsoft Azure ecosystem
• Exposure to or interest in:
• Microsoft Fabric and its evolving role in enterprise data platforms
• Azure DevOps for CI/CD and deployment
• T-SQL and dimensional modelling (Kimball methodology)
• Experience in Financial Services or Lloyd's market is a plus

Apply now or get in touch to find out more - alexh@pioneer-search.com