Senior Data Engineer

Senior Data Engineer

• 6 Month initial contract
• Hybrid role (up to 3 times a week in London)
• Start ASAP
• £500 - £540 per day Inside IR35
• Azure/Python/SQL/ELT experience

*Please note that this role requires security clearance due to the nature of the project

Essential Skills & Experience

• 10+ years' experience in Data Engineering, with a minimum of 3 years of hands-on Azure Databricks experience delivering production-grade solutions.
• Strong programming proficiency in Python and Spark (PySpark) or Scala, with the ability to build scalable and efficient data processing applications.
• Advanced understanding of data warehousing concepts, including dimensional modelling, ETL/ELT patterns, and modern data integration architectures.
• Extensive experience working with Azure data services, particularly Azure Data Factory, Azure Blob Storage, Azure SQL Database, and related components within the Azure ecosystem.
• Demonstrable experience designing, developing, and maintaining large-scale datasets and complex data pipelines in cloud environments.
• Proven capability in data architecture design, including the development and optimisation of end-to-end data pipelines for performance, reliability, and scalability.
• Expert-level knowledge of Databricks, including hands-on implementation, cluster management, performance tuning, and (ideally) relevant Databricks certifications.
• Hands-on experience with SQL and NoSQL database technologies, with strong query optimisation skills.
• Solid understanding of data quality frameworks, data governance practices, and implementing automated testing/validation within pipelines.
• Proficient with version control systems such as Git, including branching strategies and CI/CD integration.
• Experience working within Agile delivery environments, collaborating closely with cross-functional teams to deliver iterative, high-quality solutions.