Data Engineer - London/Hybrid - TWE41666

twentyAI are partnering with a globally renowned law firm currently undergoing a major digital and data transformation. With a deep-rooted legacy in legal excellence and a strong global footprint, the firm is currently modernising their data capability and building a new data platform.

About the Role

You will be part of a diverse group of engineers and machine learning experts, working with cutting-edge Azure cloud technologies, including Microsoft Fabric and related services. Your mission is to design reliable, efficient data pipelines that enable the business to access trusted, well-structured data. You will also focus on building and scaling the core data infrastructure that supports advanced analytics and machine learning efforts across the business.

Responsibilities

• Design and develop end-to-end data pipelines that ingest, transform, and prepare data for analytics and machine learning workflows.
• Work with Infrastructure as Code, primarily Terraform, to automate and manage cloud infrastructure, enabling repeatable and reliable deployment processes.
• Collaborate closely with data scientists, MLEs, and business teams in an agile environment to deliver data solutions that support key firm initiatives.
• Build scalable and efficient batch and streaming data workflows within the Azure ecosystem.
• Apply distributed processing techniques using Apache Spark to handle large datasets effectively.
• Help drive improvements in data quality, implementing validation, cleansing, and monitoring frameworks.
• Contribute to the firm’s efforts around data security, governance, and compliance by adopting best practices and integrating security controls in pipelines.
• Identify bottlenecks and optimise performance across data pipelines and cloud infrastructure.
• Participate in the ongoing migration from legacy systems to modern data platforms.

Your Background

• Experience with Microsoft Azure data tools — especially Data Factory and Synapse.
• Familiarity with Microsoft Fabric will be beneficial. Otherwise, experience with platforms like Databricks or Snowflake is also valued.
• Proficiency in Infrastructure as Code, preferably with Terraform, and understanding of CI/CD pipelines in a data engineering context.
• Practical knowledge of distributed processing frameworks, particularly Spark.
• Comfortable working in a complex environment that is evolving from legacy systems toward a modern data architecture.
• Strong problem-solving skills and the ability to work collaboratively in a cross-functional agile team.
• Exposure to data governance, security, and compliance principles is desirable.
• Background in industries where data security and governance are paramount is a plus, such as financial services, professional services, or legal.

Why join?

• Work at the forefront of a major digital transformation in a prestigious global legal organisation.
• Be part of a collaborative team that values innovation, continuous learning, and practical engineering approaches.
• Opportunity to work with the latest Azure tools and technologies - including Microsoft Fabric.

If you’re passionate about building scalable, secure data platforms and enjoy working in a dynamic, supportive environment, we want to hear from you. Click the Apply button or send your CV to mihaela.popova@twentyai.com directly.