Senior Data Engineer - AWS/Databricks

Contract: Initial 12 months
Rate: Up to £475 per day - Inside IR35
Location: Birmingham
Working Pattern: Hybrid - 3 days per week onsite at Birmingham office

We are looking for an experienced Senior Data Engineer to join a major enterprise technology programme within the financial services sector. You will be responsible for designing, building and optimising large-scale data pipelines and distributed processing solutions within an AWS environment. The role will also support modern AI/ML and Generative AI initiatives by developing scalable, secure and reliable data platforms.

Key Responsibilities

• Design, build and maintain high-throughput data pipelines using Apache Spark and cloud-native technologies.
• Develop scalable ETL/ELT workflows for structured and unstructured datasets.
• Build and optimise data-processing solutions within AWS environments.
• Develop performant solutions using Java or Scala alongside advanced SQL.
• Support modern data lake and lakehouse architectures.
• Work alongside AI/ML teams to provide data platforms supporting Generative AI, machine learning and advanced analytics use cases.
• Implement appropriate data quality, governance, security and monitoring standards.
• Troubleshoot complex data-processing and distributed-system issues within production environments.

Essential Experience

• 5+ years of commercial Data Engineering experience.
• Strong hands-on Apache Spark development experience.
• Strong programming experience with Java and/or Scala.
• Strong AWS experience, ideally including technologies such as S3, EMR, Glue and IAM.
• Advanced SQL development skills.
• Strong experience designing and developing ETL/ELT pipelines.
• Experience working with large-scale and distributed data-processing environments.
• Databricks and/or Snowflake.