SC Cleared Senior Data Engineer

Senior Data Engineer - Multi-Cloud
Location: London, United Kingdom (Hybrid)
Salary: £65,000 - £85,000 per annum

An experienced Senior Data Engineer is required to design, build, and optimise scalable data platforms for complex transformation programmes. This position demands strong expertise across cloud-based data engineering, architecture, and DevOps practices, working within both AWS and Azure environments.

You'll play a key role in enabling seamless data integration, supporting advanced analytics, and delivering robust, reusable data solutions aligned with client data strategies.

Key Responsibilities

• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Design and develop scalable, automated data pipelines using cloud-native technologies (e.g. Spark), integrating with on-premise sources where required.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Implement event-driven data architectures and optimise pipeline performance, parallelisation, and risk detection (e.g. fraud use cases).
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Build and maintain data access layers and ETL frameworks to support self-service and ad-hoc onboarding of data assets.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Develop reusable engineering patterns and tooling to improve scalability and consistency across programmes.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Apply DevOps and software engineering best practices, including Infrastructure as Code (IaC) and CI/CD pipelines.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Collaborate with cross-functional teams to ensure high-quality data integration, governance, and performance.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Deliver secure, resilient, and scalable data solutions across hybrid cloud environments (AWS and Azure).
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Configure and optimise data services including:
  • p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> AWS: EC2, EMR, S3, Lambda
  • p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Azure: Blob Storage and related services
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Ensure alignment with client data strategies, governance frameworks, and security standards.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Support knowledge sharing and capability uplift across teams.

Skills & Experience - Essential

• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Strong experience in data engineering and data architecture in complex programme environments.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Proven ability to develop and manage automated data pipelines using technologies such as Spark.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Expertise in event-driven architectures and pipeline performance optimisation.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Deep understanding of ETL processes, data integration, and self-service onboarding models.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Hands-on experience across both AWS and Azure platforms.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Practical knowledge of:
  • p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> AWS: EC2, EMR, S3, Lambda
  • p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Azure: Blob Storage and related services
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Understanding of cloud networking concepts in AWS and Azure.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Experience with Terraform (IaC) and GitLab (CI/CD).
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Solid grounding in DevOps and modern software engineering practices.

Desirable

• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Experience in government or regulated sectors.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Familiarity with large-scale data platforms and data access layers.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Exposure to fraud detection or risk analytics use cases.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Knowledge of data governance, security, and compliance frameworks.

Security Requirements

• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> This role requires Security Clearance (SC).
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Candidates must hold active clearance or be eligible and willing to undergo the process.

Engineering Practices

• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Strong commitment to automation, scalability, and reusability.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Advocate for DevOps principles, CI/CD, and Infrastructure as Code.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Experienced working in agile, collaborative teams.
• p]:pt-0 [&> p]:mb-2 [&> p]:my-0'> Dedicated to delivering robust, production-grade data solutions.

JBRP1_UKTJ