NiFi Data Engineer (SC Cleared)

Job Description

\n

Job Title: NiFi Data Engineer

\n

Location: Hybrid (up to 3 days per week on-site, across offices in London & South England)

\n

Contract Length: 8 Months (with scope to extend)

\n

Start Date: September 2026

\n

IR35: Outside

\n

Interview Process: 1 Stage, MS Teams

\n

Clearance Required Before Applying: SC (with eligibility for higher levels of clearance)

\n

We are supporting a public sector client in hiring a NiFi Data Engineer to join a collaborative development team delivering a data Lakehouse architecture across both cloud and on-premises platforms.

\n

This is a hands-on data engineering role working within Agile sprints across a full software development life cycle, building and maintaining data pipelines, developing data models and handling non-functional requirements including security, performance and metadata labelling. The successful candidate will work closely with engineers and architects to deliver robust, well-tested pipelines that support a modern Lakehouse architecture across AWS and on-premises environments.

\n

This role suits a self-starting data engineer with experience in Lakehouse architectures, event-driven design and CI/CD-integrated development, who is comfortable picking up new technologies quickly and contributing from day one within a fast-paced, security-conscious environment.

\n

Key Responsibilities

\n

\n
• Design, develop and maintain data pipelines within a Lakehouse architecture across cloud and on-premises platforms

\n

• Develop data models to support pipeline design, ensuring alignment with broader architectural standards and business requirements

\n

• Handle non-functional aspects of pipeline development including security controls, performance optimisation and metadata labelling

\n

• Implement event sourcing, change data capture and event-driven architecture patterns within the data platform

\n

• Develop unit and integration tests for data pipelines as part of a CI/CD-integrated software development life cycle

\n

• Collaborate within an Agile development team across sprint planning, code review and iterative delivery

\n

• Apply Java and Groovy within an object-oriented design approach to support pipeline development and tooling

\n\n

Essential Skills & Requirements

\n

\n
• Proven experience building data pipelines within Lakehouse architectures across cloud or on-premises environments

\n

• Must be a UK national with SC clearance held and eligibility & willingness to undertake DV clearance

\n

• Experience with event sourcing, change data capture and event-driven architecture patterns

\n

• Familiarity with Java and Groovy, with a solid understanding of object-oriented design principles

\n

• Experience working within an SDLC with an integrated development team using CI/CD tooling including Git

\n\n

Desirable Skills

\n

\n
• Hands-on experience with Apache NiFi for data pipeline development and orchestration

\n

• AWS experience including familiarity with EMR, S3 and SQS

\n

• Awareness of domain-driven design approaches and architecture

\n

• Linux awareness and Scripting skills

\n

• PostgreSQL and PL/SQL knowledge

\n