Kafka Developer

Kafka Developer (UK Public Sector Programme)

Role Title: Kafka Developer
Contract Length: 3 months (Initial)
Location: Remote Working with Occasional Site Visits (UK)
IR35: Inside IR35

Role Overview

We are seeking an experienced Kafka Developer to join a high-profile UK Public Sector programme focused on delivering scalable, secure, and resilient data integration capabilities. The successful candidate will play a key role in designing, developing, and supporting event-driven architectures and Real Time data streaming solutions using Apache Kafka.

Working within an agile, multidisciplinary delivery team, you will collaborate with architects, engineers, business analysts, and security specialists to develop robust integration services that enable the exchange of critical data across government systems and platforms.

Key Responsibilities

• Design, develop, and maintain event-driven solutions using Apache Kafka and related technologies.
• Build and support Real Time data pipelines and streaming applications.
• Develop Kafka producers, consumers, and stream processing services.
• Configure, manage, and optimise Kafka topics, partitions, replication, and retention policies.
• Implement integration patterns between Legacy and modern cloud-native services.
• Collaborate with solution architects to design scalable and resilient messaging architectures.
• Monitor and troubleshoot Kafka environments, identifying and resolving performance bottlenecks.
• Implement security controls including encryption, authentication, authorisation, and audit logging.
• Develop infrastructure automation and deployment pipelines using DevOps best practices.
• Produce high-quality technical documentation and knowledge transfer materials.
• Participate in agile ceremonies including sprint planning, stand-ups, reviews, and retrospectives.
• Support testing activities including unit, integration, performance, and operational readiness testing.

Essential Skills & Experience

• Strong commercial experience with Apache Kafka in enterprise-scale environments.
• Proven experience developing event-driven and microservices-based architectures.
• Strong knowledge of Kafka Connect, Kafka Streams, and Schema Registry.
• Experience developing applications using Java, Spring Boot, or similar technologies.
• Experience implementing and consuming RESTful APIs.
• Strong understanding of distributed systems and messaging patterns.
• Experience working with CI/CD pipelines and DevOps practices.
• Familiarity with containerisation technologies such as Docker and Kubernetes.
• Experience with cloud platforms such as AWS, Azure, or Google Cloud.

Desirable Skills

• Experience in UK Government or wider Public Sector environments.
• Knowledge of Confluent Platform and associated tooling.
• Experience with Infrastructure as Code tools such as Terraform.
• Familiarity with data governance, data quality, and security standards.
• Understanding of government digital service delivery principles.
• Experience integrating with enterprise data platforms and APIs.

Security & Clearance

• Candidates must be eligible to work in the UK.
• Existing Security Clearance (SC) is desirable; candidates must be willing to undergo security vetting if required.

Personal Attributes

• Excellent communication and stakeholder engagement skills.
• Strong analytical and problem-solving abilities.
• Self-motivated and able to work effectively in a remote environment.
• Collaborative team player with a proactive approach to delivery.
• Ability to manage competing priorities in a fast-paced programme environment.