24 of 24 Permanent Kafka Jobs in the North of England

Java development experience Deep expertise in Spring Framework and microservices architecture Proven ability to mentor and develop engineering talent Strong background in messaging systems (Kafka, RabbitMQ) Proficiency with AWS, Docker, SQL/NoSQL, and automated testing Experience in Agile environments with a DevOps mindset Mindset & Skills Ownership mentality with ...

transformation tasks across the engineering team Programmatically ingest large volumes of structured and unstructured data from REST APIs, streaming platforms (e.g. Event Hubs, Kafka), and legacy databases into ADLS Gen2 and OneLake Develop structured data models aligned to Lakehouse, Medallion Architecture, and Delta Lake patterns Continuously profile, debug ...

PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka, AWS SQS, Other Cloud Queuing Native services) SQL and NoSQL databases and storage (HDFS, Iceberg, Elastic, S3, Data Lake) Containerisation and orchestration (Docker/ ...

NoSQL databases. Secure development practices. Git-based source control. Knowledge of CI automation systems such as Jenkins Bonus points for: Event streaming (e.g. Kafka, Kinesis) and change data capture (CDC). Messaging (e.g. RabbitMQ, SQS). Docker and Kubernetes. Solution and architecture design. Test-driven development (TDD). ...

microservices, APIs and distributed systems Experience working in Agile/DevOps environments Familiarity with CI/CD pipelines and cloud-native tooling Experience with Kafka ...

integrating with third-party APIs End-to-end ownership of systems (design/build/operate) Technical Expertise Event-driven architecture (EventBridge, SQS, SNS, Kafka) Reliability patterns (retries, idempotency, DLQs) Observability and debugging in distributed systems Data modelling and schema evolution Leadership & Collaboration Ability to lead technical design ...

power intelligent decision-making. Key Responsibilities Data Pipeline Development Build, optimize, and maintain scalable data pipelines using Java (primary), plus exposure to Python, Flink, Kafka, or Spark. Develop and support real-time streaming pipelines and event-driven integrations. Integrate data from multiple sources (streaming, batch, APIs) using AWS managed ...

systems Experience integrating with third-party APIs End-to-end ownership of systems (design build operate) Technical Expertise Event-driven architecture (EventBridge, SQS, SNS, Kafka) Reliability patterns (retries, idempotency, DLQs) Observability and debugging in distributed systems Data modelling and schema evolution Leadership & Collaboration Ability to lead technical design ...

code following CI/CD and Agile practices Required Skills & Experience 5+ years' experience building big data pipelines in distributed environments Strong experience with Kafka, Hadoop, Spark and/or Python Strong experience with DBT and Snowflake Experience with Data Vault and dimensional data modelling Strong SQL skills ...

systems Experience integrating with third-party APIs End-to-end ownership of systems (design build operate) Technical Expertise Event-driven architecture (EventBridge, SQS, SNS, Kafka) Reliability patterns (retries, idempotency, DLQs) Observability and debugging in distributed systems Data modelling and schema evolution Leadership & Collaboration Ability to lead technical design ...

platforms Experience owning systems end-to-end (design, delivery, operation, and iteration) Technical Expertise Event-driven architectures and messaging systems (e.g. EventBridge, SQS, SNS, Kafka) Designing for failure retries, idempotency, backoff strategies, DLQs Observability, monitoring, and debugging in distributed systems Data modelling, transformation, and schema evolution across systems Leadership ...

operate real-time IoT data ingestion pipelines handling telemetry, logs, and events from thousands of devices. Implement processing workflows using Kinesis, Lambda, SQS, Kafka, Step Functions , or similar. Ensure data quality, schema governance, and reliability across ingestion and downstream data consumers. API Development Build and maintain RESTful and GraphQL ...

role: SQL Server MS SSIS T-SQL Azure Data Factory Visual Studio Strong written & verbal communication Desirables: Python Azure Synapse/Databricks Flink/Kafka The role will be on a hybrid working basis with 3 days per week in the Liverpool office. The role is commutable from ...

build a strong, healthy culture (No hands-on coding required, pure leadership and product‐focused engineering management.) 🔧 Tech in the ecosystem Java · Kotlin · AWS · Kafka · SQL · Swift · React (You don’t need to be an expert in everything, just credible and technically grounded.) 💡 Why it’s worth exploring Genuine ...

building data pipelines and platforms end-to-end. Hands on engineering across ingestion, transformation and data centralisation. Experience with tools such as Glue, Fivetran, Kafka, Dataflow, ADF or Kinesis. Transformation experience using dbt, Spark or Databricks. Confidence working directly with clients and engaging in technical discussions. Exposure to dashboards ...

building data pipelines and platforms end-to-end. Hands on engineering across ingestion, transformation and data centralisation. Experience with tools such as Glue, Fivetran, Kafka, Dataflow, ADF or Kinesis. Transformation experience using dbt, Spark or Databricks. Confidence working directly with clients and engaging in technical discussions. Exposure to dashboards ...

Implement solutions for ISO 20022 message formats , transformations (XML/JSON), validations, and routing. Build integrations using Mule ESB (or similar), messaging patterns, and Kafka/event streams . Write performant SQL/PL‐SQL queries for Oracle‐based transactional systems. Contribute to system designs, code reviews, and technical … volume systems. Solid understanding of XML/JSON structures and message mapping/transformation. Experience with Mule ESB or equivalent middleware , messaging tools, and Kafka . Strong understanding of Agile delivery, CI/CD pipelines, and DevOps practices. Skilled in unit/integration testing frameworks (JUnit, Cucumber/Litmus ...

/housekeeping data. Knowledge of Cloud platforms, serverless architecture preferable Good understanding of software patterns and when/how to apply them Knowledge of Kafka will be advantageous Experience of working in an Agile/DevOps software development environment Have ability to help with mentoring tasks. Have experience ...

secure engineering, and agile delivery , we continue to build innovative digital services that truly matter. Preferred Technical ExpertiseCloud Infrastructure AWS (EKS, RDS, Aurora, ElastiCache, Kafka, IAM) Secure Hosting Experience working within air-gapped or government-secure environments Container & Cluster Management Docker, Kubernetes, Rancher Helm Jenkins Monitoring & Observability … Prometheus Grafana ELK Stack Dynatrace Secrets & Identity Management HashiCorp Vault Keycloak CI/CD & DevOps Tooling Jenkins Git ServiceNow Trivy Terraform Streaming & Messaging Apache Kafka (including Kafka Replication) Data Technologies PostgreSQL Redis RDLs Automation & Scripting Infrastructure as Code (IaC) Pipeline automation Event relay tooling Bash, Python, Groovy ...

Data Engineer Salary up to £60,000 + discretionary bonus Hybrid working model with collaborative office environment Work with Databricks, Spark Structured Streaming, Kafka (MSK), and AWS Apply online or contact Chelsea Hackett via chelsea.hackett@searchability.com ABOUT THE CLIENT Due to continued growth, we’re seeking a skilled Data … Lakehouse environment. You’ll work with both streaming and batch data pipelines using technologies such as Apache Spark Structured Streaming, Delta Live Tables, and Kafka, ensuring data flows efficiently and reliably across the organisation. Working closely with BI, product, and engineering teams, you will help deliver high-quality, governed ...

Manchester (Hybrid) | Digital Transformation Consultancy Location : Manchester (Hybrid - 3 days a week) Package : £60k-£70k + Benefits Python | Cloud (AWS/Azure/GCP) | Kafka | Airflow | PostgreSQL | Observability This client builds mission‐critical digital platforms that make the UK smarter, safer, greener and healthier. They are a growing digital … Doing Design and implement scalable ETL/ELT pipelines and data models for enterprise platforms Build and operate event‐driven architectures (Kafka) and orchestrate workflows (Airflow) Implement data quality, testing and validation as first‐class concerns across pipelines Deliver monitoring and observability (Prometheus, Grafana or equivalent) and runbook automation ...

Should Have: Active eDV clearance (West) Willingness to work full-time on-site in Manchester when required. Required technical experience in the following: Apache Kafka Apache NiFI SQL and noSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java To be Considered: Please either apply … ENGINEERING/DEFENCE/NATIONAL SECURITY/DATA STRATEGY/DATA PIPELINES/DATA GOVERNANCE/SQL/NOSQL/APACHE/NIFI/KAFKA/ETL/MANCHESTER/DV/SECURITY CLEARED/DV CLEARANCE ...

container-related issues in production environments Implement and maintain monitoring, alerting, and logging solutions Support operational aspects of platforms such as: MongoDB, PostgreSQL, Apache Kafka, OpenSearch Required Skills & Experience Experience working in a DevOps or Platform Engineering role Strong knowledge of Linux and working knowledge of Windows Hands … PowerShell Familiarity with monitoring, alerting, and incident management practices Desirable Skills: Experience with GitOps tooling (Argo CD) Experience with Bamboo and Bitbucket Knowledge of Kafka, MongoDB, or PostgreSQL operations Experience operating OpenSearch or similar logging platforms Understanding of cloud security and access control (RBAC, secrets ...

Solution Architect – Shape the Future of a High-Scale Platform Kafka, SQL, C#, .NET, Golang, TypeScript Are you ready to define and drive the architecture of a platform built to handle millions of transactions per second, powering international growth and long-term scalability? We’re embarking on a full … Define solution architecture and technical strategy for a complex, high-scale platform Own end-to-end design across distributed, event-driven systems (Kafka, SQL, cloud-native) Partner with product, engineering, and business stakeholders to translate vision into architecture Lead technical planning, standards, and best practices across multiple teams Provide ...