24 of 24 Kafka Jobs in the North West

DevOps tools Bonus Points For Terraform, CloudFormation, or other infrastructure-as-code tools Experience with serverless functions (AWS Lambda, Azure Functions) Messaging systems like Kafka or RabbitMQ Familiarity with security protocols such as OAuth2 and SAML QA automation and integration with third-party platforms Why Join Capco ...

preferably GCP Experience designing cloud-native, microservices, and event-driven architectures Strong grasp of CI/CD pipelines (preferably Jenkins), SQL/NoSQL databases, Kafka, and Java Streaming Familiarity with infrastructure-as-code (e.g., CloudFormation), distributed systems, and secure design practices Experience using enterprise data platforms (eg Atacamma, Colibra ...

Machine Learning – Understanding of AI/ML concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query ...

cloud environments, preferably AWS Experience designing cloud-native, microservices, and event-driven architectures Strong grasp of CI/CD pipelines, SQL/NoSQL databases, Kafka, and Java Streaming Familiarity with infrastructure-as-code (e.g., CloudFormation), distributed systems, and secure design practices Bonus Points For Experience using GitLab ...

delivering service migrations and platform go-lives into production Nice to haves: Helm chart authoring and advanced Argo CD usage Event-driven microservices (Kafka) PostgreSQL in cloud environments Advanced CI/CD pipeline design Experience with Internal Developer Platforms (e.g. Backstage) Multi-cloud exposure and/or Python development ...

cloud solutions Proven track record architecting secure, scalable services using microservices and CQRS patterns Strong experience in CI/CD pipelines, SQL/NoSQL, Kafka, CloudFormation, and streaming frameworks Advanced understanding of system security protocols (e.g., OAuth, IAM, TLS, encryption) Leadership capability to manage and mentor teams while driving ...

agile environments. Confident communication skills, able to convey technical concepts to non-technical stakeholders. Bonus Points For Experience with real-time streaming tools like Kafka or Spark Streaming. Exposure to MLOps practices and maintaining ML models in production. Familiarity with data visualisation tools such as Power BI, Qlik ...

Hands-on experience with at least one major cloud platform (Azure, AWS, or GCP) Experience with big data technologies such as Spark, Hadoop, or Kafka Proficiency with ETL/ELT tools and frameworks (e.g. Airflow, dbt, Informatica) Strong programming skills in Python, Java, Scala, or similar Experience implementing data ...

NoSQL databases. Secure development practices. Git-based source control. Knowledge of CI automation systems such as Jenkins Bonus points for: Event streaming (e.g. Kafka, Kinesis) and change data capture (CDC). Messaging (e.g. RabbitMQ, SQS). Docker and Kubernetes. Solution and architecture design. Test-driven development (TDD). ...

Design and develop robust pipelines using DeltaLake, Spark Structured Streaming, and Unity Catalog Build real-time event-driven solutions with tools such as Kafka and Azure Event Hubs Apply DevOps principles to develop CI/CD pipelines using Azure DevOps, Jenkins, or GitHub Actions Collaborate with clients and stakeholders ...

pipelines with tools like Jenkins or CircleCI Knowledge of data security protocols and distributed system design Bonus Points For Experience with messaging systems (Kafka, Spark Streaming, Kinesis) Familiarity with schema design and semi-structured data formats Exposure to containerisation, graph databases, or machine learning concepts Proficiency with cloud-native ...

systems Experience integrating with third-party APIs End-to-end ownership of systems (design build operate) Technical Expertise Event-driven architecture (EventBridge, SQS, SNS, Kafka) Reliability patterns (retries, idempotency, DLQs) Observability and debugging in distributed systems Data modelling and schema evolution Leadership & Collaboration Ability to lead technical design ...

operate real-time IoT data ingestion pipelines handling telemetry, logs, and events from thousands of devices. Implement processing workflows using Kinesis, Lambda, SQS, Kafka, Step Functions , or similar. Ensure data quality, schema governance, and reliability across ingestion and downstream data consumers. API Development Build and maintain RESTful and GraphQL ...

with large, imperfect datasets Experience in product-led or scale-up environments Engineers who take ownership and enjoy solving complex problems Nice to have Kafka, Hadoop, Cassandra, Redis or similar Experience with real-time or event-driven systems Exposure to recommendation systems, ML or data-driven products Cloud experience ...

evidence your clearance) Required Skills & Experience Strong experience designing modern cloud-based data platforms. Hands-on architectural experience with: AWS (essential): S3, EMR, Glue Kafka/event streaming architectures Python & PySpark-based data processing Experience designing data ingestion pipelines (batch and real-time). Proficiency in Infrastructure as Code ...

Java, with strong experience in Big Data technologies such as: Spark, Hadoop etc. Practical knowledge of building real-time event streaming pipelines (e.g., Kafka, Spark Streaming, Kinesis). Proficiency in AWS cloud environments. Proven experience developing modern data architectures including Data Lakehouse and Data Warehousing. A solid understanding ...

Python DBT Cloud techs such as Snowflake, BigQuery or Databricks Dimensional Modelling such as Kimball Desirable to have experience with: CI/CD pipelines Kafka; real-time streaming Snowflake as a data warehouse Interview Process: There are 4 stages to the process: A 30 minute discussion about experience ...

regulated data Desirable Experience Experience working with healthcare data standards (e.g. FHIR, HL7) Exposure to real-time data processing or streaming tools (e.g. Kafka) Familiarity with data governance, security, and compliance frameworks in healthcare Experience supporting analytics or machine learning workloads Apply Now If you’re a contract Data ...

levels. Knowledge of industry‐standard data models and frameworks such as Lambda, Medallion, and Data Mesh. Experience with real‐time data processing (Kafka, Spark) and BI tools (Power BI, Tableau). Strong analytical and problem‐solving skills, translating business requirements into effective data solutions. Equal Opportunities & Reasonable Adjustments ...

Data Engineer up to £90,000 + Benefits Manchester (Hybrid) This is a great opportunity to join a high-growth adtech scale-up where you can take ownership of large-scale data infrastructure that directly ...

This job is with Capco, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. Principal Backend Engineer - Java Location: London ...

driving engineering direction across a large-scale distributed system. What you’ll do Design and build Java (Spring Boot) microservices Architect event-driven systems (Kafka/messaging) Apply AI/ML to real-world business problems Work with modern development tooling, including AI-assisted environments Influence technical direction … Principal/Staff/Architect level Strong Java and distributed systems background Hands-on AI/ML experience (Python or similar) Experience with Kafka or MQ, SQL/Oracle, cloud and containerisation Nice to have Financial services experience (post-trade, custody, corporate actions, trading ...

ingestion, processing, and access layers Design scalable, resilient data platforms (hub-and-spoke, lake/lakehouse) Lead implementation of batch and real-time pipelines (Kafka, APIs, SFTP) Architect solutions using: AWS (S3, EMR, Glue) PySpark/Python Support cloud migrations and platform modernisation Ensure best practices in CI/… Required Experience Active SC Clearance (essential) Strong experience with AWS data platforms (S3, EMR, Glue) Expertise in data lake/lakehouse architectures Experience with Kafka/event-driven architectures Strong Python & PySpark skills Experience with Terraform and CI/CD pipelines Proven track record designing large-scale data solutions ...

Senior Data Engineer - Manchester (Hybrid) - £80,000 + bonus The Company Our client is undergoing a significant digital transformation, modernising their data platform to better support high-volume, real-time products and analytics. This is ...