Ability to mentor and guide other engineers; comfortable stepping up to lead when required. Excellent communication and stakeholder management skills. Active SC Clearance (essential). Desirable: Experience with WireMock , Kafka , or other service mocking/event-driven test tools. Familiarity with CI/CD tooling (e.g. Jenkins, GitLab, Bamboo). More ❯
and archiving/housekeeping data. 4. Knowledge of Cloud platforms, serverless architecture preferable 5. Good understanding of software patterns and when/how to apply them 6. Knowledge of Kafka will be advantageous 7. Experience of working in an Agile/DevOps software development environment 8. Have ability to help with mentoring tasks. 9. Have experience in working on More ❯
Spark). Strong knowledge of data workflow solutions like Azure Data Factory, Apache NiFi, Apache Airflow etc Good knowledge of stream and batch processing solutions like Apache Flink, ApacheKafka Good knowledge of log management, monitoring, and analytics solutions like Splunk, Elastic Stack, New Relic etc Given that this is just a short snapshot of the role we encourage More ❯
Hadoop, Spark). Strong knowledge of data workflow solutions like Azure Data Factory, Apache NiFi, Apache Airflow Good knowledge of stream and batch processing solutions like Apache Flink, ApacheKafka Good knowledge of log management, monitoring, and analytics solutions like Splunk, Elastic Stack, New Relic Note: The following line contains removed formatting for safety. Given that this is just More ❯
and improvement opportunities. Architect cloud-native systems using Azure, AWS, and Oracle Cloud. Apply Service-Oriented Architecture (SOA) and Domain-Driven Design (DDD) principles. Design event-driven systems using Kafka and Pub/Sub. Lead API design, integration strategies, and legacy modernization to microservices. Contribute to Disaster Recovery planning and platform resilience. Implement Identity and Access Management (IDAM) and More ❯
migration projects (terabyte+ datasets, multiple legacy source systems, structures and unstructured data) Proficiency with Parquet/Delta Lake or other modern data storage formats Experience with streaming architectures using Kafka, Event Hubs, or Kinesis for real-time data processing Knowledge of data architectures supporting AI/ML workloads, including vector databases, feature stores and MLOps pipelines Experience processing and More ❯
level requirements into actionable technical designs. Contribute to defining API contracts, data flows, and integration patterns. Technical Stewardship Apply microservice and event-driven architectural patterns, leveraging technologies such as Kafka for asynchronous communication and decoupling. Champion operational excellence, supporting observability, testing, deployment, and incident response practices. Embed and advocate engineering best practices: scalable design, high availability, resilience, and continuous … have Proven experience designing and delivering architectures for large-scale, distributed microservices systems. Deep understanding of microservice patterns, event-driven architecture, and domain-driven design. Hands-on expertise with Kafka (or similar event streaming platforms), asynchronous messaging, and API design. Ability to communicate complex architectural concepts to both technical and non-technical audiences. Collaborative, pragmatic mindset with a strong More ❯
of Linux operating systems and scripting languages (e.g. Python) Knowledge of infrastructure as code and container technologies (e.g. Puppet, Docker) Knowledge of COTS integration technologies (e.g. Apache Camel, ApacheKafka) Experience of Atlassian tools (e.g. Jira, Confluence) Experience with public cloud platforms (e.g. AWS) Experience of the complete system life cycle from problem definition through to deployment Understanding of More ❯
Gloucester, Gloucestershire, South West, United Kingdom Hybrid/Remote Options
NSD
optimal operation. The Data Engineer Should Have: Active eDV clearance (West) Willingness to work full time on site in Gloucester when required. Required technical experience in the following: ApacheKafkaApache NiFI SQL and noSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java To be Considered: Please either apply by clicking online or emailing me … SKILLS: DATA ENGINEER/DATA ENGINEERING/DEFENCE/NATIONAL SECURITY/DATA STRATEGY/DATA PIPELINES/DATA GOVERNANCE/SQL/NOSQL/APACHE/NIFI/KAFKA/ETL/GLOUCESTER/DV/SECURITY CLEARED/DV CLEARANCE More ❯
understanding of relational databases (e.g., PostgreSQL). Bonus: Advanced LookML knowledge and experience building data visualisation tools. Skilled in building and managing real-time and batch data pipelines using Kafka and DBT. Familiarity with Docker, Terraform, and Kubernetes for application orchestration and deployment. A strong numerical or technical background, ideally with a degree in mathematics, physics, computer science, engineering More ❯
professional experience in data engineering roles, preferably for a customer facing data product Expertise in designing and implementing large-scale data processing systems with data tooling such as Spark, Kafka, Airflow, dbt, Snowflake, Databricks, or similar Strong programming skills in languages such as SQL, Python, Go or Scala Demonstrable use and an understanding of effective use of AI tooling More ❯
City of London, London, United Kingdom Hybrid/Remote Options
Client Server
a UK citizen or hold the right to work without visa restrictions. About you: You have strong backend software engineering experience with TypeScript and Node.js You have experience with Kafka and Event Driven Architectures You have a good knowledge of microservices, RESTful APIs and scalable system architecture You have experience of working with relational and NoSQL databases (PostgreSQL, MySQL More ❯
Glue, Redshift) and/or Azure (Data Lake, Synapse). Programming & Scripting:Proficiency in Python, SQL, PySpark etc. ETL/ELT & Streaming:Expertise in technologies like Apache Airflow, Glue, Kafka, Informatica, EventBridge etc. Industrial Data Integration:Familiarity with OT data schema originating from OSIsoft PI, SCADA, MES, and Historian systems. Information Modeling:Experience in defining semantic layers, asset hierarchies More ❯
South West London, London, United Kingdom Hybrid/Remote Options
Client Server
work from home on Fridays. About you: You have strong full stack web development experience using TypeScript and Node.js You understand AWS, microservices, APIs, Event Driven systems, Data Streaming (Kafka), DevOps environments, CI/CD pipelines and how to deploy to the cloud, they use CloudFlare You're happy to do some front end work with React You have More ❯
St. Albans, Hertfordshire, South East, United Kingdom Hybrid/Remote Options
Client Server
enhancements to complex Payments and client systems within a microservices environment (300 services). You'll be working with a modern tech stack using C# .Net Core, AWS, Kubernetes, Kafka, Redis and TypeScript/Angular; using the right tool for the job, you'll be able to pick up new technologies and make recommendations for improvements. Location/WFH More ❯
of the industry's brightest minds. Technical skills required: C#, .NET, Python, Blazor, REST APIs, gRPC, GraphQL, MS SQL Server. Any experience of the following will be useful: Docker, Kafka, Sentry, Grafana Degree education is preferred, but we will consider Software Engineers with the right level of technical ability for the role. We are looking for a C# Software More ❯
Stratford-Upon-Avon, Warwickshire, West Midlands, United Kingdom
Ascent People
of the industry's brightest minds. Technical skills required: C#, .NET, Python, Blazor, REST APIs, gRPC, GraphQL, MS SQL Server. Any experience of the following will be useful: Docker, Kafka, Sentry, Grafana Degree education is preferred, but we will consider Software Engineers with the right level of technical ability for the role. We are looking for a C# Software More ❯
Milton Keynes, Buckinghamshire, South East, United Kingdom
Ascent People
of the industry's brightest minds. Technical skills required: C#, .NET, Python, Blazor, REST APIs, gRPC, GraphQL, MS SQL Server. Any experience of the following will be useful: Docker, Kafka, Sentry, Grafana Degree education is preferred, but we will consider Software Engineers with the right level of technical ability for the role. We are looking for a C# Software More ❯
City of London, London, United Kingdom Hybrid/Remote Options
Client Server
Senior Data Engineer (AWS Kafka) Remote UK to £80k Are you a data technologist with strong AWS and Kafka experience? You could be joining a growing games company that combines state of the art technology, gaming and luxury fashion. What's in it for you: Salary to £80k Remote working Private medical care Life Assurance 25 days holiday … and reliable data pipelines that can cope with c50 million events per day, with the aim to scale that to 500 million events per day. You'll be using Kafka for real-time data streaming and AWS to build, optimise and monitor data warehousing solutions in Amazon RedShift, owning and managing AWS based systems to ensure cost effective, secure … RDS, RedShift, Glue as well as Kinesis, MongoDB and PostgreSQL You have experience of working in environments with high throughput data (millions of events per hour) You have strong Kafka experience You have strong SQL and data modelling skills and experience with both SQL and NoSQL data stores You have a strong understanding of DBT (or equivalent) and Tableau More ❯
