and MLflow for managing and tracking code deployment or model versions Experience with cloud-based data platforms such as AWS or Google Cloud Platform Nice to have: Experience with Kafka Proven track record of leading large scale data infrastructure in production Experience with container technologies (such as Docker) and orchestration technologies (such as Kubernetes) Experience working in ecommerce or More ❯
in setting up and managing monitoring, metrics, and alerting systems Experience operating production-grade services at scale Great to have: Experience with tools such as: Terraform, SaltStack, MongoDB, Elasticsearch, Kafka, Prometheus, Grafana or HashiCorp Vault Experience with securing applications, services, and data, including authentication, authorization, TLS, and encryption Exposure to Kubernetes (administering, deploying, or developing apps on K8s clusters More ❯
Leeds, West Yorkshire, Yorkshire and the Humber Hybrid / WFH Options
BAE Systems
testing both UI and large data sets Java experience Other Desirable Skills (not Mandatory) to which we would be happy to upskill if candidate is keen to learn: Kubernetes Kafka Familiarity with BDD and TDD tools Awareness of accessibility standards AWS Postgres Benefits As well as a competitive pension scheme, BAE also offers employee share plans, an extensive range More ❯
facing, complex and large scale project management experience - 5+ years of continuous integration and continuous delivery (CI/CD) experience - 5+ years of database (eg. SQL, NoSQL, Hadoop, Spark, Kafka, Kinesis) experience - 5+ years of consulting, design and implementation of serverless distributed solutions experience - 3+ years of cloud based solution (AWS or equivalent), system, network and operating system experience More ❯
professional experience in data engineering roles, preferably for a customer facing data product Expertise in designing and implementing large-scale data processing systems with data tooling such as Spark, Kafka, Airflow, dbt, Snowflake, Databricks, or similar Strong programming skills in languages such as SQL, Python, Go or Scala Demonstrable use and an understanding of effective use of AI tooling More ❯
and non-technical stakeholders A background in software engineering, MLOps, or data engineering with production ML experience Nice to have: Familiarity with streaming or event-driven ML architectures (e.g. Kafka, Flink, Spark Structured Streaming) Experience working in regulated domains such as insurance, finance, or healthcare Exposure to large language models (LLMs), vector databases, or RAG pipelines Experience building or More ❯
3+ years of experience in cloud architecture and implementation - Bachelor's degree in Computer Science, Engineering, related field, or equivalent experience - Experience in database (eg. SQL, NoSQL, Hadoop, Spark, Kafka, Kinesis) - Experience in consulting, design and implementation of serverless distributed solutions - Experience in software development with object oriented language PREFERRED QUALIFICATIONS - AWS experience preferred, with proficiency in a wide More ❯
accuracy and throughput Transform prototypes into production-ready data and ML applications that meet throughput and latency requirements Deploy and manage data and ML infrastructure necessary for productionising code (Kafka, Docker, Terraform, etc) Build efficient data pipelines between on-premise and cloud environments to handle text and audio data processing loads for ML models Deploy NLP models in cloud More ❯
understanding of relational databases (e.g., PostgreSQL). Bonus: Advanced LookML knowledge and experience building data visualisation tools. Skilled in building and managing real-time and batch data pipelines using Kafka and DBT. Familiarity with Docker, Terraform, and Kubernetes for application orchestration and deployment. A strong numerical or technical background, ideally with a degree in mathematics, physics, computer science, engineering More ❯
for? 5+ years of experience in engineering Strong node.js knowledge AWS or similar cloud experience Kubernetes, Helm and Docker knowledge Experience with Postgres/Mysql, RabbitMq/Nats/Kafka Strong system design and problem solving skills, we often deal with the "unknown" Strong communication skills, we often need to orchestrate projects with 20+ teams You are hands-on More ❯
to ensure our systems are trusted, reliable and available . The technology underpinning these capabilities includes industry leading data and analytics products such as Snowflake, Tableau, DBT, Talend, Collibra, Kafka/Confluent , Astronomer/Airflow , and Kubernetes . This forms part of a longer-term strategic direction to implement Data Mesh, and with it establish shared platform s that More ❯
Distributed Systems/JAVA & Python/ApacheKafka/Outside IR35/Fully Remote/£600 pd/ASAP Start/6 months Distributed Systems Engineers (Hiring for multiple positions) SR2 have partnered with a high impact Digital Transformation Consultancy to assist in the search for multiple Distributed System Engineers to assess suitability of an existing data federation solution … areas for improvement. You will be implementing a federated data-sharing solution whilst working closely with Gov departments to deploy and test the solution. Required Skills and Experience : ApacheKafka: Expert-level understanding including producer/consumer patterns, topic management, and real-time streaming architectures. Java: Strong backend development with Spring Boot or similar frameworks for secure, scalable microservices. … in contact to discuss the client and requirement. Alternatively, feel free to directly email Ben Sheppard ( (url removed) ) for an immediate response. Distributed Systems/JAVA & Python/ApacheKafka/Outside IR35/Fully Remote/£600 pd/ASAP Start/6 months More ❯
Gloucester, Gloucestershire, South West, United Kingdom Hybrid / WFH Options
NSD
optimal operation. The Data Engineer Should Have: Active eDV clearance (West) Willingness to work full time on site in Gloucester when required. Required technical experience in the following: ApacheKafkaApache NiFI SQL and noSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java To be Considered: Please either apply by clicking online or emailing me … SKILLS: DATA ENGINEER/DATA ENGINEERING/DEFENCE/NATIONAL SECURITY/DATA STRATEGY/DATA PIPELINES/DATA GOVERNANCE/SQL/NOSQL/APACHE/NIFI/KAFKA/ETL/GLOUCESTER/DV/SECURITY CLEARED/DV CLEARANCE More ❯
Gloucester, England, United Kingdom Hybrid / WFH Options
Searchability NS&D
optimal operation. The Data Engineer Should Have: Active eDV clearance (West) Willingness to work full-time on-site in Gloucester when required. Required technical experience in the following: ApacheKafkaApache NiFI SQL and noSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java To be Considered: Please either apply by clicking online or emailing me … SKILLS: DATA ENGINEER/DATA ENGINEERING/DEFENCE/NATIONAL SECURITY/DATA STRATEGY/DATA PIPELINES/DATA GOVERNANCE/SQL/NOSQL/APACHE/NIFI/KAFKA/ETL/GLOUCESTER/DV/SECURITY CLEARED/DV CLEARANCE More ❯
Salisbury, Wiltshire, South West, United Kingdom Hybrid / WFH Options
Anson Mccade
platforms Work directly with senior stakeholders to influence architectural decisions and standards You'll bring: Proven experience as a Lead Data Architect or Senior Data Solutions Architect Expertise in Kafka/Confluent , Databricks , Unity Catalog , and modern data lake/lakehouse architectures Strong grasp of cloud data platforms (AWS, Azure, GCP, Snowflake) Understanding of Data Mesh , Data Fabric , and More ❯
/CD pipelines. Implement and optimize automation processes using infrastructure-as-code (Terraform) Build and maintain data pipelines using Airflow. Manage our tech stack including Python, Node.js, PostgreSQL, MongoDB, Kafka, and Apache Iceberg. Optimize infrastructure costs and develop strategies for efficient resource utilization. Provide critical support by monitoring services and resolving production issues. Contribute to the development of new More ❯
/CD pipelines. Implement and optimize automation processes using infrastructure-as-code (Terraform) Build and maintain data pipelines using Airflow. Manage our tech stack including Python, Node.js, PostgreSQL, MongoDB, Kafka, and Apache Iceberg. Optimize infrastructure costs and develop strategies for efficient resource utilization. Provide critical support by monitoring services and resolving production issues. Contribute to the development of new More ❯
engage technical and non-technical stakeholders alike. The Head of Data Engineering & Insight will work within a modern, cloud-based BI ecosystem , including: Data Integration: Fivetran , HVR, Databricks , ApacheKafka, Google BigQuery , Google Analytics 4 Data Lake & Storage: Databricks Delta Lake, Amazon S3 Data Transformation: dbt Cloud Data Warehouse: Snowflake Analytics & Reporting: Power BI, Excel, Snowflake SQL REST API More ❯
Java, Developer, Java Developer, JavaScript, Spring, Software Development, Software Developer, Microservices Architecture, Kafka, MongoDB, SDLC, Software Development Life Cycle, Birmingham, Onsite. I am looking for a Java Developer to work for a World leading Investment bank located in Birmingham on a 12 month contract. You will be an Java Developer with strong skills in Java (11+), Spring, Microservices Architecture … Kafka, Mongo DB and SDLC. We are looking for a candidate who has no more than 6 years worth of IT work experience, please do not apply if you have over 6 years worth of experience. Skills: 4 - 6 years of Development experience with a focus in Java Java 11+ Spring Kafka for messaging - to some extent or More ❯
SR2 | Socially Responsible Recruitment | Certified B Corporation™
Model Software Engineer | C++ | Python | Agent-Based Modelling | Modelling Distributed Systems | Kafka | Containerisation | Data Pipelines | Mathematical Modelling | Secure Platforms SR2 is supporting a forward-leaning tech consultancy delivering simulation and decision-support tools into national security and public sector environments. They’re hiring a Modelling Software Engineer to design and scale complex, agent-based models using modern technologies and … including agent-based, event-based, or systems dynamics approaches Experience writing production-quality code with a focus on quality, testing, documentation, and performance Proficiency in distributed platform technologies – especially Kafka Comfortable with shell scripting and general Linux-based tooling Experience with database optimisation, data pipelines, and metadata management Skilled in containerised deployments using Docker, Kubernetes, and Helm A proactive More ❯
the full software engineering life cycle, including design, development, and implementation. Reporting into an Engineering Manager, the GoLang Developer will also leverage modern technologies like AWS, Kubernetes, Docker, and Kafka, whilst also architecting and implementing microservices-based solutions in collaboration with cross-functional teams. In addition, the GoLang Developer will also be responsible for optimising system performance, reliability, and … design and architecture Hands-on experience working on complex, data-intensive applications A product-focused mindset and familiarity with working in technology-driven organisations or start-ups Experience with Kafka, Cassandra, gRPC, and microservices architecture will also be beneficial, as well as experience contributing to open-source projects If you're a passionate GoLang Developer seeking a challenging and More ❯
full software engineering life cycle, including design, development, and implementation. Reporting into an Engineering Manager, the Senior Backend Engineer will also leverage modern technologies like AWS, Kubernetes, Docker, and Kafka, whilst also architecting and implementing microservices-based solutions in collaboration with cross-functional teams. In addition, the Senior Backend Engineer will also be responsible for optimising system performance, reliability … architecture Hands-on experience working on complex, data-intensive applications A product-focused mindset and familiarity with working in technology-driven organisations or start-ups Hands-on skills in Kafka, Cassandra, gRPC, and microservices architecture will also be beneficial, as well as experience contributing to open-source projects If you're a passionate Senior Backend Engineer seeking a challenging More ❯
Experience with DevOps pipelines (GitHub Actions, Jenkins). Familiarity with large-scale data management and engineering best practices. Bonus Points For Workflow orchestration tools like Airflow. Working knowledge of Kafka and Kafka Connect. Experience with Delta Lake and lakehouse architectures. Proficiency in data serialization formats: JSON, XML, PARQUET, YAML. Cloud-based data services experience. Ready to build the More ❯
Experience with DevOps pipelines (GitHub Actions, Jenkins). Familiarity with large-scale data management and engineering best practices. Bonus Points For Workflow orchestration tools like Airflow. Working knowledge of Kafka and Kafka Connect. Experience with Delta Lake and lakehouse architectures. Proficiency in data serialization formats: JSON, XML, PARQUET, YAML. Cloud-based data services experience. Ready to build the More ❯
Assurance/Testing across mix of React front-end and Node back-end on AWS stack. Must be WCAG2.2 AAA compliant (accessible). Mix of microservices, lambda functions, and (Kafka) events. Some ETL and interaction with RPA. Likely to use microsites and/or UI fragments. Responsibilities: Create and document Test Approach Estimate and size testing of components (e.g. More ❯