4 of 4 Permanent Spark Streaming Jobs in East London

Key Responsibilities Design and implement real-time data pipelines using tools like Apache Kafka, Apache Flink, or Spark Streaming. Develop and maintain event schemas using Avro, Protobuf, or JSON Schema. Collaborate with backend teams to integrate event-driven microservices. Ensure data quality, lineage, and observability across streaming systems. Optimize performance and scalability of streaming applications. Implement CI/CD pipelines for data infrastructure. Monitor and troubleshoot production data flows and streaming jobs. Required Skills & Qualifications 3+ years of experience in data engineering or backend development. Strong programming skills in Python, Java, or Scala. Hands-on experience with Kafka, Kinesis, or similar messaging systems. Familiarity with stream processing frameworks like Flink … Kafka Streams, or Spark Structured Streaming. Solid understanding of event-driven design patterns (e.g., event sourcing, CQRS). Experience with cloud platforms (AWS, GCP, or Azure) and infrastructure-as-code tools. Knowledge of data modeling, schema evolution, and serialization formats. More ❯

and analytics teams to ensure seamless data flow across the organisation. You’ll own the full data lifecycle—from ingestion to transformation and delivery—leveraging modern cloud and streaming technologies. Responsibilities Design and build scalable, low-latency real-time data pipelines Integrate diverse data sources and ensure data accuracy and integrity Develop and maintain data models optimised for … availability Implement logging, monitoring, and alerting for infrastructure health Partner with cross-functional teams to deliver robust data solutions What You’ll Bring Strong hands-on experience building streaming data platforms Deep understanding of tools like Kafka , Flink , Spark Streaming , etc. Proficiency in Python , Java , or Scala Cloud experience with AWS , GCP , or Azure More ❯

optimal. Strong experience in different software testing paradigms to ensure consistency and correctness of our data. Bonus Skills Knowledge of real-time or stream processing systems (e.g. Kafka, Spark Streaming). Domain experience in energy, IoT, or working with unreliable/messy datasets. Frontend awareness—able to contribute to or reason about how data is visualized More ❯

across the team What You’ll Need Strong experience leading data engineering teams in high-growth environments Deep expertise with real-time data processing tools (e.g. Kafka, Flink, Spark Streaming) Solid hands-on knowledge of cloud platforms (AWS, GCP or Azure) Strong proficiency in languages like Python, Java or Scala Familiarity with orchestration tools such as More ❯