22 of 22 Permanent Spark Streaming Jobs in London

data from diverse sources, transform it into usable formats, and load it into data warehouses, data lakes or lakehouses. Big Data Technologies: Utilize big data technologies such as Spark, Kafka, and Flink for distributed data processing and analytics. Cloud Platforms: Deploy and manage data solutions on cloud platforms such as AWS, Azure, or Google Cloud Platform (GCP), leveraging … for data manipulation and scripting. Strong understanding of data modelling concepts and techniques, including relational and dimensional modelling. Experience in big data technologies and frameworks such as Databricks, Spark, Kafka, and Flink. Experience in using modern data architectures, such as lakehouse. Experience with CI/CD pipelines, version control systems like Git, and containerization (e.g., Docker). Experience … deploying and managing data solutions. Strong problem-solving and analytical skills with the ability to diagnose and resolve complex data-related issues. SQL (for database management and querying) Apache Spark (for distributed data processing) Apache Spark Streaming, Kafka or similar (for real-time data streaming) Experience using data tools in at least More ❯

data from diverse sources, transform it into usable formats, and load it into data warehouses, data lakes or lakehouses. Big Data Technologies: Utilize big data technologies such as Spark, Kafka, and Flink for distributed data processing and analytics. Cloud Platforms: Deploy and manage data solutions on cloud platforms such as AWS, Azure, or Google Cloud Platform (GCP), leveraging … for data manipulation and scripting. Strong understanding of data modelling concepts and techniques, including relational and dimensional modelling. Experience in big data technologies and frameworks such as Databricks, Spark, Kafka, and Flink. Experience in using modern data architectures, such as lakehouse. Experience with CI/CD pipelines and version control systems like Git. Knowledge of ETL tools and … deploying and managing data solutions. Strong problem-solving and analytical skills with the ability to diagnose and resolve complex data-related issues. SQL (for database management and querying) Apache Spark (for distributed data processing) Apache Spark Streaming, Kafka or similar (for real-time data streaming) Experience using data tools in at least More ❯

engineering with a proven track record of building and managing real-time data pipelines across a track record of multiple initiatives. Expertise in developing data backbones using distributed streaming platforms (Kafka, Spark Streaming, Flink, etc.). Experience working with cloud platforms such as AWS, GCP, or Azure for real-time data ingestion and storage. More ❯

Key Responsibilities Design and implement real-time data pipelines using tools like Apache Kafka, Apache Flink, or Spark Streaming. Develop and maintain event schemas using Avro, Protobuf, or JSON Schema. Collaborate with backend teams to integrate event-driven microservices. Ensure data quality, lineage, and observability across streaming systems. Optimize performance and scalability of streaming applications. Implement CI/CD pipelines for data infrastructure. Monitor and troubleshoot production data flows and streaming jobs. Required Skills & Qualifications 3+ years of experience in data engineering or backend development. Strong programming skills in Python, Java, or Scala. Hands-on experience with Kafka, Kinesis, or similar messaging systems. Familiarity with stream processing frameworks like Flink … Kafka Streams, or Spark Structured Streaming. Solid understanding of event-driven design patterns (e.g., event sourcing, CQRS). Experience with cloud platforms (AWS, GCP, or Azure) and infrastructure-as-code tools. Knowledge of data modeling, schema evolution, and serialization formats. More ❯

Key Responsibilities Design and implement real-time data pipelines using tools like Apache Kafka, Apache Flink, or Spark Streaming. Develop and maintain event schemas using Avro, Protobuf, or JSON Schema. Collaborate with backend teams to integrate event-driven microservices. Ensure data quality, lineage, and observability across streaming systems. Optimize performance and scalability of streaming applications. Implement CI/CD pipelines for data infrastructure. Monitor and troubleshoot production data flows and streaming jobs. Required Skills & Qualifications 3+ years of experience in data engineering or backend development. Strong programming skills in Python, Java, or Scala. Hands-on experience with Kafka, Kinesis, or similar messaging systems. Familiarity with stream processing frameworks like Flink … Kafka Streams, or Spark Structured Streaming. Solid understanding of event-driven design patterns (e.g., event sourcing, CQRS). Experience with cloud platforms (AWS, GCP, or Azure) and infrastructure-as-code tools. Knowledge of data modeling, schema evolution, and serialization formats. More ❯

Key Responsibilities Design and implement real-time data pipelines using tools like Apache Kafka, Apache Flink, or Spark Streaming. Develop and maintain event schemas using Avro, Protobuf, or JSON Schema. Collaborate with backend teams to integrate event-driven microservices. Ensure data quality, lineage, and observability across streaming systems. Optimize performance and scalability of streaming applications. Implement CI/CD pipelines for data infrastructure. Monitor and troubleshoot production data flows and streaming jobs. Required Skills & Qualifications 3+ years of experience in data engineering or backend development. Strong programming skills in Python, Java, or Scala. Hands-on experience with Kafka, Kinesis, or similar messaging systems. Familiarity with stream processing frameworks like Flink … Kafka Streams, or Spark Structured Streaming. Solid understanding of event-driven design patterns (e.g., event sourcing, CQRS). Experience with cloud platforms (AWS, GCP, or Azure) and infrastructure-as-code tools. Knowledge of data modeling, schema evolution, and serialization formats. More ❯

market surveillance and compliance efforts. The platform leverages advanced analytics and machine learning to identify trading behaviors that could trigger regulatory attention. The tech stack includes Java, Python, Apache Spark (on Serverless EMR), AWS Lambda, DynamoDB, S3, SNS/SQS, and other cloud-native tools. You’ll work alongside a high-impact engineering team to build fault-tolerant … pipelines and services that process massive time-series datasets in both real-time and batch modes. Key Responsibilities: Design and build scalable, distributed systems using Java, Python, and Apache Spark Develop and optimize Spark jobs on AWS Serverless EMR for large-scale time-series processing Build event-driven and batch workflows using AWS Lambda, SNS/… non-technical stakeholders Qualifications: Strong backend software development experience, especially in distributed systems and large-scale data processing Advanced Java programming skills (multithreading, concurrency, performance tuning) Expertise in Apache Spark and Spark Streaming Proficiency with AWS services such as Lambda, DynamoDB, S3, SNS, SQS, and Serverless EMR Experience with SQL and NoSQL databases Hands More ❯

engineering with a proven track record of building and managing real-time data pipelines across a track record of multiple initiatives. Expertise in developing data backbones using distributed streaming platforms (Kafka, Spark Streaming, Flink, etc.). Experience working with cloud platforms such as AWS, GCP, or Azure for real-time data ingestion and storage. More ❯

below. AI techniques (supervised and unsupervised machine learning, deep learning, graph data analytics, statistical analysis, time series, geospatial analysis, NLP, sentiment analysis, pattern detection, etc.) Python, R, or Spark for data insights Data Bricks/Data QISQL for data access and processing (PostgreSQL preferred, but general SQL knowledge is important) Latest Data Science platforms (e.g., Databricks, Dataiku, AzureML … SageMaker) and frameworks (e.g., TensorFlow, MXNet, scikit-learn) Software engineering practices (coding standards, unit testing, version control, code review) Hadoop distributions (Cloudera, Hortonworks), NoSQL databases (Neo4j, Elastic), streaming technologies (Spark Streaming) Data manipulation and wrangling techniques Development and deployment technologies (virtualisation, CI tools like Jenkins, configuration management with Ansible, containerisation with Docker, Kubernetes) Data More ❯

and analytics teams to ensure seamless data flow across the organisation. You’ll own the full data lifecycle—from ingestion to transformation and delivery—leveraging modern cloud and streaming technologies. 🔧 Responsibilities Design and build scalable, low-latency real-time data pipelines Integrate diverse data sources and ensure data accuracy and integrity Develop and maintain data models optimised for … availability Implement logging, monitoring, and alerting for infrastructure health Partner with cross-functional teams to deliver robust data solutions 💡 What You’ll Bring Strong hands-on experience building streaming data platforms Deep understanding of tools like Kafka , Flink , Spark Streaming , etc. Proficiency in Python , Java , or Scala Cloud experience with AWS , GCP , or Azure More ❯

and analytics teams to ensure seamless data flow across the organisation. You’ll own the full data lifecycle—from ingestion to transformation and delivery—leveraging modern cloud and streaming technologies. 🔧 Responsibilities Design and build scalable, low-latency real-time data pipelines Integrate diverse data sources and ensure data accuracy and integrity Develop and maintain data models optimised for … availability Implement logging, monitoring, and alerting for infrastructure health Partner with cross-functional teams to deliver robust data solutions 💡 What You’ll Bring Strong hands-on experience building streaming data platforms Deep understanding of tools like Kafka , Flink , Spark Streaming , etc. Proficiency in Python , Java , or Scala Cloud experience with AWS , GCP , or Azure More ❯

and analytics teams to ensure seamless data flow across the organisation. You’ll own the full data lifecycle—from ingestion to transformation and delivery—leveraging modern cloud and streaming technologies. Responsibilities Design and build scalable, low-latency real-time data pipelines Integrate diverse data sources and ensure data accuracy and integrity Develop and maintain data models optimised for … availability Implement logging, monitoring, and alerting for infrastructure health Partner with cross-functional teams to deliver robust data solutions What You’ll Bring Strong hands-on experience building streaming data platforms Deep understanding of tools like Kafka , Flink , Spark Streaming , etc. Proficiency in Python , Java , or Scala Cloud experience with AWS , GCP , or Azure More ❯

optimal. Strong experience in different software testing paradigms to ensure consistency and correctness of our data. Bonus Skills Knowledge of real-time or stream processing systems (e.g. Kafka, Spark Streaming). Domain experience in energy, IoT, or working with unreliable/messy datasets. Frontend awareness—able to contribute to or reason about how data is visualized More ❯

optimal. Strong experience in different software testing paradigms to ensure consistency and correctness of our data. Bonus Skills Knowledge of real-time or stream processing systems (e.g. Kafka, Spark Streaming). Domain experience in energy, IoT, or working with unreliable/messy datasets. Frontend awareness—able to contribute to or reason about how data is visualized More ❯

optimal. Strong experience in different software testing paradigms to ensure consistency and correctness of our data. Bonus Skills Knowledge of real-time or stream processing systems (e.g. Kafka, Spark Streaming). Domain experience in energy, IoT, or working with unreliable/messy datasets. Frontend awareness—able to contribute to or reason about how data is visualized More ❯

across the team 🧰 What You’ll Need Strong experience leading data engineering teams in high-growth environments Deep expertise with real-time data processing tools (e.g. Kafka, Flink, Spark Streaming) Solid hands-on knowledge of cloud platforms (AWS, GCP or Azure) Strong proficiency in languages like Python, Java or Scala Familiarity with orchestration tools such as More ❯

across the team 🧰 What You’ll Need Strong experience leading data engineering teams in high-growth environments Deep expertise with real-time data processing tools (e.g. Kafka, Flink, Spark Streaming) Solid hands-on knowledge of cloud platforms (AWS, GCP or Azure) Strong proficiency in languages like Python, Java or Scala Familiarity with orchestration tools such as More ❯

across the team What You’ll Need Strong experience leading data engineering teams in high-growth environments Deep expertise with real-time data processing tools (e.g. Kafka, Flink, Spark Streaming) Solid hands-on knowledge of cloud platforms (AWS, GCP or Azure) Strong proficiency in languages like Python, Java or Scala Familiarity with orchestration tools such as More ❯

and understanding of current cyber security threats, actors and their techniques. Experience with data science, big data analytics technology stack, analytic development for endpoint and network security, and streaming technologies (e.g., Kafka, Spark Streaming, and Kinesis). Strong sense of ownership combined with collaborative approach to overcoming challenges and influencing organizational change. Amazon is More ❯

and understanding of current cyber security threats, actors and their techniques. Experience with data science, big data analytics technology stack, analytic development for endpoint and network security, and streaming technologies (e.g., Kafka, Spark Streaming, and Kinesis). Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to More ❯

GDPR, CCPA, SOC). Preferred Experience Proficiency in SQL, data modeling, ETL processes, and modern data warehousing. Experience with functional programming and real-time stream processing (e.g., Flink, Spark Streaming, or similar). Demonstrated ability to handle production environments processing tens of thousands of events per second from diverse sources. This is an opportunity to join More ❯

GDPR, CCPA, SOC). Preferred Experience Proficiency in SQL, data modeling, ETL processes, and modern data warehousing. Experience with functional programming and real-time stream processing (e.g., Flink, Spark Streaming, or similar). Demonstrated ability to handle production environments processing tens of thousands of events per second from diverse sources. This is an opportunity to join More ❯