12 of 12 Spark Streaming Jobs

data pipelines using Ab Initio (GDE, PDL, Conduct>IT, Express>IT).Build scalable batch and streaming workloads using Apache Spark (PySpark/Scala).Develop and orchestrate data ingestion flows using Apache NiFi for real-time and near-real-time use cases.Optimize ETL/ELT jobs … expertise in Ab Initio (GDE, PDL, Conduct>IT, Express>IT, EME) with a focus on parallel processing and component optimisation.Hands-on experience with Apache Spark, including Spark SQL, Spark Streaming, and performance tuning.Practical experience with Apache NiFi for flow design, processor tuning ...

improve collaboration and delivery.What We’re Looking ForStrong proficiency in Python, Scala, or Java with hands-on experience in Big Data technologies including Spark, Hadoop, or similar distributed processing frameworks.Practical experience building real-time event streaming pipelines using technologies such as Kafka, Spark Streaming … client environments.Bonus Points ForExpertise in data modelling, schema design, and working with structured and semi-structured datasets.Experience with distributed technologies including Hadoop, Spark, HDFS, Hive, Databricks, or AWS Lake Formation.Knowledge of automating cloud-based ingestion and transformation frameworks.Experience delivering data engineering solutions within highly regulated industries such ...

enhance collaboration and delivery.What We’re Looking ForStrong proficiency in Python, Scala, or Java, with hands-on experience of Big Data technologies including Spark, Hadoop, or similar distributed processing frameworks.Experience building real-time event streaming solutions using technologies such as Kafka, Spark Streaming … quality technical solutions.Bonus Points ForExpertise in data modelling, schema design, and working with structured and semi-structured datasets.Experience with distributed technologies including Hadoop, Spark, HDFS, Hive, Databricks, or AWS Lake Formation.Knowledge of automating ingestion and transformation frameworks within cloud-native data platforms.Experience delivering solutions within highly regulated industries ...

Future in the iGaming market. Technical leadership Defining and owning the data engineering architecture patterns Create Future uses across iGaming engagements – migration frameworks, streaming architectures, data integrity standards, and Lakehouse design Acting as the most senior technical authority on client engagements – making critical architecture decisions, resolving design disputes … consequences of getting it wrong AWS is your primary cloud and you can design across the full data engineering stack – from ingestion and streaming through to warehousing, serving, and cost optimisation You have deep experience with streaming architectures – Kinesis, Kafka/MSK, Flink, or Spark ...

Future in the iGaming market. Technical leadership Defining and owning the data engineering architecture patterns Create Future uses across iGaming engagements – migration frameworks, streaming architectures, data integrity standards, and Lakehouse design Acting as the most senior technical authority on client engagements – making critical architecture decisions, resolving design disputes … consequences of getting it wrong AWS is your primary cloud and you can design across the full data engineering stack – from ingestion and streaming through to warehousing, serving, and cost optimisation You have deep experience with streaming architectures – Kinesis, Kafka/MSK, Flink, or Spark ...

real-time infrastructure that powers decision-making across one of the world's largest digital asset exchanges. We operate at the intersection of streaming data, large-scale platform engineering, and AI-driven automation — processing billions of events daily across trading, compliance, and product systems.This role leads the team … responsible for Kraken's streaming and data platform layer — designing systems that move, transform, and serve data in real time. You'll own the architecture around stream processing (RisingWave, Flink), drive adoption of AI-powered automation and workflows across the data stack, and build the platform primitives that ...

Proven experience designing and building distributed systems and APIs at scale. Hands‐on experience with real‐time data processing frameworks (e.g. Kafka, Flink, Spark Streaming, Pulsar). Experience working with cloud platforms (AWS, GCP, or Azure) and containerisation/orchestration (Docker, Kubernetes). Strong understanding … data modelling, storage systems, and streaming/processing architectures. Familiarity with ML tooling such as MLflow, ZenML, or Metaflow. Familiarity with ML model lifecycle management (training, deployment, monitoring). Excellent collaboration and communication skills, with the ability to work cross‐functionally. Nice to have Experience building or working ...

time insight into live positions and risk. This platform underpins critical trading decisions and represents a major step forward in scalability, resilience, and streaming architecture across our technology estate. What You’ll Do Design and build distributed, real-time streaming components within a Flink-based architecture. … detects and publishes meaningful changes in liability. Ensure correctness in a high-throughput environment through robust handling of ordering, duplication, and idempotency. Optimise streaming jobs for latency, throughput, and resource efficiency. Collaborate closely with trading and product teams to translate domain requirements into scalable, reliable systems. Contribute ...

that are measurable and tied to business outcomesBuild — Design and validate solutions on both the customer side and the Treasure AI product sideDesign streaming architectures that handle real-world complexity: late-arriving events, schema drift, backpressure, partial failures, and sub-second SLA requirementsDrive technical decisions with clear rationale … software or data engineering experience, with at least 2 years in a customer-facing, embedded, or solutions engineering capacityStrong hands-on experience with streaming and event-driven architectures — Kafka, Flink, Kinesis, Spark Streaming, or equivalent (DynamoDB and MemoryDB/Redis)AWS Cloud Fluency - understand ...

surfaces (website, mobile app, and customer communications), and drive GenAI, Analytical, and DSML initiatives at petabyte scale. Our team builds and maintains:Streaming and batch data platforms that unlock data across TripadvisorETL pipelines and data models serving terabytes and growing datasets in Snowflake, BigQuery, and HiveOrchestration, scheduling … Java and Python code across our evolving platform stack — from data ingestion and event processing to REST services and internal tooling.Build and maintain streaming and batch processing systems (Flink, Spark, Kafka) that power real-time and analytical workloads.Take responsibility for all aspects of software engineering: design ...

surfaces (website, mobile app, and customer communications), and drive GenAI, Analytical, and DSML initiatives at petabyte scale. Our team builds and maintains: Streaming and batch data platforms that unlock data across Tripadvisor ETL pipelines and data models serving terabytes and growing datasets in Snowflake, BigQuery, and Hive Orchestration … Python code across our evolving platform stack — from data ingestion and event processing to REST services and internal tooling. Build and maintain streaming and batch processing systems (Flink, Spark, Kafka) that power real‐time and analytical workloads. Take responsibility for all aspects of software engineering: design ...

Networks training.**Nice-to-Have Qualifications*** Experience with telecom digital twin platforms.* Experience with vector databases, embeddings, semantic search, or RAG.* Experience with streaming data platforms such as Kafka, Pub/Sub, Flink, or Spark Streaming.* Experience supporting closed-loop automation, predictive assurance, or self-healing ...