3 of 3 Permanent Spark Streaming Jobs in the UK

team. What We’re Looking For Proficient in one of Python, Scala or Java, with strong experience in Big Data technologies such as: Spark, Hadoop etc. Practical knowledge of building real-time event streaming pipelines (e.g., Kafka, Spark Streaming, Kinesis). Proficiency … Points For Expertise in Data Modelling, schema design, and handling both structured and semi-structured data. Familiarity with distributed systems such as Hadoop, Spark, HDFS, Hive, Databricks. Exposure to AWS Lake Formation and automation of ingestion and transformation layers. Background in delivering solutions for highly regulated industries. Passion ...

gateway design, tenant isolation, and migration strategies for enterprise clients (parallel runs, phased cutovers). Modernize data architecture to support real-time streaming and graph-based modeling (Kafka, Flink) and unified analytics for operations and ML training. Transition release management from quarterly waterfall to continuous deployment with feature … multi-tenant isolation, Kubernetes/container orchestration, serverless patterns. Working knowledge of cloud data platforms (Snowflake, Databricks, BigQuery, Azure Synapse) and real-time streaming (Kafka, Flink, Spark Streaming). Familiarity with generative AI patterns (LLM fine-tuning, RAG, prompt engineering, function calling, agent frameworks ...

potentially work in either Glasgow or Edinburgh depending on client expectations. What You’ll Do Design and develop robust data pipelines across streaming and batch environments Lead engineering best practices including CI/CD, testing, and automation Contribute to architecture discussions and cloud migration strategies Collaborate with clients … What We’re Looking For Strong hands-on experience with Python, Java, or Scala Proficiency in AWS cloud environments and big data tech (Spark, Hadoop, Airflow) Solid understanding of SQL, ETL/ELT approaches, and data modelling techniques Experience building CI/CD pipelines with tools like Jenkins ...