4 of 4 Spark Streaming Jobs in the UK

team. What We’re Looking For Proficient in one of Python, Scala or Java, with strong experience in Big Data technologies such as: Spark, Hadoop etc. Practical knowledge of building real-time event streaming pipelines (e.g., Kafka, Spark Streaming, Kinesis). Proficiency … Points For Expertise in Data Modelling, schema design, and handling both structured and semi-structured data. Familiarity with distributed systems such as Hadoop, Spark, HDFS, Hive, Databricks. Exposure to AWS Lake Formation and automation of ingestion and transformation layers. Background in delivering solutions for highly regulated industries. Passion ...

gateway design, tenant isolation, and migration strategies for enterprise clients (parallel runs, phased cutovers). Modernize data architecture to support real-time streaming and graph-based modeling (Kafka, Flink) and unified analytics for operations and ML training. Transition release management from quarterly waterfall to continuous deployment with feature … multi-tenant isolation, Kubernetes/container orchestration, serverless patterns. Working knowledge of cloud data platforms (Snowflake, Databricks, BigQuery, Azure Synapse) and real-time streaming (Kafka, Flink, Spark Streaming). Familiarity with generative AI patterns (LLM fine-tuning, RAG, prompt engineering, function calling, agent frameworks ...

potentially work in either Glasgow or Edinburgh depending on client expectations. What You’ll Do Design and develop robust data pipelines across streaming and batch environments Lead engineering best practices including CI/CD, testing, and automation Contribute to architecture discussions and cloud migration strategies Collaborate with clients … What We’re Looking For Strong hands-on experience with Python, Java, or Scala Proficiency in AWS cloud environments and big data tech (Spark, Hadoop, Airflow) Solid understanding of SQL, ETL/ELT approaches, and data modelling techniques Experience building CI/CD pipelines with tools like Jenkins ...

Description Lead Backend Software Engineer – Data ServicesFederated Data Services | Distributed Systems | Apache Spark | DataFusion | Python | KubernetesLocation: London (2 days per week onsite)Rate: £700–£800 per day (Outside IR35)Duration: Initial 6-month contract with likely extensionOverviewWe're looking for an experienced Lead Backend Software Engineer to join … practices. Architect and develop federated data services capable of securely querying and processing data across multiple environments. Design high-performance data processing and streaming workflows using technologies such as Apache Spark, Apache DataFusion, or similar distributed query frameworks. Design and implement Kubernetes-based services and containerised ...

Lead Backend Software Engineer – Data Services Federated Data Services | Distributed Systems | Apache Spark | DataFusion | Python | Kubernetes Location: London (2 days per week onsite) Rate: £700–£800 per day (Outside IR35) Duration: Initial 6-month contract with likely extension Overview We're looking for an experienced Lead Backend Software … practices. Architect and develop federated data services capable of securely querying and processing data across multiple environments. Design high-performance data processing and streaming workflows using technologies such as Apache Spark, Apache DataFusion, or similar distributed query frameworks. Design and implement Kubernetes-based services and containerised ...