9 of 9 Spark Streaming Jobs in the UK excluding London

Accountabilities Develop robust, scalable data pipelines to serve the easyJet analyst and data science community. Highly competent hands-on experience with relevant Data Engineering technologies, such as Databricks, Spark, Spark API, Python, SQL Server, Scala. Work with data scientists, machine learning engineers and DevOps engineers to develop, develop and deploy machine learning models and algorithms aimed … partitioning. Hands-on IaC development experience with Terraform or CloudFormation. Understanding of ML development workflow and knowledge of when and how to use dedicated hardware. Significant experience with Apache Spark or any other distributed data programming frameworks (e.g. Flink, Hadoop, Beam) Familiarity with Databricks as a data and AI platform or the Lakehouse Architecture. Experience with data quality … access management, data privacy, handling of sensitive data (e.g. GDPR) Desirable Skills Experience in event-driven architecture, ingesting data in real time in a commercial production environment with Spark Streaming, Kafka, DLT or Beam. Understanding of the challenges faced in the design and development of a streaming data pipeline and the different options for More ❯

Kafka. • Configuring, deploying, and maintaining Kafka clusters to ensure high availability and scalability. • Integrating Kafka with other data processing tools and platforms such as Kafka Streams, Kafka Connect, Spark Streaming Schema Registry, Flink and Beam. • Collaborating with cross-functional teams to understand data requirements and design solutions that meet business needs. • Implementing security measures to protect … source kafka experience • Spend a lot of time on Disaster Recovery aspects • Read up and explain new features like KRAFT • Kafka Resiliency • Any Real time tech experience like Spark Required Skills Experience • Extensive Experience with Apache Kafka and real-time architecture including event driven frameworks. • Strong Knowledge of Kafka Streams, Kafka Connect, Spark Streaming More ❯

Kafka. Configuring, deploying, and maintaining Kafka clusters to ensure high availability and scalability. Integrating Kafka with other data processing tools and platforms such as Kafka Streams, Kafka Connect, Spark Streaming Schema Registry, Flink and Beam. Collaborating with cross-functional teams to understand data requirements and design solutions that meet business needs. Implementing security measures to protect … features, updates and industry practices. Required Skills Experience Extensive Experience with Apache Kafka and real-time architecture including event driven frameworks. Strong Knowledge of Kafka Streams, Kafka Connect, Spark Streaming, Schema Registry, Flink and Beam. Experience with cloud platform such as GCP Pub/Sub. Excellent problem-solving skills. Knowledge & Experience/Qualifications Knowledge of Kafka More ❯

domain. This role is fully hands-on, focused on building and optimising large-scale data pipelines, dataflows, semantic models, and lakehouse components. Key Responsibilities Design, build and optimise Spark-based data pipelines for batch and streaming workloads Develop Fabric dataflows, pipelines, and semantic models Implement complex transformations, joins, aggregations and performance tuning Build and optimise Delta … closely with analysts and stakeholders to translate requirements into scalable technical solutions Troubleshoot and improve reliability, latency and workload performance Essential Skills Strong hands-on experience with PySpark, Spark SQL, Spark Streaming, DataFrames Microsoft Fabric (Fabric Spark jobs, dataflows, pipelines, semantic models) Azure: ADLS, cloud data engineering, notebooks Python programming; Java exposure More ❯

of ETL/ELT pipelines , data modeling , and data warehousing . Experience with cloud platforms (AWS, Azure, GCP) and tools like Snowflake, Databricks, or BigQuery . Familiarity with streaming technologies (Kafka, Spark Streaming, Flink) is a plus. Tools & Frameworks: Airflow, dbt, Prefect, CI/CD pipelines, Terraform. Mindset: Curious, data-obsessed, and driven to More ❯

and data warehousing principles. Experience working with cloud platforms such as AWS, Azure, or GCP. Exposure to modern data tools such as Snowflake, Databricks, or BigQuery. Familiarity with streaming technologies (e.g., Kafka, Spark Streaming, Flink) is an advantage. Experience with orchestration and infrastructure tools such as Airflow, dbt, Prefect, CI/CD pipelines, and More ❯

API & Micro services Architecture: Comfortable working with REST APIs and micro services architectures. Real-time Stream Processing: Understanding of real-time stream processing frameworks (e.g., PubSub, Kafka, Flink, Spark Streaming). BI Tools & Visualisation Platforms: Experience supporting BI tools or visualization platforms (e.g. Looker, Grafana, PowerBI etc.). Data Pipelines & APIs: Experience in building and maintaining … both batch and streaming data pipelines and APIs. ETL/ELT Processes: Exposure to ETL/ELT processes in medium-to-large scale data environments (experience handling millions of records/events daily is a plus). Preferred Skills Python: Knowledge for data automation and scripting. Containerization: Familiarity with tools like Docker and Kubernetes. Workflow/Orchestration Tools More ❯

Technical Skills Required • Hands-on software development experience with Python and experience with modern software development and release engineering practices (e.g. TDD, CI/CD). • Experience with Apache Spark or any other distributed data programming frameworks. • Comfortable writing efficient SQL and debugging on cloud warehouses like Databricks SQL or Snowflake. • Experience with cloud infrastructure like AWS or … Hands-on development experience in an airline, e-commerce or retail industry • Experience in event-driven architecture, ingesting data in real time in a commercial production environment with Spark Streaming, Kafka, DLT or Beam. • Experience implementing end-to-end monitoring, quality checks, lineage tracking and automated alerts to ensure reliable and trustworthy data across the platform. More ❯

and performance tuning API development using Django or Flask Solid understanding of legacy-to-cloud migration Familiarity with RDBMS (Sybase IQ, Oracle, SQL Server, PostgreSQL) Bonus: Kafka/Spark Streaming, data governance exposure More ❯