12 of 12 Permanent Apache Spark Jobs in the North of England

Glue, ECR, EC2, EMR, DynamoDB, Lake Formation Experience with Infrastructure as Code (Terraform) Proficiency in Python, SQL, and familiarity with PySpark Experience with Apache Spark and data processing frameworks Hands-on experience with Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/or Amazon Athena ...

engineering best practices Skills & Experience * 3+ years' commercial experience in Data Engineering * Strong Java or Python development skills * Experience with Kafka, Flink or Spark * Experience building ETL/ELT or streaming data pipelines * Knowledge of AWS (Azure or GCP also considered) * Experience with Docker, Kubernetes and CI/ ...

platforms for performance and reliability Collaborate with Agile teams to deliver secure, high-quality solutions Write clean, tested, maintainable code Required Skills: Experience with Apache Spark, Hadoop or similar big data technologies AWS Apache NiFi Python or Java Experience building data pipelines and ETL processes Desirable ...

Work with container technologies Core Data Engineering Strong programming proficiency in Java or Python. Hands-on experience with at least one of: Kafka, Flink, Spark Solid understanding of stream processing concepts Understanding of software engineering best practices: testing, design patterns, CI/CD, Git. Experience building ETL/ ...

driven decision making. What youll be doing Building and optimising scalable data pipelines using Java (primary) and technologies like Kafka, Flink, or Spark Developing real-time streaming solutions and event-driven architectures using AWS services (Kinesis, MSK, Lambda, Glue) Contributing to data architecture, modelling, and best practices across modern ...

Work with container technologies Core Data Engineering Strong programming proficiency in Java or Python. Hands-on experience with at least one of: Kafka, Flink, Spark Solid understanding of stream processing concepts Understanding of software engineering best practices: testing, design patterns, CI/CD, Git. Experience building ETL/ ...

grade ML workflows. Experience designing CI/CD pipelines using Azure DevOps, GitHub Actions or similar tools. Advanced Python skills, with knowledge of SQL, Spark and testing approaches for ML. Understanding of data engineering concepts such as ETL/ELT, feature stores and lineage tracking. Awareness of security, governance ...

platforms that enable real‐time and batch data processing at scale. Responsibilities Drive platform capability improvements – continuously enhance distributed data platforms using knowledge of Spark, Databricks, Kafka, Pulsar, Cassandra, and object stores to maintain best‐in‐class capabilities. Enable application data consumption – understand how applications produce and consume data … structures, partitioning strategies, consistency models, and performance optimisation, as well as OLTP databases such as PostgreSQL and MySQL. Distributed compute frameworks – strong experience with Apache Spark and distributed data processing, including optimisation, performance tuning, and large‐scale batch/streaming workloads. Cloud‐native data platforms – demonstrated experience with ...

including data modelling, DAX and semantic model design Hands-on experience with Microsoft Fabric, particularly lakehouse architecture, notebooks and data pipelines Strong understanding of Apache Spark/PySpark and large-scale data processing Experience designing and operating modern data platforms Experience developing or enabling AI and machine learning ...

teams to drive outcomes Core skills (indicative) Production pipeline design and deployment experience Strong engineering capability with Python, SQL, plus big data tooling (e.g., Spark, and Java/Scala where relevant AWS, Azure, GCP Working pattern Hybrid working, with the team on client site or in the office ...

with large-scale and unstructured datasets. Python development experience, including technologies such as PySpark, Pandas and PyArrow. Expertise in distributed data processing frameworks, particularly Apache Spark. Experience building and maintaining ETL pipelines using technologies such as Apache Airflow, AWS Step Functions or Apache NiFi. Cloud platform experience ...

architecture and deployment. Key Responsibilities Data Pipeline Development Build, optimize, and maintain scalable data pipelines using Java, plus exposure to Python, Flink, Kafka, or Spark click apply for full job details ...