126 to 133 of 133 Apache Spark Jobs in the UK excluding London

team leadership and upskilling responsibilities. Key Responsibilities Build and maintain Databricks Delta Live Tables (DLT) pipelines across Bronze → Silver → Gold layers, ensuring quality, scalability, and reliability. Develop and optimise Spark (PySpark) jobs for large-scale distributed processing. Design and implement streaming data pipelines with Kafka/MSK, applying best practices for late event handling and throughput. Use Terraform and … role) Mentor and upskill engineers, define coding standards, and embed engineering excellence across the team. What’s Expected Proven experience delivering end-to-end data pipelines in Databricks and Spark environments. Strong understanding of data modelling, schema evolution, and data contract management. Hands-on experience with Kafka, streaming architectures, and real-time processing principles. Proficiency with Docker, Terraform, and More ❯

team leadership and upskilling responsibilities. Key Responsibilities Build and maintain Databricks Delta Live Tables (DLT) pipelines across Bronze → Silver → Gold layers, ensuring quality, scalability, and reliability. Develop and optimise Spark (PySpark) jobs for large-scale distributed processing. Design and implement streaming data pipelines with Kafka/MSK, applying best practices for late event handling and throughput. Use Terraform and … role) Mentor and upskill engineers, define coding standards, and embed engineering excellence across the team. What’s Expected Proven experience delivering end-to-end data pipelines in Databricks and Spark environments. Strong understanding of data modelling, schema evolution, and data contract management. Hands-on experience with Kafka, streaming architectures, and real-time processing principles. Proficiency with Docker, Terraform, and More ❯

optimising end-to-end data pipelines using Azure Databricks, PySpark, ADF, and Delta Lake Implementing a medallion architecture - from raw to enriched to curated Working with Delta Lake and Spark for both batch and streaming data Collaborating with analysts to validate and refine datasets for reporting Applying CI/CD and DevOps best practices (Git, Azure DevOps) Enforcing data … governance using Azure Purview and Unity Catalog Optimising Spark jobs and SQL queries for performance and cost efficiency Exploring emerging tech like Kafka/Event Hubs and Knowledge Graphs What they're looking for: A strong communicator - someone who can build relationships across technical and business teams Hands-on experience building pipelines in Azure using Databricks, ADF, and PySpark More ❯

business stakeholders to understand data requirements. Optimize data workflows for performance and reliability. Ensure data quality, integrity, and security across systems. Work with large datasets using tools like Hadoop, Spark, and SAS. Integrate data from various sources including IBM Mainframe systems. Troubleshoot and resolve data-related issues efficiently. Required Skills & Experience:- Proven experience as a Data Engineer with a … foundation in data analysis. Expert-level proficiency in SAS for data manipulation and reporting. Working knowledge of IBM Mainframe systems and data structures. Advanced programming skills in Hadoop, SQL, Spark, and Python. Strong problem-solving and analytical skills. Experience with data modeling, warehousing, and performance tuning. Familiarity with Santander UK systems and processes is a strong advantage. Preferred Qualifications More ❯

PhD, or a Master's degree and experience building machine learning models or developing algorithms for business application Preferred Qualifications Experience with modeling tools such as R, scikit-learn, Spark MLLib, MxNet, Tensorflow, numpy, scipy etc. Experience with large scale distributed systems such as Hadoop, Spark etc. Amazon is an equal opportunities employer. We believe passionately that employing More ❯

ll do Lead the architecture and evolution of large scale streaming and backend systems, ensuring they are resilient, performant, and future-ready Apply deep knowledge of technologies such as Apache Flink, Kafka, and Spark to design reliable data pipelines and event driven systems Leverage your experience with modern backend languages such as Java, Scala, or Go to build … the curve What we’re looking for Experience operating at Staff Engineer or Principal Engineer level for atleast 2 years Expertise in streaming platforms such as Flink, Kafka, and Spark, with experience designing large-scale event driven architectures Strong background in backend engineering with one or more modern languages Ability to communicate complex technical concepts clearly to both engineers More ❯

ll do Lead the architecture and evolution of large scale streaming and backend systems, ensuring they are resilient, performant, and future-ready Apply deep knowledge of technologies such as Apache Flink, Kafka, and Spark to design reliable data pipelines and event driven systems Leverage your experience with modern backend languages such as Java, Scala, or Go to build … the curve What we’re looking for Experience operating at Staff Engineer or Principal Engineer level for atleast 2 years Expertise in streaming platforms such as Flink, Kafka, and Spark, with experience designing large-scale event driven architectures Strong background in backend engineering with one or more modern languages Ability to communicate complex technical concepts clearly to both engineers More ❯

ll do Lead the architecture and evolution of large scale streaming and backend systems, ensuring they are resilient, performant, and future-ready Apply deep knowledge of technologies such as Apache Flink, Kafka, and Spark to design reliable data pipelines and event driven systems Leverage your experience with modern backend languages such as Java, Scala, or Go to build … the curve What we’re looking for Experience operating at Staff Engineer or Principal Engineer level for atleast 2 years Expertise in streaming platforms such as Flink, Kafka, and Spark, with experience designing large-scale event driven architectures Strong background in backend engineering with one or more modern languages Ability to communicate complex technical concepts clearly to both engineers More ❯