126 to 138 of 138 Apache Spark Jobs in England

that are accurate, maintainable and easy for the business to use. Key Responsibilities: Build, enhance and maintain data pipelines within a Databricks environment. Use Spark, SQL and Python or Scala to support data transformation and automation. Develop structured data layers to support reporting, analytics and wider business use. Extract … What We're Looking For: Experience working as a Data Engineer within a modern cloud data environment. Strong hands-on experience with Databricks and Apache Spark. Experience building structured data layers, ideally within a Bronze, Silver and Gold architecture. Strong SQL skills for transformation, validation and analysis. Python ...

code review, source control, build, test, deploy, and operations Awesome If You: Are experienced in Rust/Java/Kotlin Have experience with AWS, Apache Kafka, Kafka Streams, Apache Beam/Flink/Spark – especially deployment, monitoring & debugging Have experience with productisation of Machine Learning research projects ...

handling edge cases, improving data quality and solving complex data challenges. The environment is modern and cloud based, with exposure to technologies such as Spark, Snowflake and BigQuery across AWS, GCP and Azure. You will also have the opportunity to support and mentor other engineers through collaboration, code reviews … Looking For Strong Python and SQL expertise gained within large scale production environments Experience delivering complex data transformation projects end to end Experience with Spark, Snowflake, BigQuery or similar technologies Exposure to cloud environments such as AWS, GCP or Azure Comfortable working with large, messy and evolving datasets Experience ...

workloads. Practical experience working with AWS cloud services, including compute, storage, and security tooling. Infrastructure-as-code experience using Terraform. Distributed data processing using Apache Spark. Workflow scheduling and orchestration with Apache Airflow. Containerisation and environment management using Docker. CI/CD pipeline implementation and maintenance using GitLab. ...

infrastructure. Analyses and reports on test activities, results, issues and risks. Production data analysis to identify root cause of production environment issues. Using Apache Airflow for task orchestration and reviewing output logs. Basic understanding of EMR and reviewing output logs. Basic understanding of data processing using Apache Spark. ...

Role Summary With specific expertise in AI, Data Science and Natural Language Processing (NLP) domain, this pre‐sales role collates and synthesises requirements to design and present innovative end‐to‐end solution options to enable ...

working We need an AI Engineer with a strong data engineering core to build AI‐powered analytics for front‐office trading teams using Databricks, Spark, and modern LLM workflows. Key Responsibilities: Design and deliver AI‐driven analytics for traders, including seasonality, correlations, regression, forecasting, and scenario modelling. Build scalable … reusable data pipelines in Databricks using PySpark, Spark SQL, Delta Lake, and Unity Catalog. Perform statistical and econometric analysis on large market and fundamentals time‐series datasets. Work directly with traders and analysts to turn ambiguous questions into production‐ready tools and clear insights. Implement LLM and agentic workflows ...

maintaining, and supporting the reliability, architecture, and evolution of our production machine learning and delivery platform; Designing, operating, and improving large scale Databricks/Spark pipelines and workflows Partner with our data scientists to productionize new data and ML-powered products; Leading improvements on the software integration layers between … production systems Experience integrating ML models into reliable production data pipelines, including batch inference and model serving patterns Experience building and operating Databricks (or Spark) based pipelines Experience with SQL Experience with at least one major cloud infrastructure provider (GCP, AWS, Azure) Experience with supporting business-critical systems with ...

teams and setting best practice standards Designing data models, ETL pipelines, warehousing solutions and distributed systems Building and supporting real-time streaming platforms using Apache Flink and related technologies Ensuring compliance with industry standards and government regulations Tracking delivery against agreed designs and managing design deviations Supporting pre-sales … data architecture on complex, secure programmes Strong hands-on AWS experience (EC2, S3, RDS, Lambda, Redshift); Azure or GCP a bonus Proven experience using Apache Flink for real-time, high-throughput data processing Experience leading, mentoring and developing technical teams Confident engaging with senior technical and non-technical stakeholders ...

scalability Build and integrate RESTful microservices within a complex distributed architecture Work with large-scale data sets, leveraging SQL and big data tools (e.g., Spark, Kafka) Influence system design and contribute to the technical roadmap Collaborate closely with a highly skilled engineering team to deliver innovative solutions What … experience building and consuming RESTful services Solid SQL skills and familiarity with large-scale data processing Knowledge of big data technologies such as Spark and Kafka Experience in finance is preferred, but not essential Strong academic background, ideally in Computer Science or a related discipline If this opportunity excites ...

Senior Software Engineer - Real-Time Data (Kafka/Spark) Chester | Hybrid working | 12-month contract | Inside IR35 | Global Investment Bank A leading global investment bank is looking for a hands-on Senior Engineer to build high-performance, real-time data platforms supporting regulatory compliance across global operations. This … processing , ideal for someone who enjoys solving complex data challenges at scale. What You'll Do Design and build Kafka-based streaming pipelines and Spark processing jobs Develop low-latency, high throughput systems handling large-scale datasets Create data matching and reconciliation solutions across multiple sources Work on event ...

Senior Software Engineer - Real-Time Data (Kafka/Spark) Chester Hybrid working 12-month contract Inside IR35 Global Investment Bank A leading global investment bank is looking for a hands‐on Senior Engineer to build high‐performance, real‐time data platforms supporting regulatory compliance across global operations. This … processing, ideal for someone who enjoys solving complex data challenges at scale. What You'll Do Design and build Kafka‐based streaming pipelines and Spark processing jobs Develop low‐latency, high throughput systems handling large‐scale datasets Create data matching and reconciliation solutions across multiple sources Work on event ...

Senior Software Engineer - Real-Time Data to join their team in Chester. This role involves designing and building Kafka-based streaming pipelines and Spark processing jobs, focusing on delivering low-latency, high-throughput systems. The ideal candidate will have strong hands-on experience with real-time data processing ...