1 to 25 of 33 Batch Processing Jobs in London

/09/2025 End Date: 27/03/2026 Rate: £650 per day - PAYE via Umbrella Only Role Overview: We are looking for a seasoned Azure Batch Cloud Architect to lead the design and implementation of scalable, secure, and high-performance batch processing solutions in Microsoft Azure. This role is pivotal in supporting data-intensive … workloads and compute-heavy operations critical to financial services, including risk modelling, regulatory reporting, and large-scale data transformation. Key Responsibilities: Architect and implement Azure Batch solutions to support large-scale, parallel compute workloads. Design and optimise cloud-native batch processing pipelines for financial analytics, simulations, and reporting. Collaborate with data engineering, DevOps, and security teams to … ensure end-to-end automation, compliance, and observability. Define and enforce governance, cost optimisation, and security best practices for batch workloads in Azure. Lead proof-of-concept initiatives, performance tuning, and capacity planning for compute-intensive applications. Provide architectural guidance on containerised batch jobs using Docker and Azure Kubernetes Service (AKS) where applicable. Maintain documentation and provide training More ❯

data solutions that support diverse trading activities. Take full ownership of data products, guiding them from initial concept through to stable production. Design, build, and maintain systems for both batch processing and real-time streaming of time series datasets, ensuring high data quality and reliability. Develop APIs and data access methods for fast, intuitive retrieval of historical and … data solutions that support diverse trading activities. Take full ownership of data products, guiding them from initial concept through to stable production. Design, build, and maintain systems for both batch processing and real-time streaming of time series datasets, ensuring high data quality and reliability. Develop APIs and data access methods for fast, intuitive retrieval of historical and … Lake, Apache Iceberg), and relational databases Have a Bachelors or advanced degree in Computer Science, Mathematics, Statistics, Physics, Engineering, or equivalent work experience For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy Notice at . California residents, please review the California Privacy Notice for information about certain legal rights More ❯

UK's energy future while working with state-of-the-art technology in a highly innovative environment. What You'll Do: Design and implement HPC environments supporting interactive and batch processing workloads, including MPI-capable systems. Deliver HPC solutions that are operating system agnostic, with a strong focus on both Windows and Linux platforms. Architect platform-agnostic HPC … technical guidance and best practices for HPC deployment, scalability, and performance tuning. What We're Looking For: Proven experience in High Performance Computing design and implementation, including interactive environments, batch processing, and MPI. Expertise working across multiple operating systems, specifically Windows and Linux. Ability to design HPC solutions deployable on both MS Azure and AWS cloud platforms. Experience More ❯

UK's energy future while working with state-of-the-art technology in a highly innovative environment. What You'll Do: Design and implement HPC environments supporting interactive and batch processing workloads, including MPI-capable systems. Deliver HPC solutions that are operating system agnostic, with a strong focus on both Windows and Linux platforms. Architect platform-agnostic HPC … technical guidance and best practices for HPC deployment, scalability, and performance tuning. What We're Looking For: Proven experience in High Performance Computing design and implementation, including interactive environments, batch processing, and MPI. Expertise working across multiple operating systems, specifically Windows and Linux. Ability to design HPC solutions deployable on both MS Azure and AWS cloud platforms. Experience More ❯

systems are written in Elixir, but where necessary, we use small amounts of Python and Java where vendor SDKs require it. Designing, developing and maintaining realtime data streaming, and batch processing workloads. Providing on-call support as part of our teamwide rotation. Our on-call rotation is split across US and UK time zones, ensuring coverage whilst keeping … parallelism for speed/space performance tradeoffs. Bonus Experience: Exchange traded financial instruments. Statistics, discrete mathematics, linear algebra. Problem-solving and proof construction. For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy Notice at . California residents, please review the California Privacy Notice for information about certain legal rights More ❯

management, data lineage tracking, and access control to ensure data integrity and compliance. Optimize data pipelines and transformation logic using SAP Datasphere's capabilities to support real-time and batch processing needs. Collaborate with business and technical teams to translate data requirements into robust, reusable data products and services. Monitor and tune system performance, proactively identifying and resolving More ❯

management, data lineage tracking, and access control to ensure data integrity and compliance. * Optimize data pipelines and transformation logic using SAP SAC's capabilities to support real-time and batch processing needs. * Collaborate with business and technical teams to translate data requirements into robust, reusable data products and services. * Monitor and tune system performance, proactively identifying and resolving More ❯

management, data lineage tracking, and access control to ensure data integrity and compliance. * Optimize data pipelines and transformation logic using SAP Datasphere's capabilities to support real-time and batch processing needs. * Collaborate with business and technical teams to translate data requirements into robust, reusable data products and services. * Monitor and tune system performance, proactively identifying and resolving More ❯

they operate. Hands on Experience in Java , Spark , Scala ( or Java) Production scale hands-on Experience to write Data pipelines using Spark/any other distributed real time/batch processing. Strong skill set in SQL/Databases Strong understanding of Messaging tech like Kafka, Solace , MQ etc. Writing production scale applications to use the Caching technologies. Understanding of More ❯

building scalable, reliable, highly available and fault-tolerant systems, broad knowledge of software engineering and site reliability engineering in areas such as Large-Scale Data and Compute Infrastructure, Stream Processing, Kubernetes, High-Performance Networking, Observability and Infrastructure Automation. RESPONSIBILITIES Set the technology strategy for our cloud infrastructure, factoring in the AD/ADAS cloud development needs and our cloud … to automate routine tasks. NICE TO HAVES Master's degree in Computer Science. Experience working as a Software Engineer on data-intensive applications, data platforms, data pipelines, workflow orchestration, batch processing, and/or distributed databases. Previous experience in monitoring, tracking and optimising cloud compute and storage costs Experience working with RPC protocols and their formats, e.g., gRPC More ❯

who is confident, thorough and tenacious Bonus Points Hands-on, production experience with K8S Hands-on, production experience with cloud (AWS, Azure, GCP) Experience with data engineering - streaming and batch processing, spark, tryno, iceberg, clickhouse, parquet Are you a Do'er? Be your truest self. Work on your terms. Make a difference. We are home to a global More ❯

field Experience: Proven experience in ML model lifecycle management Core Competencies : Model lifecycle: You've got hands-on experience with managing the ML model lifecycle, including both online and batch processes Statistical Methodology: You have worked with GLMs and other machine learning algorithms and have in-depth knowledge of how they work Python: You have built and deployed production More ❯

field Experience: Proven experience in ML model lifecycle management Core Competencies : Model lifecycle: You've got hands-on experience with managing the ML model lifecycle, including both online and batch processes Statistical Methodology: You have worked with GLMs and other machine learning algorithms and have in-depth knowledge of how they work Python: You have built and deployed production More ❯

server-side languages - Java being must have Proven experience with microservices architecture and scalable, distributed systems. Proficient in data technologies like MySQL , Hadoop , or Cassandra . Experience with batch processing , data pipelines , and data integrity practices. Familiarity with AWS services (e.g., RDS, Step Functions, EC2, Kinesis) is a plus. Solid understanding of unit , integration , and E2E testing paradigms. More ❯

ORM Frameworks: Entity Framework, Dapper. Experience in building and consuming web services using micro-services architecture using REST APIs. Experience in keeping multiple applications in synch - real time/batch processing. Experience of leading teams of technical specialists through to successful delivery. Experience of providing technical leadership and oversight with offshore and/or third-party delivery teams. Experience More ❯

ORM Frameworks: Entity Framework, Dapper Experience in building and consuming web services using micro-services architecture using REST APIs Experience in keeping multiple applications in synch - real time/batch processing. Experience of leading teams of technical specialists through to successful delivery Experience of providing technical leadership and oversight with offshore and/or third-party delivery teams Experience More ❯

testable and modular code Experience working in a fast-paced environment, collaborating across teams and disciplines Experience designing, deploying, and maintaining distributed systems Data pipelines, data platforms, workflow orchestration, batch processing WHAT WE OFFER We are committed to creating a modern work environment that supports our employees and their loved ones. We offer many options of the best More ❯

project managers, and developers. The primary focus of this role will be to work through a pre-defined statement of work aimed at enhancing and automating various reports and batch processes. This is a hands-on delivery role, ideal for someone who can hit the ground running and enjoys a fast-paced, high-demand environment where priorities can change More ❯

that serve,process and transform large quantities of data in the cloud Minimum Qualifications: Experience with Python (or other Object Oriented language) Experience building reliable, distributed applications for Data Processing or similar areas Hands-on experience developing cloud applications (e.g. AWS, GCP, Azure) Experience with technologies like BigQuery and SnowFlake Preferred Qualifications: Experience writing testable and modular code Experience … working in a fast-paced environment, collaborating across teams and disciplines Experience designing, deploying, and maintaining distributed systems Data pipelines, data platforms, workflow orchestration, batch processing Experience on building ML pipelines WHAT WE OFFER We are committed to creating a modern work environment that supports our employees and their loved ones. We offer many options of the best More ❯

testable and modular code Experience working in a fast-paced environment, collaborating across teams and disciplines Experience designing, deploying, and maintaining distributed systems Data pipelines, data platforms, workflow orchestration, batch processing WHAT WE OFFER We are committed to creating a modern work environment that supports our employees and their loved ones. We offer many options of the best More ❯

passion for working in: Java Spring Boot Developing and using enterprise APIs Various testing methodologies System design at high scale and commercial experience with: SQL and NoSQL databases Async processing Cloud native applications Working in a Continuous Delivery environment Modern observability practices Nice to have Not vital, but you'll have the edge if you also have experience with … Grafana Prometheus Kotlin or a least the willingness to learn it Batch processing data pipelines or have worked in: an eCommerce organisation a shipping/logistics/exports organisation What you bring Agile: Test-Driven Development, collaboration and continuous delivery are your preferred engineering practices? We take the best bits of Lean, Scrum and Kanban too. Architecture: In More ❯

Python, SKLearn, XGBoost, SparkML, etc. Experience with deep learning frameworks like TensorFlow or PyTorch. Knowledge of ML model deployment options (e.g., Azure Functions, FastAPI, Kubernetes) for real-time and batch processing. Experience with CI/CD pipelines (e.g., DevOps pipelines, Git actions). Knowledge of infrastructure as code (e.g., Terraform, ARM Template, Databricks Asset Bundles). Understanding of advanced … machine learning techniques, including graph-based processing, computer vision, natural language processing, and simulation modeling. Experience with generative AI and LLMs, such as LLamaIndex and LangChain Understanding of MLOps or LLMOps. Familiarity with Agile methodologies, preferably Scrum We are actively seeking candidates for full-time, remote work within the UK. More ❯

Natural on mainframe or UNIX platforms. ADABAS Database: Strong understanding of ADABAS data structures, querying, and performance optimization. Preferred technical and professional experience JCL (Job Control Language): Experience with batch processing and job scheduling in mainframe environments is a plus. Mainframe Tools: Knowledge of tools like TSO/ISPF, FTP, are a plus. IBM is committed to creating More ❯

ministers and the nation's digitised history from the 11th to the early 21st century are all to be preserved in the system. We currently operate a series of batch-processing workflows constructed predominantly in Scala, Java, XSLT and XML Schema running in a Linux environment. You will influence the future design of the system and will be More ❯

Integration). Expert-level understanding of API design principles (REST, SOAP), API management platforms, and microservices architecture. Proficient in various integration patterns (e.g., ETL, EAI, B2B, real-time streaming, batch processing). Strong experience with cloud-native integration services and hybrid cloud environments. Expertise in various data formats (JSON, XML, EDI) and messaging protocols (Kafka, RabbitMQ, JMS). More ❯