1 to 25 of 41 Batch Processing Jobs in London

/09/2025 End Date: 27/03/2026 Rate: £650 per day - PAYE via Umbrella Only Role Overview: We are looking for a seasoned Azure Batch Cloud Architect to lead the design and implementation of scalable, secure, and high-performance batch processing solutions in Microsoft Azure. This role is pivotal in supporting data-intensive … workloads and compute-heavy operations critical to financial services, including risk modelling, regulatory reporting, and large-scale data transformation. Key Responsibilities: Architect and implement Azure Batch solutions to support large-scale, parallel compute workloads. Design and optimise cloud-native batch processing pipelines for financial analytics, simulations, and reporting. Collaborate with data engineering, DevOps, and security teams to … ensure end-to-end automation, compliance, and observability. Define and enforce governance, cost optimisation, and security best practices for batch workloads in Azure. Lead proof-of-concept initiatives, performance tuning, and capacity planning for compute-intensive applications. Provide architectural guidance on containerised batch jobs using Docker and Azure Kubernetes Service (AKS) where applicable. Maintain documentation and provide training More ❯

UK's energy future while working with state-of-the-art technology in a highly innovative environment. What You'll Do: Design and implement HPC environments supporting interactive and batch processing workloads, including MPI-capable systems. Deliver HPC solutions that are operating system agnostic, with a strong focus on both Windows and Linux platforms. Architect platform-agnostic HPC … technical guidance and best practices for HPC deployment, scalability, and performance tuning. What We're Looking For: Proven experience in High Performance Computing design and implementation, including interactive environments, batch processing, and MPI. Expertise working across multiple operating systems, specifically Windows and Linux. Ability to design HPC solutions deployable on both MS Azure and AWS cloud platforms. Experience More ❯

UK's energy future while working with state-of-the-art technology in a highly innovative environment. What You'll Do: Design and implement HPC environments supporting interactive and batch processing workloads, including MPI-capable systems. Deliver HPC solutions that are operating system agnostic, with a strong focus on both Windows and Linux platforms. Architect platform-agnostic HPC … technical guidance and best practices for HPC deployment, scalability, and performance tuning. What We're Looking For: Proven experience in High Performance Computing design and implementation, including interactive environments, batch processing, and MPI. Expertise working across multiple operating systems, specifically Windows and Linux. Ability to design HPC solutions deployable on both MS Azure and AWS cloud platforms. Experience More ❯

systems are written in Elixir, but where necessary, we use small amounts of Python and Java where vendor SDKs require it. Designing, developing and maintaining realtime data streaming, and batch processing workloads. Providing on-call support as part of our teamwide rotation. Our on-call rotation is split across US and UK time zones, ensuring coverage whilst keeping … parallelism for speed/space performance tradeoffs. Bonus Experience: Exchange traded financial instruments. Statistics, discrete mathematics, linear algebra. Problem-solving and proof construction. For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy Notice at . California residents, please review the California Privacy Notice for information about certain legal rights More ❯

management, data lineage tracking, and access control to ensure data integrity and compliance. Optimize data pipelines and transformation logic using SAP Datasphere's capabilities to support real-time and batch processing needs. Collaborate with business and technical teams to translate data requirements into robust, reusable data products and services. Monitor and tune system performance, proactively identifying and resolving More ❯

management, data lineage tracking, and access control to ensure data integrity and compliance. * Optimize data pipelines and transformation logic using SAP SAC's capabilities to support real-time and batch processing needs. * Collaborate with business and technical teams to translate data requirements into robust, reusable data products and services. * Monitor and tune system performance, proactively identifying and resolving More ❯

management, data lineage tracking, and access control to ensure data integrity and compliance. * Optimize data pipelines and transformation logic using SAP Datasphere's capabilities to support real-time and batch processing needs. * Collaborate with business and technical teams to translate data requirements into robust, reusable data products and services. * Monitor and tune system performance, proactively identifying and resolving More ❯

they operate. Hands on Experience in Java , Spark , Scala ( or Java) Production scale hands-on Experience to write Data pipelines using Spark/any other distributed real time/batch processing. Strong skill set in SQL/Databases Strong understanding of Messaging tech like Kafka, Solace , MQ etc. Writing production scale applications to use the Caching technologies. Understanding of More ❯

they operate. Hands on Experience in Java , Spark , Scala ( or Java) Production scale hands-on Experience to write Data pipelines using Spark/any other distributed real time/batch processing. Strong skill set in SQL/Databases Strong understanding of Messaging tech like Kafka, Solace , MQ etc. Writing production scale applications to use the Caching technologies. Understanding of More ❯

they operate. Hands on Experience in Java , Spark , Scala ( or Java) Production scale hands-on Experience to write Data pipelines using Spark/any other distributed real time/batch processing. Strong skill set in SQL/Databases Strong understanding of Messaging tech like Kafka, Solace , MQ etc. Writing production scale applications to use the Caching technologies. Understanding of More ❯

building scalable, reliable, highly available and fault-tolerant systems, broad knowledge of software engineering and site reliability engineering in areas such as Large-Scale Data and Compute Infrastructure, Stream Processing, Kubernetes, High-Performance Networking, Observability and Infrastructure Automation. RESPONSIBILITIES Set the technology strategy for our cloud infrastructure, factoring in the AD/ADAS cloud development needs and our cloud … to automate routine tasks. NICE TO HAVES Master's degree in Computer Science. Experience working as a Software Engineer on data-intensive applications, data platforms, data pipelines, workflow orchestration, batch processing, and/or distributed databases. Previous experience in monitoring, tracking and optimising cloud compute and storage costs Experience working with RPC protocols and their formats, e.g., gRPC More ❯

needs within each department. MAIN RESPONSIBILITIES Database estate administration and monitoring Managing application system Disaster Recovery and high availability Managing backup and recovery strategies/automation of maintenance and batch processing Database performance tuning including TSQL/Procedural performance tuning (Optimizer and Index usage) Database design and architecture, development and administration Setting up best practice database development and More ❯

Data Scientists, Business Analysts Extensive experience using Python, SQL and AWS Define and enforce data architecture across mesh and domain-driven data products. Implement and govern real-time/batch processing (Kafka, Spark, Glue). Ensure strong metadata, cataloguing, and lineage practices across the enterprise. Lead teams of engineers across global hubs, mentoring and supporting high standards. Knowledge More ❯

server-side languages - Java being must have Proven experience with microservices architecture and scalable, distributed systems. Proficient in data technologies like MySQL , Hadoop , or Cassandra . Experience with batch processing , data pipelines , and data integrity practices. Familiarity with AWS services (e.g., RDS, Step Functions, EC2, Kinesis) is a plus. Solid understanding of unit , integration , and E2E testing paradigms. More ❯

needs within each department. MAIN RESPONSIBILITIES Database estate administration and monitoring Managing application system Disaster Recovery and high availability Managing backup and recovery strategies/automation of maintenance and batch processing Database performance tuning including TSQL/Procedural performance tuning (Optimizer and Index usage) Database design and architecture, development and administration Setting up best practice database development and More ❯

needs within each department. MAIN RESPONSIBILITIES Database estate administration and monitoring Managing application system Disaster Recovery and high availability Managing backup and recovery strategies/automation of maintenance and batch processing Database performance tuning including TSQL/Procedural performance tuning (Optimizer and Index usage) Database design and architecture, development and administration Setting up best practice database development and More ❯

and review code, paired programming, and debugging code related performance issues, SQL tuning, etc. Experience with AWS services such as S3, RDS, Aurora, NoSQL, MSK (Kafka). Experience with batch processing/ETL using Glue Jobs, AWS Lambda, and Step functions. Experience with designing bespoke & tailored front-end solutions (GUI based) using open source technology such as React More ❯

and review code, paired programming, and debugging code related performance issues, SQL tuning, etc. Experience with AWS services such as S3, RDS, Aurora, NoSQL, MSK (Kafka). Experience with batch processing/ETL using Glue Jobs, AWS Lambda, and Step functions. Experience with designing bespoke & tailored front-end solutions (GUI based) using open source technology such as React More ❯

/review code/paired programming/debugging code related performance issues, SQL tuning etc. Experience with AWS services such as S3, RDS, Aurora, NoSQL, MSK (Kafka) Experience with batch processing/ETL using Glue Jobs, AWS Lambda and Step functions. Experience with designing bespoke & tailored front-end solutions (GUI based) using open source technology such as React More ❯

ORM Frameworks: Entity Framework, Dapper. Experience in building and consuming web services using micro-services architecture using REST APIs. Experience in keeping multiple applications in synch - real time/batch processing. Experience of leading teams of technical specialists through to successful delivery. Experience of providing technical leadership and oversight with offshore and/or third-party delivery teams. Experience More ❯

ORM Frameworks: Entity Framework, Dapper Experience in building and consuming web services using micro-services architecture using REST APIs Experience in keeping multiple applications in synch - real time/batch processing. Experience of leading teams of technical specialists through to successful delivery Experience of providing technical leadership and oversight with offshore and/or third-party delivery teams Experience More ❯

ORM Frameworks: Entity Framework, Dapper. Experience in building and consuming web services using micro-services architecture using REST APIs. Experience in keeping multiple applications in synch - real time/batch processing. Experience of leading teams of technical specialists through to successful delivery. Experience of providing technical leadership and oversight with offshore and/or third-party delivery teams. Experience More ❯

ORM Frameworks: Entity Framework, Dapper. Experience in building and consuming web services using micro-services architecture using REST APIs. Experience in keeping multiple applications in synch - real time/batch processing. Experience of leading teams of technical specialists through to successful delivery. Experience of providing technical leadership and oversight with offshore and/or third-party delivery teams. Experience More ❯

mindset with strong organizational skills and attention to detail. Familiarity with tools like JIRA or ServiceNow. Certifications in ITIL, Linux, or cloud platforms (e.g., AWS, Azure). Experience with batch processing, data interfaces, or monitoring tools. Exposure to automation or scripting. Learn more about the LexisNexis Risk team and how we work We are committed to providing a More ❯

that serve,process and transform large quantities of data in the cloud Minimum Qualifications: Experience with Python (or other Object Oriented language) Experience building reliable, distributed applications for Data Processing or similar areas Hands-on experience developing cloud applications (e.g. AWS, GCP, Azure) Experience with technologies like BigQuery and SnowFlake Preferred Qualifications: Experience writing testable and modular code Experience … working in a fast-paced environment, collaborating across teams and disciplines Experience designing, deploying, and maintaining distributed systems Data pipelines, data platforms, workflow orchestration, batch processing Experience on building ML pipelines WHAT WE OFFER We are committed to creating a modern work environment that supports our employees and their loved ones. We offer many options of the best More ❯