1 to 25 of 55 Batch Processing Jobs in London

Solution Architect

Location
Greater London, England, United Kingdom
Posted 11/06/2026 Design, develop, test, deploy, and maintain scalable ETL/ELT pipelines for structured and unstructured data. Build batch and real-time processing solutions using BigQuery, Dataflow, Cloud Storage, Pub/Sub, and related GCP services. Develop data-processing components and integrations … transformation frameworks supporting full, incremental, and event-driven loads. Design data models and optimize BigQuery tables through partitioning, clustering, efficient SQL, and cost-aware processing patterns. Produce high-level and low-level technical designs, document assumptions, and contribute to architecture reviews. Implement data-quality checks, schema validation, reconciliation, audit ...

Azure Data Engineer (Fabric & Cloud Data Platform)

Location
Greater London, England, United Kingdom
enterprise data platform. In this role, you will implement a modern data lakehouse architecture capable of supporting raw and curated data zones, scalable batch processing, and future analytical workloads. You will play a key role in integrating our existing data pipelines, models, and transformation logic into a unified … Management: Maintain architectural segregation between raw storage (immutability, landing) and curated data zones (cleansed, conformed business layers) while securely handling identifiable data. Scalable Data Processing: Develop, tune, and monitor large-scale batch processing pipelines using Azure Databricks (Spark-based processing) and PySpark. Orchestration & Workflow: Build automated ...

Java Developer

Location
Greater London, England, United Kingdom
Description We are seeking an experienced Java Developer to join our team. This role requires experience in Java development, AWS, database management, and batch processing. As a Java Developer, you will contribute to designing, developing, and maintaining applications that meet business needs. Key Responsibilities Design and develop scalable Java … deploy cloud-based solutions, supporting availability and scalability Design, implement, and maintain database schemas, write SQL queries, and improve performance Develop and maintain batch processing systems to handle large volumes of data Maintain code quality through reviews, testing, and adherence to standards Work with cross-functional teams including ...

Cloud Platform Engineer (DevOps)

Location
Greater London, England, United Kingdom
infrastructure team. You will be responsible for the design and management of our AWS environment, ensuring high availability through robust networking, scalable data processing pipelines (ETL/ELT), and secure API management. You will bridge the gap between infrastructure stability, automated deployment via GitHub Actions, and a seamless developer … scaling of API Gateways and serverless functions to ensure reliable service delivery. Data Engineering Support: Design and maintain robust ETL/ELT pipelines and batch processing workflows to ensure seamless data flow across the platform. CI/CD & Automation: Own and improve GitHub Workflows, automating deployments and reducing ...

AI Solutions Architect

Location
Greater London, England, United Kingdom
enterprise data, APIs and microservices. Ensure AI solutions integrate cleanly into cloud-native PaaS and SaaS environments, with robust approaches to real-time and batch processing where required. Establish best practices for prompt engineering, model selection, grounding, evaluation and guardrails across AI solutions. Lead AI and LLM proof … guardrails. Experience working with cloud environments and enterprise platforms, with Azure experience preferred, and an understanding of PaaS, SaaS, real-time and batch processing patterns. Familiarity with Docker, CI/CD, infrastructure approaches, MLOps, LLMOps, AIOps, observability, versioning and production operations would be advantageous. Strong understanding of responsible ...

TECHNICAL LEAD

Location
Greater London, England, United Kingdom
knowledge. Object store V2 advanced configuration. Database and stored procedures connector advanced configuration. Oracle database knowledge. Queuing anypoint MQ, Oracle advanced queue. Advanced Mulesoft batch processing knowledge. MuleSoft advanced connector config and tuning. Salesforce integration knowledge. Object store V2 advanced configuration. Database and stored procedures connector advanced configuration. … Oracle database knowledge. Queuing anypoint MQ, Oracle advanced queue. Advanced MuleSoft batch processing knowledge. MuleSoft advanced connector config and tuning. Performance tuning experience. Munit advanced knowledge. Azure DEvops CI/CD knowledge and experience. Raml1.0 knowledge and building reusable fragments. and common CDM. REST API standards. Cloudhub 2.0 ...

Senior Oracle Database Administrator

Location
Greater London, England, United Kingdom
tuning, monitoring, and troubleshooting.· Administer and support Integration Broker, gateways, web services, and APIs.· Collaborate with functional teams, developers, and DBAs to resolve application, batch processing, search, and integration issues.· Support production releases, system monitoring, incident response, root cause analysis, and operational improvements.· Develop and maintain automation solutions … WebLogic, Tuxedo and JDK environments.· Support single sign-on (SSO), SSL certificates, and security integrations.· Assist with vulnerability management and infrastructure planning.· Optimize PeopleSoft batch processing, Application Engine programs, Process Scheduler performance, and reporting workloads.· Troubleshoot database performance issues impacting PeopleSoft online and batch operations.Performance Tuning & Optimization ...

Data Integration Engineer - Active SC

Hiring Organisation
Investigo Change Solutions
Location
London, United Kingdom
Employment Type
Contract
Contract Rate
GBP 540 Daily
week on site 6 months We are seeking experienced Data Integration Engineers to join a growing delivery team supporting enterprise data, API and batch processing services within a DevOps environment. The successful candidate will play a key role in the design, support and continuous improvement of critical data … integration platforms, ensuring the delivery of reliable, secure and high-performing services. Key Requirements Strong experience supporting and developing data integration, API and batch processing solutions within a DevOps environment. Hands-on experience with Java/Microservices, RabbitMQ, MongoDB/MySQL and Kubernetes. Experience building, supporting and monitoring ...

Data Engineer

Hiring Organisation
Harnham - Data & Analytics Recruitment
Location
London, South East England, United Kingdom
Employment Type
Full-Time
Salary
£600.00 - £800.00 per day
take ownership of the future-state data platform for a large financial services organisation. The business is looking to move away from traditional, slow batch processing towards a more modern, scalable and streaming-led data architecture. You'll play a key role in defining the data strategy, shaping … across Bronze, Silver and Gold layers. Establish a robust Gold layer to support reliable, scalable analytics and business reporting. Drive the transition from monthly batch processing towards more frequent, near-real-time and streaming-based data ingestion where appropriate. Take ownership of technical decisions, helping prioritise data ...

JDE Business Analyst - Sales & Distribution

Hiring Organisation
Mackenzie Jones IT
Location
UK - Remote, Euston, Greater London, United Kingdom
Employment Type
Permanent
Salary
£55000 - £60000/annum c£55k - £60K + Excellent Benefits Pa
functional experience supporting & configuring JDE EnterpriseOne 9.2, particularly Sales & Distribution modules - ideally 3 years' experience. Sales Order Management: Hands-on experience supporting sales order processing, order management workflows and associated Order-to-Cash processes. Distribution Management: Strong knowledge of JDE Pricing, Sales Order Management & Address Book functionality. JDE Functional … Configuration: Experience configuring & maintaining JDE application processing options, batch processes & UBE jobs. EDI Integration: Experience supporting Electronic Data Interchange (EDI) transactions, sales order integrations & customer/supplier interfaces - ideally 1 year's experience Business Analysis: Requirements gathering, process mapping, functional specifications & translating business needs into system solutions. Production ...

Lead Software Engineer - Production Support & SRE

Location
Greater London, England, United Kingdom
Rates application estate with hands-on software engineering. You will design and build automation, observability, and AI-driven tooling that delivers early detection of batch, risk, and market data issues, accelerates root cause analysis, and enable safe, automated remediation. You will serve as a trusted technical authority during major … principles and the responsible adoption of AI across the team. Key Responsibilities Serve as the technical production owner for critical Rates front-office services - batch/EOD processing, real-time pricing, risk computations, trade management, and market data pipelines Lead technical response and service restoration for P1/ ...

Data Engineer

Location
Greater London, England, United Kingdom
quality, integration, and unit testing Understanding of CI/CD pipelines to maintain code quality and facilitate deployments Knowledge of Data Engineering fundamentals, including Batch processing and ETL/ELT Basic understanding of various data architectures, such as Lakes and Warehouses Knowledge of data cleansing and transformation within ...

Data Engineer

Location
Greater London, England, United Kingdom
Experiment with open source tools Participate in community-building initiatives such as meetups Continuously learn and improve engineering practices What we’re looking for batch processing distributed data processing Experience with technologies such as: Kafka Flink Spark Databricks experience is a plus Experience implementing: Data Lakes ...

Mainframe Automation Consultant

Hiring Organisation
ECS Resource Group Ltd
Location
London, United Kingdom
Employment Type
Contract
Contract Rate
£550 - £600/day Outside IR35
looking for an experienced Mainframe Automation Engineer with strong CA7 expertise to support their client environments. You will take responsibility for maintaining and improving batch schedules, investigating production issues and developing automation that reduces manual intervention. This is a hands-on technical role for somebody who understands complex mainframe … workloads and the importance of reliable, timely processing. Key Responsibilities Create, maintain and amend CA7 job definitions, schedules, calendars, triggers and dependencies. Monitor batch processing, investigate failed or delayed jobs and resolve issues affecting production schedules. Develop and maintain automation to improve workload management, reduce repetitive tasks ...

Data Solutions Architect

Location
Greater London, England, United Kingdom
data platforms—particularly Snowflake—covering performance, workload management, security, access controls, cost governance, multi-environment deployment, data sharing, replication and disaster recovery. Define scalable batch, real-time, event-driven, API, orchestration and ELT/ETL integration patterns using modern data integration technologies. Establish standards for analytics engineering … Experience with BigQuery, Databricks, Synapse, Redshift or similar platforms is highly desirable. Expertise in scalable enterprise ELT/ETL, data integration, real-time and batch processing, event-driven systems, APIs and orchestration patterns. Experience with Fivetran, Workato, Boomi, Airbyte or similar integration platforms. Ability to evaluate technology options ...

Senior Data Engineer

Location
Greater London, England, United Kingdom
dimensional modelling, including Kimball-style fact tables, dimensions and slowly changing dimensions. Understanding of enterprise integration patterns including APIs, messaging, event-driven integration, batch processing and secure file transfer. Experience implementing automated testing, CI/CD, source control using Azure DevOps and infrastructure-as-code practices. Strong understanding ...

Software Engineer (Data Platforms)

Location
Greater London, England, United Kingdom
systems while preserving provenance and auditability. Build APIs and services that make processed data available to products, customers and AI systems. Develop workflows for batch processing, event-driven processing and long-running computational jobs. Create monitoring and alerting for pipeline failures, unexpected data changes and quality regressions. ...

Senior Python Engineer

Location
Greater London, England, United Kingdom
Overview My client arelooking for an experienced Python & SQL Developer with strong hands‐on experience in developing, maintaining, and optimising applications and data‐processing solutions. The ideal candidate will have excellent Python programming skills, advanced SQL expertise, and practical experience working in Linux environments and with Apache Kafka. Strong … database performance issues. Work with relational databases to retrieve, transform, validate, and process large volumes of data. Develop and maintain scripts, automation tasks, and batch‐processing jobs in Linux environments. Work with Apache Kafka for event‐driven processing, data streaming, and message‐based integrations. Monitor application performance ...

Senior Python Engineer (Hybrid in London)

Location
Greater London, England, United Kingdom
Overview My client arelooking for an experienced Python & SQL Developer with strong hands-on experience in developing, maintaining, and optimising applications and data-processing solutions. The ideal candidate will have excellent Python programming skills, advanced SQL expertise, and practical experience working in Linux environments and with Apache Kafka. Strong … database performance issues. Work with relational databases to retrieve, transform, validate, and process large volumes of data. Develop and maintain scripts, automation tasks, and batch-processing jobs in Linux environments. Work with Apache Kafka for event-driven processing, data streaming, and message-based integrations. Monitor application performance ...

Senior Data & Analytics Consultant

Location
Greater London, England, United Kingdom
learning techniques, including supervised ML, unsupervised ML, and LLMs Experience with deployment tools and processes, such as Azure Data Factory (ADF), Azure ML and batch processing Experience using DevOps or equivalent agile development environments 4+ years’ experience leading clients through complex technical projects — you know how to navigate ...

AWS Data Engineer

Location
Greater London, England, United Kingdom
native technologies, and applying robust software engineering practices. Key Responsibilities Design, develop, and optimise scalable, testable data pipelines using Python and Apache Spark. Implement batch workflows and ETL processes adhering to modern engineering standards. Develop Cloud-Based Workflows Orchestrate data workflows using AWS services such as Glue, EMR Serverless … experience writing clean, maintainable Python code, ideally using type hints, linters, and test frameworks such as pytest. Solid understanding of data engineering fundamentals including batch processing, schema evolution, and ETL pipeline development. Experience with—or strong interest in learning—Apache Spark for large-scale data processing. Familiarity with ...

Python Engineer Data Domains

Location
Greater London, England, United Kingdom
cutting edge What we’re looking for Software & Data Engineering Strong expertise in Python Passion for writing: clean reliable production-grade code Distributed Data Processing Experience building and operating: batch processing systems distributed data systems near real-time streaming pipelines Familiarity with: Kafka Flink Spark Databricks experience ...

Compute Platform Engineer II

Hiring Organisation
GSK
Location
Greater London, United Kingdom
Employment Type
Full Time
metadata-forward, CI/CD-driven platform represents and enables the entire application and analysis lifecycle including interactive development and explorations (notebooks), large-scale batch processing, observability and production application deployments. A Compute Platform Engineer II is a technical contributor who can consistently take a poorly defined business … business problems, Responsible for development of key components of a hybrid on-prem/cloud compute platform for both interactive and scalable batch computing and establishing of processes and workflows to transition existing HPC users and teams to this platform Responsible for code-driven environment, applications, and container/ ...

Senior OAT Tester

Hiring Organisation
scrumconnect ltd
Location
London, United Kingdom
Employment Type
Permanent
Salary
GBP 48,000 Annual
Execution Execute and coordinate testing for: Monitoring and alert validation Alert simulation and escalation testing Backup and restore validation Failover and disaster recovery testing Batch processing and restart validation Stability and soak testing Operational support procedures Security and access validation Environment verification Defect & Evidence Management Produce comprehensive operational ...

Lead MuleSoft Developer

Location
Greater London, England, United Kingdom
doing: Design, develop, and maintain APIs using API-led connectivity across System, Process, and Experience layers. Implement complex integration patterns including event-driven flows, batch processing, streaming, and orchestration. Lead development on projects, ensuring high-quality engineering and adherence to Capita C4EE standards. Review and contribute to design ...