1 to 25 of 26 Batch Processing Jobs in the UK

data solutions that support diverse trading activities. Take full ownership of data products, guiding them from initial concept through to stable production. Design, build, and maintain systems for both batch processing and real-time streaming of time series datasets, ensuring high data quality and reliability. Develop APIs and data access methods for fast, intuitive retrieval of historical and … data solutions that support diverse trading activities. Take full ownership of data products, guiding them from initial concept through to stable production. Design, build, and maintain systems for both batch processing and real-time streaming of time series datasets, ensuring high data quality and reliability. Develop APIs and data access methods for fast, intuitive retrieval of historical and … Lake, Apache Iceberg), and relational databases Have a Bachelors or advanced degree in Computer Science, Mathematics, Statistics, Physics, Engineering, or equivalent work experience For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy Notice at . More ❯

UK's energy future while working with state-of-the-art technology in a highly innovative environment. What You'll Do: Design and implement HPC environments supporting interactive and batch processing workloads, including MPI-capable systems. Deliver HPC solutions that are operating system agnostic, with a strong focus on both Windows and Linux platforms. Architect platform-agnostic HPC … technical guidance and best practices for HPC deployment, scalability, and performance tuning. What We're Looking For: Proven experience in High Performance Computing design and implementation, including interactive environments, batch processing, and MPI. Expertise working across multiple operating systems, specifically Windows and Linux. Ability to design HPC solutions deployable on both MS Azure and AWS cloud platforms. Experience More ❯

Proven experience with international clients, particularly in US, UK, European, or Australian markets, strongly preferred Technical Expertise Deep expertise in AWS and GCP cloud services, covering storage, compute, data processing, analytics, and serverless platforms (e.g., AWS S3/Redshift/Glue/Lambda, GCP BigQuery/Cloud Storage/Dataflow/Cloud Functions) Strong proficiency in SQL, Python, and … Grafana), and automated scaling Proven experience designing and supporting multi-tenant SaaS data platforms with strategies for data partitioning, tenant isolation, and cost management Exposure to real-time data processing technologies such as Kafka, Kinesis, Flink, or Spark Streaming, alongside batch processing capabilities Strong knowledge of SaaS compliance practices and security frameworks Core Competencies Excellent problem-solving More ❯

or .NET like web applications on Azure cloud infrastructure with Docker, Kubernetes, automation testing and DevOps (ideally terraform + github actions). Create, monitor and maintain data pipelines and batch processes (databricks, pyspark, lakehouses, kafka). Productionise quant models into software applications, ensuring robust day-to-day operation, monitoring and back testing are in place. Stakeholder engagement: Elicit, validate More ❯

teams • Strong hands-on expertise in cloud platforms (GCP, AWS, or Azure) with emphasis on cloud-native architectures • Deep understanding of data engineering - ETL/ELT, real-time/batch processing, data lakes/warehouses, and data governance • Track record of delivering complex, scalable systems that drive measurable business impact • Strong software engineering foundation across different languages and More ❯

systems are written in Elixir, but where necessary, we use small amounts of Python and Java where vendor SDKs require it. Designing, developing and maintaining realtime data streaming, and batch processing workloads. Providing on-call support as part of our teamwide rotation. Our on-call rotation is split across US and UK time zones, ensuring coverage whilst keeping … algorithms. Analysis of concurrency and parallelism for speed/space performance tradeoffs. Bonus Experience: Exchange traded financial instruments. Problem-solving and proof construction. For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy Notice at . More ❯

sustainable, and agile data analytics platform for a data driven digital bank. You will be using a range of database technologies and a combination of event driven techniques and batch processing methodologies. In addition you will be expanding and optimizing our cloud-based data & data pipelines as part of a cross functional team. As a Data Engineer at More ❯

imaging specialists, building robust data pipelines that power AI workflows and integrate seamlessly with our robotic systems Key Responsibilities Design and implement scalable data pipelines for real-time and batch processing Build and maintain data lakes, warehouses, and streaming platforms Develop architectures that support AI workflows and integration with clinical/robotic software Data Operations and Quality Ensure More ❯

side languages - Java being must have. Experience working in Kubernetes environment. Proven experience with microservices architecture and scalable, distributed systems. Proficient in data technologies like MySQL . Experience with batch processing , data pipelines , and data integrity practices. Familiarity with AWS services (e.g., RDS, Step Functions, EC2, Kinesis) is a plus. Solid understanding of unit , integration , and E2E testing More ❯

tolerant workflows using AWS services such as Lambda, Kinesis, SQS, DynamoDB Streams, and EventBridge. -Implement processors that ensure data consistency across DynamoDB, PostgreSQL (Aurora), OpenSearch, and BigQuery. -Modernise legacy batch processes into stream-first architectures. -Build and integrate APIs to enable smooth publishing and consumption of events across systems. -Collaborate with engineers on greenfield and existing projects, balancing speed More ❯

tolerant workflows using AWS services such as Lambda, Kinesis, SQS, DynamoDB Streams, and EventBridge. -Implement processors that ensure data consistency across DynamoDB, PostgreSQL (Aurora), OpenSearch, and BigQuery. -Modernise legacy batch processes into stream-first architectures. -Build and integrate APIs to enable smooth publishing and consumption of events across systems. -Collaborate with engineers on greenfield and existing projects, balancing speed More ❯

field Experience: Proven experience in ML model lifecycle management Core Competencies : Model lifecycle: You've got hands-on experience with managing the ML model lifecycle, including both online and batch processes Statistical Methodology: You have worked with GLMs and other machine learning algorithms and have in-depth knowledge of how they work Python: You have built and deployed production More ❯

Good start-up. This is a 3-month contract initially with scope for extension. In this role, you will design, build, and maintain robust data pipelines using streaming and batch processing technologies and help to lead the modernization of their data lakehouse. You will implement and manage secure cloud infrastructure using AWS, championing a culture of data quality … security, and technical excellence. Key Responsibilities: Architect and maintain data pipelines (streaming & batch) Build and manage scalable data storage solutions (data lakehouse) Design and maintain secure AWS cloud infrastructure Implement and manage CI/CD and DevOps pipelines Champion data quality, security, and best practices Collaborate with cross-functional teams Implement and manage MLOps capabilities Essential Skills: Advanced Python More ❯

Integration). Expert-level understanding of API design principles (REST, SOAP), API management platforms, and microservices architecture. Proficient in various integration patterns (e.g., ETL, EAI, B2B, real-time streaming, batch processing). Strong experience with cloud-native integration services and hybrid cloud environments. Expertise in various data formats (JSON, XML, EDI) and messaging protocols (Kafka, RabbitMQ, JMS). More ❯

5. Support internal and external audits, ensuring compliance and documentation. 6. Collaborate with business and operations teams to prioritize defect fixes and enhancements. 7. Handle event-driven and scheduled batch processes, ensuring smooth operations. 8. Troubleshoot technical issues related to Java, Middleware, Unix/Windows, and databases. 9. Participate in on-call support for real-time issue triaging and More ❯

Key Responsibilities Design, develop, test, and deploy custom extensions for general journal processing, including data entities like LedgerJournalHeader and LedgerJournalLine. Implement business logic for transaction validation, reconciliation, and posting workflows within D365 Finance. Collaborate with integration teams. Troubleshoot and optimize performance for high volume transaction processing. Document technical specifications and support deployment in cloud environments. Required Skills and Experience … D365 tools like Visual Studio, LCS, and Azure DevOps for CI/CD, along with unit testing frameworks (eg, SysTest) and debugging. Familiarity with financial data handling, error management, batch processing, and integration patterns (eg, DMF, OData). Experience with D365 workflows, security roles, and reporting tools (eg, SSRS). Experience in working with PowerBI reports and creation More ❯

with a passion for technology and finance to design and implement innovative solutions within D365 Finance & Operations. Key Responsibilities: - Design, develop, test, and deploy custom extensions for general journal processing, including data entities like LedgerJournalHeader and LedgerJournalLine. - Implement business logic for transaction validation, reconciliation, and posting workflows within D365 Finance. - Collaborate with integration teams to ensure seamless functionality and … D365 tools like Visual Studio, LCS, and Azure DevOps for CI/CD, along with unit testing frameworks (e.g., SysTest) and debugging. - Familiarity with financial data handling, error management, batch processing, and integration patterns (e.g., DMF, OData). - Experience with D365 workflows, security roles, and reporting tools (e.g., SSRS). - Experience in working with PowerBI reports and creation More ❯

day (Inside IR35) Remote/must be UK based 1 2 months We're looking for an experienced D365 Finance & Operations Developer to lead the development of custom journal processing extensions. This role suits someone with strong X++ skills and a deep understanding of D365 Finance, especially in high-volume, cross-functional environments. Key Requirements: Experience in D365 F … and CI/CD pipelines. Proficient in Power BI: report creation, configuration, and export. Skilled in workflows, security roles, SSRS, and unit testing (SysTest). Familiar with DMF, OData, batch processing, and error handling. Ability to optimize performance for high-volume transactions. Strong documentation and collaboration skills. Nice to have: Integration experience across systems and platforms. If you More ❯

per day (Inside IR35) Remote/must be UK based 12 months We're looking for an experienced D365 Finance & Operations Developer to lead the development of custom journal processing extensions. This role suits someone with strong X++ skills and a deep understanding of D365 Finance, especially in high-volume, cross-functional environments. Key Requirements: 5+ years in D365 … and CI/CD pipelines. Proficient in Power BI: report creation, configuration, and export. Skilled in workflows, security roles, SSRS, and unit testing (SysTest). Familiar with DMF, OData, batch processing, and error handling. Ability to optimize performance for high-volume transactions. Strong documentation and collaboration skills. Nice to have: Integration experience across systems and platforms. If you More ❯

performance spikes and deadlocks Develop BI reports and semantic models (Power BI/DAX) Govern data assets in Azure and ensure compliance (GDPR) Tech Environment: High-availability, distributed deployment Batch processing + real-time inserts/updates .NET applications using ORM and stored procedures Azure SQL + Storage Containers Looking for someone who thrives in fast-paced environments More ❯

working with modern tools in a fast-moving, high-performance environment. Your responsibilities may include: Build and maintain scalable, efficient ETL/ELT pipelines for both real-time and batch processing. Integrate data from APIs, streaming platforms, and legacy systems, with a focus on data quality and reliability. Design and manage data storage solutions, including databases, warehouses, and lakes. … Leverage cloud-native services and distributed processing tools (e.g., Apache Flink, AWS Batch) to support large-scale data workloads. Operations & Tooling Monitor, troubleshoot, and optimize data pipelines to ensure performance and cost efficiency. Implement data governance, access controls, and security measures in line with best practices and regulatory standards. Develop observability and anomaly detection tools to support Tier … maintaining scalable ETL/ELT pipelines and data architectures. Hands-on expertise with cloud platforms (e.g., AWS) and cloud-native data services. Comfortable with big data tools and distributed processing frameworks such as Apache Flink or AWS Batch. Strong understanding of data governance, security, and best practices for data quality. Effective communicator with the ability to work across technical More ❯

service transition from a third-party provider (on-site in London) to an in-house IT infrastructure team in India. This 24/7 service encompasses system monitoring and batch processing, involving the seamless transfer of responsibilities and technical knowledge from a 13-person London-based team to a 17-strong team in India. The successful candidate will More ❯

service transition from a third-party provider (on-site in London) to an in-house IT infrastructure team in India. This 24/7 service encompasses system monitoring and batch processing, involving the seamless transfer of responsibilities and technical knowledge from a 13-person London-based team to a 17-strong team in India. The successful candidate will More ❯

system changes, coordinate UAT, and ensure solutions meet objectives. Support system updates, patches, and upgrades with minimal disruption. Administer applications, maintaining security, integrity, and performance, and monitor integrations and batch processes. Promote knowledge sharing within the team to prevent single points of failure. Keep technical skills up to date and support ad hoc initiatives as needed. Qualifications & Experience: Proven More ❯

full-time, remote UK-based role. ***Unfortunately we can't consider candidates that require sponsorship*** Key Responsibilities: Research and deploy LLM-based solutions (e.g., LangChain, Mastra.ai, Pydantic) for document processing, summarization, and clinical Q&A systems. Develop and optimize predictive models using scikit-learn, PyTorch, TensorFlow, and XGBoost. Design robust data pipelines using tools like Spark and Kafka for … real-time and batch processing. Manage ML lifecycle with tools such as Databricks , MLflow , and cloud-native platforms (Azure preferred). Collaborate with engineering teams to ensure scalable, secure ML infrastructure aligned with compliance standards (e.g., ISO27001). Ensure data governance, particularly around sensitive healthcare data. Share best practices and stay current with developments in AI, ML, and LLMs. More ❯