1 to 25 of 32 Batch Processing Jobs in the UK

role : Hands on Experience on design & architecture of the Data Platform, Technical components and Data Engg. related activities for supporting the key capabilities Technical Data Architecture Data Ingestion framework ( Batch/Micro Batch processing ) Data Contract & Data management( including DQ, metadata & lineage) Delivery, Scaling & Op Model Discovery/assessment along with the Data platform requirements Identify and … and data workflows to determine the integration points and any design considerations/framework and patterns for the data factory as per the need for TSA and Non-TSA batch business/technical requirements for both batch and micro-batch processing. Define the E2E Data architecture framework Define Solution patterns for Data collection & ingestion, Data transformation andcuration More ❯

cross-functional teams including data architects, compliance officers, and cybersecurity specialists to integrate data from various systems such as databases, APIs, and cloud platforms. Your work will directly support batch processing, real-time streaming, and event-driven data pipelines across a variety of use cases. We re looking for candidates with over 3 years of relevant experience in … the following skills or proven experience: Apache NiFi Expertise: • Deep understanding of core NiFi concepts: FlowFiles, Processors, Controller Services, Schedulers, Web UI. • Experience designing and optimizing data flows for batch, real-time streaming, and event-driven architectures. • Ability to identify and resolve flow issues, optimize performance, and implement errorhandling strategies. • Optional scripting skills for creating custom NiFi processors. Programming More ❯

cross-functional teams including data architects, compliance officers, and cybersecurity specialists to integrate data from various systems such as databases, APIs, and cloud platforms. Your work will directly support batch processing, real-time streaming, and event-driven data pipelines across a variety of use cases. We’re looking for candidates with over 3 years of relevant experience in … the following skills or proven experience: Apache NiFi Expertise: • Deep understanding of core NiFi concepts: FlowFiles, Processors, Controller Services, Schedulers, Web UI. • Experience designing and optimizing data flows for batch, real-time streaming, and event-driven architectures. • Ability to identify and resolve flow issues, optimize performance, and implement errorhandling strategies. • Optional scripting skills for creating custom NiFi processors. Programming More ❯

a focus on clean, maintainable code (bonus for experience with type hints, linters, and testing frameworks like pytest) Solid understanding of data engineering fundamentals: ETL/ELT, schema evolution, batch processing Experience or strong interest in Apache Spark for distributed data processing Familiarity with AWS data tools (e.g., S3, Glue, Lambda, EMR) Strong communication skills and a More ❯

currently recruiting for a Platfrom Support Engineer to join an industry leading and ambitious financial services company. This is an exciting opportunity with multifaceted responsibilities, including infrastructure, application and batch support. As well as developing and maintaining Grafana dashboards. What you’ll do: • Support batch processing and carry out proactive analysis across squads and services. • Support and … resolve production incidents, problems, and change requests. • Offer expertise on business processes for projects and production support issues. • Maintain core systems and related infrastructure like Kubernetes and SOLR. • Improve batch efficiency and reduce costs, including audit-ready data archiving strategies. • Provide data analysis to support business decisions • Deliver patching, database fixes, and business-critical data correction services. What we More ❯

currently recruiting for a Platfrom Support Engineer to join an industry leading and ambitious financial services company. This is an exciting opportunity with multifaceted responsibilities, including infrastructure, application and batch support. As well as developing and maintaining Grafana dashboards. What you'll do: • Support batch processing and carry out proactive analysis across squads and services. • Support and … resolve production incidents, problems, and change requests. • Offer expertise on business processes for projects and production support issues. • Maintain core systems and related infrastructure like Kubernetes and SOLR. • Improve batch efficiency and reduce costs, including audit-ready data archiving strategies. • Provide data analysis to support business decisions • Deliver patching, database fixes, and business-critical data correction services. What we More ❯

and enthusiastic in these domains, this is a phenomenal role for you! Responsibilities and Duties of the Role: Build components of large-scale data platform for real-time and batch processing, and own features of big data applications to fit evolving business needs Build next-gen cloud based big data infrastructure for batch and streaming data applications More ❯

sustainable, and agile data analytics platform for a data driven digital bank. You will be using a range of database technologies and a combination of event driven techniques and batch processing methodologies. In addition you will be expanding and optimizing our cloud-based data & data pipelines as part of a cross functional team. As a Data Engineer at More ❯

systems are written in Elixir, but where necessary, we use small amounts of Python and Java where vendor SDKs require it. Designing, developing and maintaining realtime data streaming, and batch processing workloads. Providing on-call support as part of our teamwide rotation. Our on-call rotation is split across US and UK time zones, ensuring coverage whilst keeping … parallelism for speed/space performance tradeoffs. Bonus Experience Exchange traded financial instruments. Statistics, discrete mathematics, linear algebra. Problem-solving and proof construction. For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy Notice at California residents, please review the California Privacy Notice for information about certain legal rights at More ❯

tolerant workflows using AWS services such as Lambda, Kinesis, SQS, DynamoDB Streams, and EventBridge. -Implement processors that ensure data consistency across DynamoDB, PostgreSQL (Aurora), OpenSearch, and BigQuery. -Modernise legacy batch processes into stream-first architectures. -Build and integrate APIs to enable smooth publishing and consumption of events across systems. -Collaborate with engineers on greenfield and existing projects, balancing speed More ❯

tolerant workflows using AWS services such as Lambda, Kinesis, SQS, DynamoDB Streams, and EventBridge. -Implement processors that ensure data consistency across DynamoDB, PostgreSQL (Aurora), OpenSearch, and BigQuery. -Modernise legacy batch processes into stream-first architectures. -Build and integrate APIs to enable smooth publishing and consumption of events across systems. -Collaborate with engineers on greenfield and existing projects, balancing speed More ❯

tolerant workflows using AWS services such as Lambda, Kinesis, SQS, DynamoDB Streams, and EventBridge. -Implement processors that ensure data consistency across DynamoDB, PostgreSQL (Aurora), OpenSearch, and BigQuery. -Modernise legacy batch processes into stream-first architectures. -Build and integrate APIs to enable smooth publishing and consumption of events across systems. -Collaborate with engineers on greenfield and existing projects, balancing speed More ❯

tolerant workflows using AWS services such as Lambda, Kinesis, SQS, DynamoDB Streams, and EventBridge. -Implement processors that ensure data consistency across DynamoDB, PostgreSQL (Aurora), OpenSearch, and BigQuery. -Modernise legacy batch processes into stream-first architectures. -Build and integrate APIs to enable smooth publishing and consumption of events across systems. -Collaborate with engineers on greenfield and existing projects, balancing speed More ❯

tolerant workflows using AWS services such as Lambda, Kinesis, SQS, DynamoDB Streams, and EventBridge. -Implement processors that ensure data consistency across DynamoDB, PostgreSQL (Aurora), OpenSearch, and BigQuery. -Modernise legacy batch processes into stream-first architectures. -Build and integrate APIs to enable smooth publishing and consumption of events across systems. -Collaborate with engineers on greenfield and existing projects, balancing speed More ❯

Batch System Operations Technician - Newcastle Upon Tyne - Benton Park View About the job Job summary Discover a career in your hands at HMRC. Whether you're seeking purpose, growth, or a workplace that gives you a true sense of belonging, hear from some of our employees as they share their story about what it's really like to work … at HMRC. Visit our YouTube channel to watch the full series and come and discover your potential. This is an exciting opportunity to become a Batch System Operations Technician working on the NPS (National PAYE System) Service as part of HMRC's Critical National Infrastructure. The NPS Batch Execution team is part of the wider Contact Engagement Platform … within HMRC's CDIO Chief Engineering and Platform Office (CEPO) business area. Successful applicants will, through in-house training, become a vital cog in the success of the NPS Batch Execution team in Newcastle which is, fundamentally, accountable for the successful end-to-end operational running of a diverse array of daily NPS System batch processes. As part More ❯

Batch System Operations Technician - Newcastle Upon Tyne - Benton Park View About the job Job summary Discover a career in your hands at HMRC. Whether you're seeking purpose, growth, or a workplace that gives you a true sense of belonging, hear from some of our employees as they share their story about what it's really like to work … at HMRC. Visit our YouTube channel to watch the full series and come and discover your potential. This is an exciting opportunity to become a Batch System Operations Technician working on the NPS (National PAYE System) Service as part of HMRC's Critical National Infrastructure. The NPS Batch Execution team is part of the wider Contact Engagement Platform … within HMRC's CDIO Chief Engineering and Platform Office (CEPO) business area. Successful applicants will, through in-house training, become a vital cog in the success of the NPS Batch Execution team in Newcastle which is, fundamentally, accountable for the successful end-to-end operational running of a diverse array of daily NPS System batch processes. As part More ❯

Batch System Operations Technician - Newcastle Upon Tyne - Benton Park View About the job Job summary Discover a career in your hands at HMRC. Whether you're seeking purpose, growth, or a workplace that gives you a true sense of belonging, hear from some of our employees as they share their story about what it's really like to work … at HMRC. Visit our YouTube channel to watch the full series and come and discover your potential. This is an exciting opportunity to become a Batch System Operations Technician working on the NPS (National PAYE System) Service as part of HMRC's Critical National Infrastructure. The NPS Batch Execution team is part of the wider Contact Engagement Platform … within HMRC's CDIO Chief Engineering and Platform Office (CEPO) business area. Successful applicants will, through in-house training, become a vital cog in the success of the NPS Batch Execution team in Newcastle which is, fundamentally, accountable for the successful end-to-end operational running of a diverse array of daily NPS System batch processes. As part More ❯

security, and privacy policies. Document data architecture, pipelines, and processes for transparency and knowledge sharing. Contribute to the development of a modern data platform that supports real-time and batch processing. Reasonable Adjustments: Respect and equality are core values to us. We are proud of the diverse and inclusive community we have built, and we welcome applications from people More ❯

field Experience: Proven experience in ML model lifecycle management Core Competencies : Model lifecycle: You've got hands-on experience with managing the ML model lifecycle, including both online and batch processes Statistical Methodology: You have worked with GLMs and other machine learning algorithms and have in-depth knowledge of how they work Python: You have built and deployed production More ❯

with a passion for technology and finance to design and implement innovative solutions within D365 Finance & Operations. Key Responsibilities: - Design, develop, test, and deploy custom extensions for general journal processing, including data entities like LedgerJournalHeader and LedgerJournalLine. - Implement business logic for transaction validation, reconciliation, and posting workflows within D365 Finance. - Collaborate with integration teams to ensure seamless functionality and … D365 tools like Visual Studio, LCS, and Azure DevOps for CI/CD, along with unit testing frameworks (e.g., SysTest) and debugging. - Familiarity with financial data handling, error management, batch processing, and integration patterns (e.g., DMF, OData). - Experience with D365 workflows, security roles, and reporting tools (e.g., SSRS). - Experience in working with PowerBI reports and creation More ❯

team with suggestions on how to improve and optimize processes Responsibilities: Responsible for the support & integration of Guidewire application to business estates Identify opportunities and propose integration solutions including batch processing and message queues Manage and provide technical for Guidewire applications in production. Liaise & interact with 3rd party vendors Engage with technical teams and business analysts to achieve More ❯

data principles (Findable, Accessible, Interoperable, Reusable)." - Experience architecting for High-Performance Computing (HPC) environments, including knowledge of workload schedulers (e.g., SLURM) and applying cloud-native patterns to scientific, batch-processing workloads. - Familiarity with scientific workflow management tools (e.g., Nextflow, Snakemake) and the use of containerization (Docker, Kubernetes) in research environments. Familiarity with infrastructure considerations (networking, hosting, security More ❯

data solutions that support diverse trading activities. Take full ownership of data products, guiding them from initial concept through to stable production. Design, build, and maintain systems for both batch processing and real-time streaming of time series datasets, ensuring high data quality and reliability. Develop APIs and data access methods for fast, intuitive retrieval of historical and … Lake, Apache Iceberg), and relational databases Have a Bachelor or advanced degree in Computer Science, Mathematics, Statistics, Physics, Engineering, or equivalent work experience For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy Notice at More ❯

experience of using Visual Studio in a commercial environment (ideally a healthcare setting) and experience of interfacing highly complex datasets between bespoke systems both in real time and as batch processing. They will have in-depth knowledge of SSIS, XML, HL7, ODBC and C#, be able to provide expert advice on highly complex digital issues and communicate clearly and … leadership of a development team Experience of using Visual Studio in a commercial environment Experience of interfacing highly complex datasets between bespoke systems both in real time and as batch processing Desirable Experience of using Visual Studio in a Healthcare environment either within the NHS or private sector. Experience of interfacing highly complex datasets between bespoke healthcare systems More ❯

have: Backend Automation testing skills with Python and Pytest. Solid SQL skills and experience with relational databases. Solid experience in automation testing using tools such as Selenium and Appium. Batch processing experience. Financial services/fintech experience. If you're passionate about testing, automation, and continuous learning, we would love to hear from you More ❯