26 to 50 of 96 Batch Processing Jobs in the UK

alongside data science colleagues, model versioning, automated retraining, and the monitoring that catches drift and degradation before a client does Build and maintain the batch and ELT pipelines the ML work sits on, using Python, PySpark, and cloud data services (e.g., AWS Glue, EMR, S3, Redshift, Athena … domain-driven modelling), this is the load-bearing, provider-agnostic skill underneath almost all of our pipeline work Have strong Python and PySpark for batch processing, with production-grade, well-tested code. The bulk of this work is batch and ELT, but you’ll need to have ...

methods to transform raw data into processed data. You will own the DataOps infrastructure, ensuring the performance, reliability, and scalability of data ingestion and processing pipelines to support evolving business and operational demands. To succeed in this data engineering position, you should have strong problem-solving skills … ability to combine data from different sources. Data engineer skills also include familiarity with several programming languages. What you’ll do Orchestrate the ingestion, processing, and storage of raw data within structured and unstructured data repositories. Design, Develop, Deploy and Support data infrastructure, pipelines and architecture. Implement reliable, scalable ...

native technologies, and applying robust software engineering practices. Key Responsibilities Design, develop, and optimise scalable, testable data pipelines using Python and Apache Spark. Implement batch workflows and ETL processes adhering to modern engineering standards. Develop Cloud-Based Workflows Orchestrate data workflows using AWS services such as Glue, EMR Serverless … experience writing clean, maintainable Python code, ideally using type hints, linters, and test frameworks such as pytest. Solid understanding of data engineering fundamentals including batch processing, schema evolution, and ETL pipeline development. Experience with—or strong interest in learning—Apache Spark for large-scale data processing. Familiarity with ...

cutting edge What we’re looking for Software & Data Engineering Strong expertise in Python Passion for writing: clean reliable production-grade code Distributed Data Processing Experience building and operating: batch processing systems distributed data systems near real-time streaming pipelines Familiarity with: Kafka Flink Spark Databricks experience ...

metadata-forward, CI/CD-driven platform represents and enables the entire application and analysis lifecycle including interactive development and explorations (notebooks), large-scale batch processing, observability and production application deployments. A Compute Platform Engineer II is a technical contributor who can consistently take a poorly defined business … business problems, Responsible for development of key components of a hybrid on-prem/cloud compute platform for both interactive and scalable batch computing and establishing of processes and workflows to transition existing HPC users and teams to this platform Responsible for code-driven environment, applications, and container/ ...

Doing Designing and developing scalable Python-based services and APIs. Building cloud-native platforms supporting analytical and risk workloads. Developing orchestration frameworks for batch and on-demand calculations. Managing containerised environments and distributed compute services. Implementing monitoring, observability and operational tooling. Building resilient services with fault tolerance, recovery … monitoring, observability and operational support. Strong stakeholder engagement and communication skills. Desirable Experience Risk systems, portfolio analytics or financial services platforms. Large-scale batch processing or calculation-heavy environments. Model governance, version control and auditable execution frameworks. Leading engineering teams or mentoring technical colleagues. #J-18808-Ljbffr ...

Doing Designing and developing scalable Python-based services and APIs. Building cloud-native platforms supporting analytical and risk workloads. Developing orchestration frameworks for batch and on-demand calculations. Managing containerised environments and distributed compute services. Implementing monitoring, observability and operational tooling. Building resilient services with fault tolerance, recovery … monitoring, observability and operational support. Strong stakeholder engagement and communication skills. Desirable Experience Risk systems, portfolio analytics or financial services platforms. Large-scale batch processing or calculation-heavy environments. Model governance, version control and auditable execution frameworks. Leading engineering teams or mentoring technical colleagues. Why Apply? This ...

national GP clinical research datasets. The role focuses on maintaining and improving a mature on‐premises SQL Server 2019 environment responsible for ingesting and processing high‐volume clinical data used in real‐world research. This role requires strong hands‐on SQL Server and SSIS experience. … engineering role. The position is script‐heavy and database‐centric, involving significant T‐SQL development, ETL scripting, SSIS package maintenance, and large‐scale data processing across very large databases. Key Responsibilities SQL Server ETL Development Develop and maintain T‐SQL based ETL pipelines running on SQL Server 2019. Data ...

Engineer II within our Engineering Maritime & Logistics division, you will play a pivotal role in designing, building, and operating high-throughput data processing and API distribution systems. You will take end-to-end ownership of real-time streaming applications, backend services, and reliable pipelines that deliver critical maritime intelligence … build, and maintain high-performance backend microservices and APIs using Java, Scala, and Python. Design, develop, and operate real-time data pipelines and stream-processing applications using Java, Scala, Kafka Streams, and Apache Flink, as well as batch-processing workflows using Python and Apache Airflow. Drive ...

complex enterprise application landscapes undergoing legacy transformation. You will focus on shaping scalable, resilient, secure, and governable architectures that progressively move monolithic, tightly coupled, batch-oriented systems toward modular, event-driven, cloud-enabled and operationally observable platforms. The role requires strong hands‐on architecture depth across application modernization, integration … modernization, distributed systems, integration platforms, database ecosystems, cloud-enabled platforms, and regulated enterprise environments. Strong ability to assess monolithic, tightly coupled, database-centric and batch-driven estates and translate findings into practical target-state architecture and migration roadmaps. Deep understanding of microservices architecture, domain-driven design, service decomposition, database ...

across various operational processes. Key Responsibilities: Design, implement, and execute data quality controls to uphold the completeness, accuracy, and integrity of reporting data across batch runs. Execute validation checks for daily batch processing cycles, ensuring adherence to established data quality standards. Conduct reconciliations between upstream, downstream ...

resolve complex application, database and system issues. Plan and support software deployments across development, test and production environments. Support IBM i operational processes, including batch processing, job scheduling and subsystem management. Contribute to testing, code reviews, documentation and continuous improvement initiatives. Share technical knowledge, mentor engineers and support … with JSON and XML data formats. Knowledge of CL programming and IBM i operational processes. Experience managing software deployments and production releases. Experience supporting batch workloads, job queues and subsystem operations. Strong troubleshooting and problem-solving skills. Understanding of software engineering best practices including testing, code quality and maintainability. ...

resolve complex application, database and system issues. Plan and support software deployments across development, test and production environments. Support IBM i operational processes, including batch processing, job scheduling and subsystem management. Contribute to testing, code reviews, documentation and continuous improvement initiatives. Share technical knowledge, mentor engineers and support … with JSON and XML data formats. Knowledge of CL programming and IBM i operational processes. Experience managing software deployments and production releases. Experience supporting batch workloads, job queues and subsystem operations. Strong troubleshooting and problem-solving skills. Understanding of software engineering best practices including testing, code quality and maintainability. ...

Databricks, Apache Spark, Apache Kafka, Apache Flink, Apache Beam and DBT. Demonstrated success in designing, building, and operating data platforms at scale, encompassing both batch processing and real-time streaming architectures. Expertise in data warehousing concepts, advanced ETL/ELT processes, and analytics engineering best practices. Exceptional programming ...

production and non‐production environments on an ongoing basis. Cloud & Application Operations Proactively monitor infrastructure and application health — including CPU, memory, file systems, databases, batch jobs, and network performance — and respond swiftly to anomalies. Identify and resolve operational issues including infrastructure failures, batch processing errors, network disruptions ...

improvements Supporting D365 configuration, workflows, security, user administration and functional troubleshooting Investigating incidents and system issues, identifying root causes and coordinating resolutions Supporting integrations, batch processes, data interfaces and data management Planning and supporting UAT, regression and validation testing, including test scripts, defect management and stakeholder sign-off Supporting … hands-on approach and the ability to take ownership of issues through to resolution Knowledge of D365 workflows, security administration, LCS, DMF, integrations and batch processing would be particularly useful. Exposure to Power BI, Power Automate, Power Apps, SQL or other ERP platforms would also be advantageous. ...

improvements Supporting D365 configuration, workflows, security, user administration and functional troubleshooting Investigating incidents and system issues, identifying root causes and coordinating resolutions Supporting integrations, batch processes, data interfaces and data management Planning and supporting UAT, regression and validation testing, including test scripts, defect management and stakeholder sign-off Supporting … hands-on approach and the ability to take ownership of issues through to resolution Knowledge of D365 workflows, security administration, LCS, DMF, integrations and batch processing would be particularly useful. Exposure to Power BI, Power Automate, Power Apps, SQL or other ERP platforms would also be advantageous. ...

resilience, availability, performance and capacity. * Manage deployments, releases, production readiness and failover activities. * Implement monitoring, alerting and observability solutions. * Troubleshoot complex application, infrastructure and batch-processing issues. * Identify root causes and deliver sustainable technical solutions. * Contribute to CI/CD, automation and wider DevOps and engineering best practices. ...

degree in Computer Science, Electrical Engineering or a related field, or equivalent practical experience. Experience with AWS or GCP, Kubernetes, SQL warehouse and batch-processing technologies such as Trino, Presto or Spark, Kafka or other streaming systems, and Go or Python would be useful. Node.js experience ...

/Config Management, Scripting - 4+ years hands-on experience with CICD (e.g. Ansible), and GIT (proficiency) - Experience with Back-Office Workflows, Datamart, Market Data, Batch processing, End of Day flow(s), interfaces and Trade Life Cycle - Leadership: Proven experience in a technical lead or environment management lead capacity ...

with AI/ML teams to design infrastructure for agent development, testing and deployment; manage inference serving infrastructure. Optimize AI infrastructure costs: GPU utilization, batch processing, inference optimization, cost-per-inference efficiency. Ensure AI infrastructure is observable and performant; integrate with observability platform for AI workload monitoring. Infrastructure ...

Responsibilities Teamcenter Configuration & Customisation Design and implement Teamcenter customisations using Teamcenter ITK. Develop and maintain: Custom handlers User exits Workflow extensions Custom business rules Batch processes Extend standard Teamcenter functionality to meet evolving business requirements. Support Teamcenter upgrades and migration activities. Integration Development Design and implement integrations between Teamcenter ...

maintain reports using Endur reporting tools Support CI/CD processes and deployments within Endur environments Troubleshoot issues across development, testing, and production Support batch processes and EOD activities as needed Work with business users and analysts to understand requirements and deliver working solutions Required (Endur-specific) Hands … working within Endur reporting tools (Report Builder or similar) Familiarity with CI/CD processes in Endur environments Experience supporting or working with Endur batch/EOD processes 3–5+ years of Endur‐specific development experience Transferable experience (CTRM/general development) Development experience with Java, JavaScript, .NET ...

maintain reports using Endur reporting tools Support CI/CD processes and deployments within Endur environments Troubleshoot issues across development, testing, and production Support batch processes and EOD activities as needed Work with business users and analysts to understand requirements and deliver working solutions Required (Endur-specific) Hands … working within Endur reporting tools (Report Builder or similar) Familiarity with CI/CD processes in Endur environments Experience supporting or working with Endur batch/EOD processes 3-5+ years of Endur-specific development experience Transferable experience (CTRM/general development) Development experience with Java, JavaScript, .NET ...

applications on **IBM z/OS** platform* Design and code programs using **COBOL, JCL, CICS, DB2, IMS, and VSAM*** Support both **online (CICS)** and batch processing environments* Perform debugging, root cause analysis, and resolution of production issues/abends* Optimize SQL queries and DB2 performance tuning* Work with ...