1 to 25 of 46 Batch Processing Jobs in London

Role OverviewThe E2E Environments & Global Batch Manager is responsible for defining, implementing, and governing end-to-end environment and batch management capabilities across complex client implementation programs. This role serves as the primary point of coordination across Product, Technology, Operations, Delivery, and Client teams to ensure environment readiness … batch execution, testing support, conversion planning, operational readiness, and go-live success. The position combines strategic planning, governance leadership, technical coordination, and operational oversight to support successful implementation outcomes. Key ResponsibilitiesStrategy, Planning & GovernanceDefine and drive the end-to-end environment and batch strategy aligned to program objectives ...

/Servlets) Dependency injection, IoC, AOP Functional programming concepts Test‐driven development (TDD) Data access frameworks (JPA/Hibernate or similar) Transaction management Batch processing concepts API development and integration DevOps practices (CI/CD, automation, Git‐based workflows) Debugging and problem‐solving Communication and collaboration skills Ability … functionality and user experience. Knowledge of Functional Programming: Exposure to applying functional programming principles, improving code efficiency and scalability in application development. Familiarity with Batch Processing: Exposure to batch processing frameworks and tools, ensuring efficient data processing and system integration. IBM is committed to creating ...

business analysis and production support groups, ensuring platform improvements translate into stronger production stability, better compute utilisation, more reliable lower-lane execution, improved batch observability, data quality controls, disciplined regression and release processes, and sustained adoption of agreed engineering changes. Role Description: This is a critical technical platform leadership … FRTB Standardised Approach. This is not a programme management, advisory or coordination role. You should be technically credible, able to understand platform architecture, batch execution, data flows, dependencies, production failure modes, hardware utilisation, computation cost and engineering trade-offs, and capable of challenging technical assumptions. We require an execution ...

enterprise data platform. In this role, you will implement a modern data lakehouse architecture capable of supporting raw and curated data zones, scalable batch processing, and future analytical workloads. You will play a key role in integrating our existing data pipelines, models, and transformation logic into a unified … Management: Maintain architectural segregation between raw storage (immutability, landing) and curated data zones (cleansed, conformed business layers) while securely handling identifiable data. Scalable Data Processing: Develop, tune, and monitor large-scale batch processing pipelines using Azure Databricks (Spark-based processing) and PySpark. Orchestration & Workflow: Build automated ...

enterprise data platform. In this role, you will implement a modern data lakehouse architecture capable of supporting raw and curated data zones, scalable batch processing, and future analytical workloads. You will play a key role in integrating our existing data pipelines, models, and transformation logic into a unified … Management: Maintain architectural segregation between raw storage (immutability, landing) and curated data zones (cleansed, conformed business layers) while securely handling identifiable data. Scalable Data Processing: Develop, tune, and monitor large-scale batch processing pipelines using Azure Databricks (Spark-based processing) and PySpark. Orchestration & Workflow: Build automated ...

enterprise data platform. In this role, you will implement a modern data lakehouse architecture capable of supporting raw and curated data zones, scalable batch processing, and future analytical workloads. You will play a key role in integrating our existing data pipelines, models, and transformation logic into a unified … Management: Maintain architectural segregation between raw storage (immutability, landing) and curated data zones (cleansed, conformed business layers) while securely handling identifiable data. Scalable Data Processing: Develop, tune, and monitor large-scale batch processing pipelines using Azure Databricks (Spark-based processing) and PySpark. Orchestration & Workflow: Build automated ...

enterprise data, APIs and microservices. Ensure AI solutions integrate cleanly into cloud-native PaaS and SaaS environments, with robust approaches to real-time and batch processing where required. Establish best practices for prompt engineering, model selection, grounding, evaluation and guardrails across AI solutions. Lead AI and LLM proof … guardrails. Experience working with cloud environments and enterprise platforms, with Azure experience preferred, and an understanding of PaaS, SaaS, real-time and batch processing patterns. Familiarity with Docker, CI/CD, infrastructure approaches, MLOps, LLMOps, AIOps, observability, versioning and production operations would be advantageous. Strong understanding of responsible ...

high-quality data products everywhere; and we directly own a deliberate set of business-critical, cross-cutting data products. Our work spans: Streaming and batch data platforms that unlock data across TripadvisorETL pipelines and data models serving terabytes and growing datasets in Snowflake, BigQuery, and HiveOrchestration, scheduling …/ELT data pipelines that process terabyte- to petabyte-scale data across Snowflake, BigQuery, and other platforms. Design, build and maintain streaming and batch systems that power real-time and analytical workloads, and lead simplification of legacy ingestion paths. Drive warehouse standardization and technical-debt reduction as we consolidate ...

learning techniques, including supervised ML, unsupervised ML, and LLMs Experience with deployment tools and processes, such as Azure Data Factory (ADF), Azure ML and batch processing Experience using DevOps or equivalent agile development environments 4+ years’ experience leading clients through complex technical projects — you know how to navigate ...

Experiment with open source tools Participate in community-building initiatives such as meetups Continuously learn and improve engineering practices What we’re looking for batch processing distributed data processing Experience with technologies such as: Kafka Flink Spark Databricks experience is a plus Experience implementing: Data Lakes ...

build the services and infrastructure behind portfolio risk and analytics calculations. You'll turn analytical models into reliable production services, building APIs, cloud processing and workflows that connect portfolio data with calculation engines and deliver results to existing systems. There is also scope to provide hands-on technical leadership … across the wider team. The roleBuild Python APIs and calculation services. Develop cloud-based processing for scheduled batch and on-demand jobs. Implement containers, orchestration and queues, with robust retries and recovery. Integrate SQL data sources and downstream systems, ensuring accurate, traceable inputs and outputs. Build deployment pipelines ...

build the services and infrastructure behind portfolio risk and analytics calculations. You’ll turn analytical models into reliable production services, building APIs, cloud processing and workflows that connect portfolio data with calculation engines and deliver results to existing systems. There is also scope to provide hands‐on technical leadership … across the wider team. The role Build Python APIs and calculation services. Develop cloud-based processing for scheduled batch and on‐demand jobs. Implement containers, orchestration and queues, with robust retries and recovery. Integrate SQL data sources and downstream systems, ensuring accurate, traceable inputs and outputs. Build deployment ...

systems while preserving provenance and auditability. Build APIs and services that make processed data available to products, customers and AI systems. Develop workflows for batch processing, event-driven processing and long-running computational jobs. Create monitoring and alerting for pipeline failures, unexpected data changes and quality regressions. ...

data platforms—particularly Snowflake—covering performance, workload management, security, access controls, cost governance, multi-environment deployment, data sharing, replication and disaster recovery. Define scalable batch, real-time, event-driven, API, orchestration and ELT/ETL integration patterns using modern data integration technologies. Establish standards for analytics engineering … Experience with BigQuery, Databricks, Synapse, Redshift or similar platforms is highly desirable. Expertise in scalable enterprise ELT/ETL, data integration, real-time and batch processing, event-driven systems, APIs and orchestration patterns. Experience with Fivetran, Workato, Boomi, Airbyte or similar integration platforms. Ability to evaluate technology options ...

cutting edge What we’re looking for Software & Data Engineering Strong expertise in Python Passion for writing: clean reliable production-grade code Distributed Data Processing Experience building and operating: batch processing systems distributed data systems near real-time streaming pipelines Familiarity with: Kafka Flink Spark Databricks experience ...

native technologies, and applying robust software engineering practices. Key Responsibilities Design, develop, and optimise scalable, testable data pipelines using Python and Apache Spark. Implement batch workflows and ETL processes adhering to modern engineering standards. Develop Cloud-Based Workflows Orchestrate data workflows using AWS services such as Glue, EMR Serverless … experience writing clean, maintainable Python code, ideally using type hints, linters, and test frameworks such as pytest. Solid understanding of data engineering fundamentals including batch processing, schema evolution, and ETL pipeline development. Experience with—or strong interest in learning—Apache Spark for large-scale data processing. Familiarity with ...

transformation technologies such as Liquid templates and XSLT for data manipulation. Design, document, and build integration solutions using a variety of approaches including APIs, batch processes, and flat-file exchanges. Integrate core business applications across multiple business domains, spanning on premises systems and SaaS platforms. Lead data mapping ...

across various operational processes. Key Responsibilities: Design, implement, and execute data quality controls to uphold the completeness, accuracy, and integrity of reporting data across batch runs. Execute validation checks for daily batch processing cycles, ensuring adherence to established data quality standards. Conduct reconciliations between upstream, downstream ...

complex enterprise application landscapes undergoing legacy transformation. You will focus on shaping scalable, resilient, secure, and governable architectures that progressively move monolithic, tightly coupled, batch-oriented systems toward modular, event-driven, cloud-enabled and operationally observable platforms. The role requires strong hands‐on architecture depth across application modernization, integration … modernization, distributed systems, integration platforms, database ecosystems, cloud-enabled platforms, and regulated enterprise environments. Strong ability to assess monolithic, tightly coupled, database-centric and batch-driven estates and translate findings into practical target-state architecture and migration roadmaps. Deep understanding of microservices architecture, domain-driven design, service decomposition, database ...

Doing Designing and developing scalable Python-based services and APIs. Building cloud-native platforms supporting analytical and risk workloads. Developing orchestration frameworks for batch and on-demand calculations. Managing containerised environments and distributed compute services. Implementing monitoring, observability and operational tooling. Building resilient services with fault tolerance, recovery … monitoring, observability and operational support. Strong stakeholder engagement and communication skills. Desirable Experience Risk systems, portfolio analytics or financial services platforms. Large-scale batch processing or calculation-heavy environments. Model governance, version control and auditable execution frameworks. Leading engineering teams or mentoring technical colleagues. #J-18808-Ljbffr ...

Doing Designing and developing scalable Python-based services and APIs. Building cloud-native platforms supporting analytical and risk workloads. Developing orchestration frameworks for batch and on-demand calculations. Managing containerised environments and distributed compute services. Implementing monitoring, observability and operational tooling. Building resilient services with fault tolerance, recovery … monitoring, observability and operational support. Strong stakeholder engagement and communication skills. Desirable Experience Risk systems, portfolio analytics or financial services platforms. Large-scale batch processing or calculation-heavy environments. Model governance, version control and auditable execution frameworks. Leading engineering teams or mentoring technical colleagues. Why Apply? This ...

consumers. You've built architectures that held up as they grew with a focus on data warehousing, data lakes, and both real-time and batch processing. Teams bring you their hardest data problems. You've made the hard tradeoffs in distributed, multi-tenant systems and lived with the consequences. ...

Databricks, Apache Spark, Apache Kafka, Apache Flink, Apache Beam and DBT. Demonstrated success in designing, building, and operating data platforms at scale, encompassing both batch processing and real-time streaming architectures. Expertise in data warehousing concepts, advanced ETL/ELT processes, and analytics engineering best practices. Exceptional programming ...

client inquiries in a timely and professional manner in support superior customer experience model Install my clients software products on site or remotely Use batch processes to integrate products into client processes Use software application programming to integrate products into client systems as needed Technical Support Consultants Must have … years commercial experience Arabic Speaker Experience with SQL databases and SQL programming language Experience of Application support or Implementation Experience with scripting, batch processing, data file manipulation, or Web services Knowledge of data management concepts, data analytics, data quality, data integration, business intelligence, CRM, Report Writer, or similar ...

improvements Supporting D365 configuration, workflows, security, user administration and functional troubleshooting Investigating incidents and system issues, identifying root causes and coordinating resolutions Supporting integrations, batch processes, data interfaces and data management Planning and supporting UAT, regression and validation testing, including test scripts, defect management and stakeholder sign-off Supporting … hands-on approach and the ability to take ownership of issues through to resolution Knowledge of D365 workflows, security administration, LCS, DMF, integrations and batch processing would be particularly useful. Exposure to Power BI, Power Automate, Power Apps, SQL or other ERP platforms would also be advantageous. ...