1 to 25 of 42 Batch Processing Jobs in London

dynamic team. The ideal candidate will have a strong background in Java development, extensive experience with AWS, database management, and preferred work experience in batch processing. As a Senior Java Developer, you will play a crucial role in designing, developing, and maintaining high-performance applications that meet our business … cloud-based solutions, ensuring high availability and scalability. Database Management:Design, implement, and maintain database schemas, write complex SQL queries, and optimize database performance. Batch Processing:Develop and manage batch processing systems to handle large volumes of data efficiently. Code Quality:Ensure code quality through code ...

enterprise data platform. In this role, you will implement a modern data lakehouse architecture capable of supporting raw and curated data zones, scalable batch processing, and future analytical workloads. You will play a key role in integrating our existing data pipelines, models, and transformation logic into a unified … Management: Maintain architectural segregation between raw storage (immutability, landing) and curated data zones (cleansed, conformed business layers) while securely handling identifiable data. Scalable Data Processing: Develop, tune, and monitor large-scale batch processing pipelines using Azure Databricks (Spark-based processing) and PySpark. Orchestration & Workflow: Build automated ...

enterprise data platform. In this role, you will implement a modern data lakehouse architecture capable of supporting raw and curated data zones, scalable batch processing, and future analytical workloads. You will play a key role in integrating our existing data pipelines, models, and transformation logic into a unified … Management: Maintain architectural segregation between raw storage (immutability, landing) and curated data zones (cleansed, conformed business layers) while securely handling identifiable data. Scalable Data Processing: Develop, tune, and monitor large-scale batch processing pipelines using Azure Databricks (Spark-based processing) and PySpark. Orchestration & Workflow: Build automated ...

enterprise data, APIs and microservices. Ensure AI solutions integrate cleanly into cloud-native PaaS and SaaS environments, with robust approaches to real-time and batch processing where required. Establish best practices for prompt engineering, model selection, grounding, evaluation and guardrails across AI solutions. Lead AI and LLM proof … guardrails. Experience working with cloud environments and enterprise platforms, with Azure experience preferred, and an understanding of PaaS, SaaS, real-time and batch processing patterns. Familiarity with Docker, CI/CD, infrastructure approaches, MLOps, LLMOps, AIOps, observability, versioning and production operations would be advantageous. Strong understanding of responsible ...

quality, integration, and unit testing Understanding of CI/CD pipelines to maintain code quality and facilitate deployments Knowledge of Data Engineering fundamentals, including Batch processing and ETL/ELT Basic understanding of various data architectures, such as Lakes and Warehouses Knowledge of data cleansing and transformation within ...

Experiment with open source tools Participate in community-building initiatives such as meetups Continuously learn and improve engineering practices What we’re looking for batch processing distributed data processing Experience with technologies such as: Kafka Flink Spark Databricks experience is a plus Experience implementing: Data Lakes ...

learning techniques, including supervised ML, unsupervised ML, and LLMs Experience with deployment tools and processes, such as Azure Data Factory (ADF), Azure ML and batch processing Experience using DevOps or equivalent agile development environments 4+ years’ experience leading clients through complex technical projects — you know how to navigate ...

build the services and infrastructure behind portfolio risk and analytics calculations. You’ll turn analytical models into reliable production services, building APIs, cloud processing and workflows that connect portfolio data with calculation engines and deliver results to existing systems. There is also scope to provide hands‐on technical leadership … across the wider team. The role Build Python APIs and calculation services. Develop cloud-based processing for scheduled batch and on‐demand jobs. Implement containers, orchestration and queues, with robust retries and recovery. Integrate SQL data sources and downstream systems, ensuring accurate, traceable inputs and outputs. Build deployment ...

configuration, Murex Market Risk/Enterprise Risk Management, Murex DataMart, Murex Reporting, Murex Workflows, Murex Interfaces/Integration, Murex trade and market-data processing, Murex batch processing, Murex static/reference data, Murex environments and deployment processes and Murex troubleshooting and production support. The candidate should have ...

systems while preserving provenance and auditability. Build APIs and services that make processed data available to products, customers and AI systems. Develop workflows for batch processing, event-driven processing and long-running computational jobs. Create monitoring and alerting for pipeline failures, unexpected data changes and quality regressions. ...

data platforms—particularly Snowflake—covering performance, workload management, security, access controls, cost governance, multi-environment deployment, data sharing, replication and disaster recovery. Define scalable batch, real-time, event-driven, API, orchestration and ELT/ETL integration patterns using modern data integration technologies. Establish standards for analytics engineering … Experience with BigQuery, Databricks, Synapse, Redshift or similar platforms is highly desirable. Expertise in scalable enterprise ELT/ETL, data integration, real-time and batch processing, event-driven systems, APIs and orchestration patterns. Experience with Fivetran, Workato, Boomi, Airbyte or similar integration platforms. Ability to evaluate technology options ...

cutting edge What we’re looking for Software & Data Engineering Strong expertise in Python Passion for writing: clean reliable production-grade code Distributed Data Processing Experience building and operating: batch processing systems distributed data systems near real-time streaming pipelines Familiarity with: Kafka Flink Spark Databricks experience ...

native technologies, and applying robust software engineering practices. Key Responsibilities Design, develop, and optimise scalable, testable data pipelines using Python and Apache Spark. Implement batch workflows and ETL processes adhering to modern engineering standards. Develop Cloud-Based Workflows Orchestrate data workflows using AWS services such as Glue, EMR Serverless … experience writing clean, maintainable Python code, ideally using type hints, linters, and test frameworks such as pytest. Solid understanding of data engineering fundamentals including batch processing, schema evolution, and ETL pipeline development. Experience with—or strong interest in learning—Apache Spark for large-scale data processing. Familiarity with ...

alongside data science colleagues, model versioning, automated retraining, and the monitoring that catches drift and degradation before a client does Build and maintain the batch and ELT pipelines the ML work sits on, using Python, PySpark, and cloud data services (e.g., AWS Glue, EMR, S3, Redshift, Athena … domain-driven modelling), this is the load-bearing, provider-agnostic skill underneath almost all of our pipeline work Have strong Python and PySpark for batch processing, with production-grade, well-tested code. The bulk of this work is batch and ELT, but you’ll need to have ...

across various operational processes. Key Responsibilities: Design, implement, and execute data quality controls to uphold the completeness, accuracy, and integrity of reporting data across batch runs. Execute validation checks for daily batch processing cycles, ensuring adherence to established data quality standards. Conduct reconciliations between upstream, downstream ...

complex enterprise application landscapes undergoing legacy transformation. You will focus on shaping scalable, resilient, secure, and governable architectures that progressively move monolithic, tightly coupled, batch-oriented systems toward modular, event-driven, cloud-enabled and operationally observable platforms. The role requires strong hands‐on architecture depth across application modernization, integration … modernization, distributed systems, integration platforms, database ecosystems, cloud-enabled platforms, and regulated enterprise environments. Strong ability to assess monolithic, tightly coupled, database-centric and batch-driven estates and translate findings into practical target-state architecture and migration roadmaps. Deep understanding of microservices architecture, domain-driven design, service decomposition, database ...

Doing Designing and developing scalable Python-based services and APIs. Building cloud-native platforms supporting analytical and risk workloads. Developing orchestration frameworks for batch and on-demand calculations. Managing containerised environments and distributed compute services. Implementing monitoring, observability and operational tooling. Building resilient services with fault tolerance, recovery … monitoring, observability and operational support. Strong stakeholder engagement and communication skills. Desirable Experience Risk systems, portfolio analytics or financial services platforms. Large-scale batch processing or calculation-heavy environments. Model governance, version control and auditable execution frameworks. Leading engineering teams or mentoring technical colleagues. #J-18808-Ljbffr ...

Doing Designing and developing scalable Python-based services and APIs. Building cloud-native platforms supporting analytical and risk workloads. Developing orchestration frameworks for batch and on-demand calculations. Managing containerised environments and distributed compute services. Implementing monitoring, observability and operational tooling. Building resilient services with fault tolerance, recovery … monitoring, observability and operational support. Strong stakeholder engagement and communication skills. Desirable Experience Risk systems, portfolio analytics or financial services platforms. Large-scale batch processing or calculation-heavy environments. Model governance, version control and auditable execution frameworks. Leading engineering teams or mentoring technical colleagues. Why Apply? This ...

improvements Supporting D365 configuration, workflows, security, user administration and functional troubleshooting Investigating incidents and system issues, identifying root causes and coordinating resolutions Supporting integrations, batch processes, data interfaces and data management Planning and supporting UAT, regression and validation testing, including test scripts, defect management and stakeholder sign-off Supporting … hands-on approach and the ability to take ownership of issues through to resolution Knowledge of D365 workflows, security administration, LCS, DMF, integrations and batch processing would be particularly useful. Exposure to Power BI, Power Automate, Power Apps, SQL or other ERP platforms would also be advantageous. ...

improvements Supporting D365 configuration, workflows, security, user administration and functional troubleshooting Investigating incidents and system issues, identifying root causes and coordinating resolutions Supporting integrations, batch processes, data interfaces and data management Planning and supporting UAT, regression and validation testing, including test scripts, defect management and stakeholder sign-off Supporting … hands-on approach and the ability to take ownership of issues through to resolution Knowledge of D365 workflows, security administration, LCS, DMF, integrations and batch processing would be particularly useful. Exposure to Power BI, Power Automate, Power Apps, SQL or other ERP platforms would also be advantageous. ...

Databricks, Apache Spark, Apache Kafka, Apache Flink, Apache Beam and DBT. Demonstrated success in designing, building, and operating data platforms at scale, encompassing both batch processing and real-time streaming architectures. Expertise in data warehousing concepts, advanced ETL/ELT processes, and analytics engineering best practices. Exceptional programming ...

resilience, availability, performance and capacity. * Manage deployments, releases, production readiness and failover activities. * Implement monitoring, alerting and observability solutions. * Troubleshoot complex application, infrastructure and batch-processing issues. * Identify root causes and deliver sustainable technical solutions. * Contribute to CI/CD, automation and wider DevOps and engineering best practices. ...

activities, including data analysis, mapping, transformation, validation and resolution of complex migration issues. Support integrations between ANAQUA and other platforms or systems, including batch processes and software development where required. Work effectively within a matrix organisation, collaborating with Project Managers, Business Consultants, technical teams and wider stakeholders to coordinate ...

data mapping, transformation, reconciliation, and validation. * Strong SQL skills and experience working with relational databases. * Experience with Scripting and/or automation for data processing and migration. * Good understanding of integration concepts such as APIs, messaging, batch processing, and file-based interfaces. * Experience troubleshooting complex technical ...

maintain reports using Endur reporting tools Support CI/CD processes and deployments within Endur environments Troubleshoot issues across development, testing, and production Support batch processes and EOD activities as needed Work with business users and analysts to understand requirements and deliver working solutions Required (Endur-specific) Hands … working within Endur reporting tools (Report Builder or similar) Familiarity with CI/CD processes in Endur environments Experience supporting or working with Endur batch/EOD processes 3–5+ years of Endur‐specific development experience Transferable experience (CTRM/general development) Development experience with Java, JavaScript, .NET ...