19 of 19 Batch Processing Jobs in London

market data, reference data, pricing data, data quality, data lineage, data controls, and integration workflows. Mandatory experience analysing and documenting data pipelines, APIs, batch processes, event-driven workflows, databases, file-based interfaces, or cloud-based data platforms. Ability to write clear functional specifications, data mappings, user stories, acceptance criteria ...

market data, reference data, pricing data, data quality, data lineage, data controls, and integration workflows. Mandatory experience analysing and documenting data pipelines, APIs, batch processes, event-driven workflows, databases, file-based interfaces, or cloud-based data platforms. Ability to write clear functional specifications, data mappings, user stories, acceptance criteria ...

systems while preserving provenance and auditability. Build APIs and services that make processed data available to products, customers and AI systems. Develop workflows for batch processing, event-driven processing and long-running computational jobs. Create monitoring and alerting for pipeline failures, unexpected data changes and quality regressions. ...

metadata-forward, CI/CD-driven platform represents and enables the entire application and analysis lifecycle including interactive development and explorations (notebooks), large-scale batch processing, observability and production application deployments. A Sr. Compute Platform Engineer is a leading technical contributor who can consistently take a poorly defined … business problems, Responsible for development of key components of a hybrid on-prem/cloud compute platform for both interactive and scalable batch computing and establishing of processes and workflows to transition existing HPC users and teams to this platform Responsible for code-driven environment, applications, and container/ ...

metadata-forward, CI/CD-driven platform represents and enables the entire application and analysis lifecycle including interactive development and explorations (notebooks), large-scale batch processing, observability and production application deployments. A Compute Platform Engineer II is a technical contributor who can consistently take a poorly defined business … business problems, Responsible for development of key components of a hybrid on-prem/cloud compute platform for both interactive and scalable batch computing and establishing of processes and workflows to transition existing HPC users and teams to this platform Responsible for code-driven environment, applications, and container/ ...

stop breaches, and we've redefined modern security with the world's most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion events per day and this traffic is growing daily. Our customers span all industries, and they count on CrowdStrike to keep … seeking a talented Senior Software Engineer to join our Real-Time Streaming Platform team. Our mission is to provide enterprise-grade streaming processing capabilities that empower teams across CrowdStrike to build real-time data pipelines, perform complex event processing, and derive actionable insights from massive data streams. ...

data or software engineer, including some exposure to production systems. Proficiency in Python, SQL, and Spark. Basic understanding of data engineering concepts including batch processing and relational databases. Some hands-on experience with data pipeline tools; exposure to Spark or Airflow is a plus. Familiarity with cloud platforms ...

data engineering squad, responsible for delivering the data infrastructure, pipelines, and reporting capabilities the wider business depends on. They are moving away from traditional batch processing toward a real-time, event-driven architecture and you will be leading that technical journey while keeping everything running reliably … Real-time and event-driven data platform experience Lakehouse or cloud-native data platform architecture experience Data lakes, object storage, and large-scale analytical processing ML or advanced analytics exposure AWS security, networking and access control GDPR and data governance awareness Day to Day: Leading and developing your data ...

platforms used by banks, governments and regulated organisations. Their software helps organisations understand exactly who they're doing business with, combining large-scale data processing, entity resolution, graph intelligence and modern cloud architecture. Join an engineering team solving genuinely difficult technical problems - processing millions of records through asynchronous … Design and evolve backend services for high-volume, distributed screening workloads Build event-driven pipelines (AWS SNS/SQS) with idempotent consumers, DLQs and batch processing Design clean APIs and contribute to architecture across multiple products Integrate external identity, KYC and payments platforms (e.g. Senzing, Auth0, Stripe ...

Develop and optimise Oracle database objects (PL/SQL procedures, functions, queries) Write and maintain UNIX shell scripts for automation, job scheduling, and data processing Perform data analysis, mapping, and transformation for complex data sources Ensure data quality, integrity, and performance optimisation across ETL processes Collaborate with business … experience working in Data Warehouse/ETL environments Experience in data integration, transformation, and performance tuning Strong understanding of ETL frameworks, data pipelines, and batch processing Desirable Skills Working knowledge of SAS (especially in ETL context) Experience in migrating SAS workloads to Informatica ETLs Familiarity with data modelling ...

Role Design and deliver APIs using API-led connectivity (System, Process, Experience layers) Build complex integration solutions including event-driven architectures, orchestration, streaming, and batch processing Lead development on key projects, ensuring high-quality engineering standards Produce reusable assets, modules, and DataWeave transformations Contribute to API specifications (RAML ...

Ability to compartmentalise complex problems and propose practical, controlled solutions for production environments Strong problem-solving, analytical and troubleshooting skills across application, infrastructure and batch processing issues Excellent verbal and written communication skills, with ability to work effectively across technology, infrastructure and business stakeholders Experience working as part ...

microservices, APIs, cloud-native design). AI & Data Platform Architecture/Deployment: Architect robust, secure AI and data platforms that support real-time inference, batch processing, and scalable enterprise workloads, integrating with clients' incumbent technology estates. Implement hybrid-cloud and multi-cloud strategies using modern data infrastructure (Kafka ...

Full-Stack Engineers across various seniority levels. You will design and deliver secure, scalable and resilient financial technology solutions, developing Java services, APIs and batch-processing applications while contributing to modern Angular frontends. What you'll do: Build backend services, microservices, APIs and batch applications using Java ...

Design and oversee integration approaches between Endur and surrounding applications, data platforms and enterprise systems Shape integration and data architecture patterns across APIs, messaging, batch, event-driven flows, canonical data models and enterprise data platforms Facilitate architecture, solution design and requirements workshops with business and technology stakeholders Advise clients … Endur architecture, configuration principles and application deployment models Strong understanding of Endur technical components, including JVS/OpenComponents, Connex or equivalent integration mechanisms, reporting, batch processing, environment strategy, upgrade impacts and performance considerations Proven capability designing integrations between Endur and surrounding enterprise applications, data platforms and reporting environments ...

supporting end users with reporting, analytics, and data-related queries. Ability to investigate and resolve data anomalies and unexpected reporting results. Experience monitoring BW batch processing and service performance. Exposure to enterprise reporting, data integration, and business intelligence platforms. Experience working across cross-functional teams to deliver business ...

Sr. Compute Platform Engineer will design, build, and operate tools and workflows across on‐prem and cloud, with emphasis on automation, observability, and scalable batch processing. You will mentor teammates and set software engineering best practices. You will collaborate with science users to optimize performance on large datasets, apply ...

modern engineering and platform standards. Duties and Responsibilities Lead technical analysis, solution design and hands‐on development of applications, services, APIs, event‐driven components, batch processes and platform‐aligned capabilities. Act as a C#/.NET SME, setting engineering direction and ensuring solutions are clean, maintainable, testable and aligned … event‐driven solutions using appropriate messaging patterns including publish/subscribe, competing consumers, retries, dead‐letter handling, schema/version management and effectively‐once processing where required. Use AI‐enabled engineering tooling responsibly to improve delivery productivity, including integrated agentic coding assistants, prompt engineering, code review support and, where ...

offers the chance to shape the future-state data platform for a large, complex financial services organisation. The focus is on moving from slow, batch-based analytics towards a more modern, streaming-led architecture, with clear ownership over data vision, use cases, and delivery direction. The Company They … Databricks-based medallion architecture, with a strong focus on a robust Gold layer for analytics. Improve data availability by shifting from monthly batch processes towards more frequent or streaming ingestion where appropriate. Act as the senior technical decision-maker on data use cases, prioritisation, and trade-offs. Work closely ...