14 of 14 Batch Processing Jobs in London

Design Implement enterprise-level features such as: Dependency Injection/Inversion of Control (IoC) Aspect-Oriented Programming (AOP) Functional programming patterns Transaction management frameworks Batch processing solutions Participate in systems design and architecture discussions Support systems integration across enterprise platforms and services DevOps & Delivery Apply DevOps methodologies …/JEE and web technologies Knowledge of: Dependency Injection (e.g., Spring) AOP and IoC Data access frameworks (e.g., Hibernate/JPA) Transaction management systems Batch processing frameworks Familiarity with DevOps tools and practices Understanding of: Information security principles System integration and APIs Modern software engineering standards Key Competencies ...

experience AWS Azure Desirable Skills: Hands-on expertise in AWS & Spark: Amazon EMR, S3, Lambda; strong PySpark/Python and SQL for large-scale batch processing. Data engineering at scale in government or similarly complex domains, including performance tuning and data quality management. CI/CD & DevOps: pipelines ...

modern data engineering principles. Desirable Skills Python and PySpark development. AWS services including EMR, S3, Lambda, Athena, and Redshift. Apache Spark and large-scale batch processing. Terraform, CI/CD pipelines, and Infrastructure as Code. Git/GitLab and modern software development practices. API integration and data services. Data ...

with observability frameworks (Prometheus, ElasticSearch , Grafana, OpenTelemetry ) for metrics, logs and traces Platform Technologies & BSS Vendor Platforms - Open source workflow engines (Temporal) - Stream/batch processing (Flink, Spark) - Test automation: Robot Framework and BDD - can review BDD robot files - BSS vendor platforms: Salesforce, Amdocs, Ciena - BSS/ ...

environments. Degree in Computer Science, Software Engineering or equivalent. Knowledge of on‐premise and cloud technology operation. Experience with large‐scale real‐time and batch process orchestration, optionally with third‐party engines. Understanding of cyber‐security standards, secure‐by‐design principles. Agile framework experience (Scrum, Kanban). Background ...

experience working in Data Warehouse/ETL environments Experience in data integration, transformation, and performance tuning Strong understanding of ETL frameworks, data pipelines, and batch processing Desirable Skills Working knowledge of SAS (especially in ETL context) Experience in migrating SAS workloads to Informatica ETLs Familiarity with data modelling ...

Data Agent Kit, MCP Toolbox for Databases)Familiarity with dbt for transformation and data modeling at scaleExperience building or operating streaming data infrastructure alongside batch processingBackground in compliance-sensitive environments (SOX, regulatory reporting, audit systems)Published work, conference talks, or open-source contributions in knowledge engineering, semantic ...

leadership and mentorship to cross-functional data teams Define best practices across data architecture, cloud engineering, and security Design and oversee real-time and batch processing solutions, including streaming architectures Ensure compliance with government standards, security policies, and data regulations Monitor delivery against architectural designs and manage ...

function, setting standards, mentoring engineers, and working closely with both internal teams and external delivery partners. The Role Data Engineering & Pipeline Development Transform legacy batch processes into scalable, observable data pipelines Build and deliver high-quality data products in line with architectural designs Work across modern cloud platforms including ...

Role Design and deliver APIs using API-led connectivity (System, Process, Experience layers) Build complex integration solutions including event-driven architectures, orchestration, streaming, and batch processing Lead development on key projects, ensuring high-quality engineering standards Produce reusable assets, modules, and DataWeave transformations Contribute to API specifications (RAML ...

Role Design and deliver APIs using API-led connectivity (System, Process, Experience layers) Build complex integration solutions including event-driven architectures, orchestration, streaming, and batch processing Lead development on key projects, ensuring high-quality engineering standards Produce reusable assets, modules, and DataWeave transformations Contribute to API specifications (RAML ...

dataset publishing* Build and maintain FastAPI backends, vanilla JavaScript frontends, CLI tooling, and supporting services* Improve recording workflows, state machines, config validation, batch processing, stale-data recovery, and archival processes* Containerise and orchestrate services to ensure reliable deployment across machines and lab environments* Partner with ML engineers … pipelines, distributed systems, or performance-sensitive IO workloads* Experience orchestrating external CLI tools from Python via subprocess, ideally ffmpeg or similar data/media processing tools Nice to Have It would be useful to have: * Distributed ML training infrastructure experience, such as PyTorch DDP, FSDP, DeepSpeed, sharded datasets, streaming ...

highly desirable. Strong capability across Salesforce Lightning, Flow automation, and security configuration is expected, along with familiarity in Apex development (including triggers and batch processing) and Lightning Web Components. Experience with Omnistudio (including Omniscripts, Flex Cards, DataRaptors, and Integration Procedures) will be highly advantageous, particularly in enterprise-scale ...

systems where stability is critical. Data Savvy: Experience working with data pipelines and a solid grasp of the trade-offs between real-time and batch processing. Mindset: An eye for usability. You know how to make high-stakes, complex workflows feel simple and foolproof for non-technical users. ...