13 of 13 Batch Processing Jobs in London

Design Implement enterprise-level features such as: Dependency Injection/Inversion of Control (IoC) Aspect-Oriented Programming (AOP) Functional programming patterns Transaction management frameworks Batch processing solutions Participate in systems design and architecture discussions Support systems integration across enterprise platforms and services DevOps & Delivery Apply DevOps methodologies …/JEE and web technologies Knowledge of: Dependency Injection (e.g., Spring) AOP and IoC Data access frameworks (e.g., Hibernate/JPA) Transaction management systems Batch processing frameworks Familiarity with DevOps tools and practices Understanding of: Information security principles System integration and APIs Modern software engineering standards Key Competencies ...

experience AWS Azure Desirable Skills: Hands-on expertise in AWS & Spark: Amazon EMR, S3, Lambda; strong PySpark/Python and SQL for large-scale batch processing. Data engineering at scale in government or similarly complex domains, including performance tuning and data quality management. CI/CD & DevOps: pipelines ...

modern data engineering principles. Desirable Skills Python and PySpark development. AWS services including EMR, S3, Lambda, Athena, and Redshift. Apache Spark and large-scale batch processing. Terraform, CI/CD pipelines, and Infrastructure as Code. Git/GitLab and modern software development practices. API integration and data services. Data ...

collaborate more effectively, work smarter, and produce high-quality research more efficiently. Key responsibilities: Design, build, and maintain scalable data pipelines and services Develop batch and ELT/ETL data workflows for ingesting and processing large datasets Work with technologies such as Python, Java, Spark, SQL, and Airflow … Strong skills in Python and experience with at least one backend language (such as Java) Understanding of data engineering concepts such as data lakes, batch processing, and distributed systems Experience building or operating data pipelines using tools like Spark and Airflow Familiarity with cloud platforms (AWS preferred ...

metadata-forward, CI/CD-driven platform represents and enables the entire application and analysis lifecycle including interactive development and explorations (notebooks), large-scale batch processing, observability and production application deployments. A Sr. Compute Platform Engineer is a leading technical contributor who can consistently take a poorly defined … business problems, Responsible for development of key components of a hybrid on-prem/cloud compute platform for both interactive and scalable batch computing and establishing of processes and workflows to transition existing HPC users and teams to this platform Responsible for code-driven environment, applications, and container/ ...

with observability frameworks (Prometheus, ElasticSearch , Grafana, OpenTelemetry ) for metrics, logs and traces Platform Technologies & BSS Vendor Platforms - Open source workflow engines (Temporal) - Stream/batch processing (Flink, Spark) - Test automation: Robot Framework and BDD - can review BDD robot files - BSS vendor platforms: Salesforce, Amdocs, Ciena - BSS/ ...

Data Agent Kit, MCP Toolbox for Databases)Familiarity with dbt for transformation and data modeling at scaleExperience building or operating streaming data infrastructure alongside batch processingBackground in compliance-sensitive environments (SOX, regulatory reporting, audit systems)Published work, conference talks, or open-source contributions in knowledge engineering, semantic ...

leadership and mentorship to cross-functional data teams Define best practices across data architecture, cloud engineering, and security Design and oversee real-time and batch processing solutions, including streaming architectures Ensure compliance with government standards, security policies, and data regulations Monitor delivery against architectural designs and manage ...

function, setting standards, mentoring engineers, and working closely with both internal teams and external delivery partners. The Role Data Engineering & Pipeline Development Transform legacy batch processes into scalable, observable data pipelines Build and deliver high-quality data products in line with architectural designs Work across modern cloud platforms including ...

Role Design and deliver APIs using API-led connectivity (System, Process, Experience layers) Build complex integration solutions including event-driven architectures, orchestration, streaming, and batch processing Lead development on key projects, ensuring high-quality engineering standards Produce reusable assets, modules, and DataWeave transformations Contribute to API specifications (RAML ...

Role Design and deliver APIs using API-led connectivity (System, Process, Experience layers) Build complex integration solutions including event-driven architectures, orchestration, streaming, and batch processing Lead development on key projects, ensuring high-quality engineering standards Produce reusable assets, modules, and DataWeave transformations Contribute to API specifications (RAML ...

dataset publishing* Build and maintain FastAPI backends, vanilla JavaScript frontends, CLI tooling, and supporting services* Improve recording workflows, state machines, config validation, batch processing, stale-data recovery, and archival processes* Containerise and orchestrate services to ensure reliable deployment across machines and lab environments* Partner with ML engineers … pipelines, distributed systems, or performance-sensitive IO workloads* Experience orchestrating external CLI tools from Python via subprocess, ideally ffmpeg or similar data/media processing tools Nice to Have It would be useful to have: * Distributed ML training infrastructure experience, such as PyTorch DDP, FSDP, DeepSpeed, sharded datasets, streaming ...

highly desirable. Strong capability across Salesforce Lightning, Flow automation, and security configuration is expected, along with familiarity in Apex development (including triggers and batch processing) and Lightning Web Components. Experience with Omnistudio (including Omniscripts, Flex Cards, DataRaptors, and Integration Procedures) will be highly advantageous, particularly in enterprise-scale ...