16 of 16 Batch Processing Jobs in London

Design Implement enterprise-level features such as: Dependency Injection/Inversion of Control (IoC) Aspect-Oriented Programming (AOP) Functional programming patterns Transaction management frameworks Batch processing solutions Participate in systems design and architecture discussions Support systems integration across enterprise platforms and services DevOps & Delivery Apply DevOps methodologies …/JEE and web technologies Knowledge of: Dependency Injection (e.g., Spring) AOP and IoC Data access frameworks (e.g., Hibernate/JPA) Transaction management systems Batch processing frameworks Familiarity with DevOps tools and practices Understanding of: Information security principles System integration and APIs Modern software engineering standards Key Competencies ...

experience AWS Azure Desirable Skills: Hands-on expertise in AWS & Spark: Amazon EMR, S3, Lambda; strong PySpark/Python and SQL for large-scale batch processing. Data engineering at scale in government or similarly complex domains, including performance tuning and data quality management. CI/CD & DevOps: pipelines ...

modern data engineering principles. Desirable Skills Python and PySpark development. AWS services including EMR, S3, Lambda, Athena, and Redshift. Apache Spark and large-scale batch processing. Terraform, CI/CD pipelines, and Infrastructure as Code. Git/GitLab and modern software development practices. API integration and data services. Data ...

collaborate more effectively, work smarter, and produce high-quality research more efficiently. Responsibilities: Design, build, and maintain scalable data pipelines and services Develop batch and ELT/ETL data workflows for ingesting and processing large datasets Work with technologies such as Python, Java, Spark, SQL, and Airflow Apply … Strong skills in Python and experience with at least one backend language (such as Java) Understanding of data engineering concepts such as data lakes, batch processing, and distributed systems Experience building or operating data pipelines using tools like Spark and Airflow Familiarity with cloud platforms (AWS preferred ...

noting that you will be unable to work at home 100% of the time. Your Role: Design, develop, and maintain scalable data pipelines and processing frameworks using Databricks, Python, and PySpark Build and optimize data workflows to support analytics, reporting, and operational needs Collaborate with cross-functional teams … data lakes and (optionally) Snowflake Your Skills: Strong hands-on experience with Databricks (must-have) Proficiency in Python and PySpark for large-scale data processing Solid understanding of data engineering fundamentals (ETL/ELT, batch processing, transformations) Proven experience building scalable and efficient data pipelines Exposure ...

metadata-forward, CI/CD-driven platform represents and enables the entire application and analysis lifecycle including interactive development and explorations (notebooks), large-scale batch processing, observability and production application deployments. A Compute Platform Engineer II is a technical contributor who can consistently take a poorly defined business … business problems, Responsible for development of key components of a hybrid on-prem/cloud compute platform for both interactive and scalable batch computing and establishing of processes and workflows to transition existing HPC users and teams to this platform Responsible for code-driven environment, applications, and container/ ...

with observability frameworks (Prometheus, ElasticSearch , Grafana, OpenTelemetry ) for metrics, logs and traces Platform Technologies & BSS Vendor Platforms - Open source workflow engines (Temporal) - Stream/batch processing (Flink, Spark) - Test automation: Robot Framework and BDD - can review BDD robot files - BSS vendor platforms: Salesforce, Amdocs, Ciena - BSS/ ...

Rate: 300 GBP Per Day Eight years of experience with Mainframe technologies including COBOL, CICS, JCL, DB2, and VSAM, with proven expertise in batch processing and online transaction processing (OLTP) environmen tsSolid experience in application support, with a good understanding of Incident and Problem Management processes, along ...

leadership and mentorship to cross-functional data teams Define best practices across data architecture, cloud engineering, and security Design and oversee real-time and batch processing solutions, including streaming architectures Ensure compliance with government standards, security policies, and data regulations Monitor delivery against architectural designs and manage ...

function, setting standards, mentoring engineers, and working closely with both internal teams and external delivery partners. The Role Data Engineering & Pipeline Development Transform legacy batch processes into scalable, observable data pipelines Build and deliver high-quality data products in line with architectural designs Work across modern cloud platforms including ...

Role Design and deliver APIs using API-led connectivity (System, Process, Experience layers) Build complex integration solutions including event-driven architectures, orchestration, streaming, and batch processing Lead development on key projects, ensuring high-quality engineering standards Produce reusable assets, modules, and DataWeave transformations Contribute to API specifications (RAML ...

dataset publishing* Build and maintain FastAPI backends, vanilla JavaScript frontends, CLI tooling, and supporting services* Improve recording workflows, state machines, config validation, batch processing, stale-data recovery, and archival processes* Containerise and orchestrate services to ensure reliable deployment across machines and lab environments* Partner with ML engineers … pipelines, distributed systems, or performance-sensitive IO workloads* Experience orchestrating external CLI tools from Python via subprocess, ideally ffmpeg or similar data/media processing tools Nice to Have It would be useful to have: * Distributed ML training infrastructure experience, such as PyTorch DDP, FSDP, DeepSpeed, sharded datasets, streaming ...

highly desirable. Strong capability across Salesforce Lightning, Flow automation, and security configuration is expected, along with familiarity in Apex development (including triggers and batch processing) and Lightning Web Components. Experience with Omnistudio (including Omniscripts, Flex Cards, DataRaptors, and Integration Procedures) will be highly advantageous, particularly in enterprise-scale ...

solutions using Business Objects and related tools • Ensure payroll processes remain compliant, auditable and fully supported • Identify opportunities to automate manual activities through workflows, batch processes, scheduled reporting and process chains • Support and enhance integrations between iTrent and third-party applications • Deliver end-user training and help develop system ...

care and focus on customer improvements Administer the change control procedures and acceptance Identify opportunities for automation and improved communication through the use of Batch Processes, Scheduled Reports, Workflows and Process Chains/Chesklists Ensure our payroll is ‘audit ready’ and assist users with producing user audit journeys ...

offers the chance to shape the future-state data platform for a large, complex financial services organisation. The focus is on moving from slow, batch-based analytics towards a more modern, streaming-led architecture, with clear ownership over data vision, use cases, and delivery direction. The Company They … Databricks-based medallion architecture, with a strong focus on a robust Gold layer for analytics. Improve data availability by shifting from monthly batch processes towards more frequent or streaming ingestion where appropriate. Act as the senior technical decision-maker on data use cases, prioritisation, and trade-offs. Work closely ...