24 of 24 Batch Processing Jobs in London

collaboration, and high performance. Hold team members accountable for their performance and professional growth. Technical Expertise: Bring a deep understanding of data technologies, including batch processing, stream processing, storage, analytics, machine learning, and AI. Collaborate with Engineering teams to ensure the data platform meets technical requirements and … data-focused environment. Strong technical background and a degree in Computer Science, Engineering, or a related field. Hands-on experience with data technologies, including batch processing, stream processing, data warehousing, and analytics platforms. Proven track record of successfully leading product strategies that drive significant business outcomes. Experience … experience in mentoring and developing Product Managers. Ability to inspire and lead cross-functional teams toward common goals. Deep understanding of data architectures, data processing frameworks, and data modelling. Familiarity with technologies such as AWS and other cloud-based data services. Knowledge of data privacy laws and compliance requirements More ❯

metadata-forward, CI/CD-driven platform represents and enables the entire application and analysis lifecycle including interactive development and explorations (notebooks), large-scale batch processing, observability and production application deployments. The optimization team's focus is on maximizing scale and performance of all aspects of the platforms. … for delivery of scalable solutions to the Compute and AIML Platforms that supports the entire application lifecycle (interactive development and explorations/analysis, scalable batch processing, application deployment) with particular focus on performance at scale Partner with both AIML and Compute platform teams as well as scientific users More ❯

metadata-forward, CI/CD-driven platform represents and enables the entire application and analysis lifecycle including interactive development and explorations (notebooks), large-scale batch processing, observability and production application deployments. The optimization team's focus is on maximizing scale and performance of all aspects of the platforms. … for delivery of scalable solutions to the Compute and AIML Platforms that supports the entire application lifecycle (interactive development and explorations/analysis, scalable batch processing, application deployment) with particular focus on performance at scale Partner with both AIML and Compute platform teams as well as scientific users More ❯

experts, stakeholders, and senior management across the NHSCFA and our wider stakeholders. The primary responsibilities will include developing, evaluating, maintaining, and managing data pipelines, processing, preparing data suitable for analysis, and developing and maintaining data models. You will support the design of analytical services, products, and models that meet … Lakehouses. Build data pipelines that clean, transform, and present granular and aggregate data from disparate sources. Design, build, test, automate, and maintain architectures and processing workflows for analytics and business intelligence (BI) systems which are scalable. Plan, develop and evaluate methods and processes for gathering, extracting, transforming, and cleaning … data warehouse, infrastructure and ETL solutions covering multiple sources of data. Knowledge of using techniques to ensure datasets and efficient queries are optimised, including batch processing, partitions, and indexing etc. Ability to build API integration to integrate various external NHS data sources and optimise code and data pipelines. More ❯

robust ETL pipelines to integrate data from diverse sources, including APIs like Facebook, Google Analytics, and payment providers. Develop and optimize data models for batch processing and real-time streaming using tools like AWS Redshift, S3, and Kafka. Lead efforts in acquiring, storing, processing, and provisioning data More ❯

business area. Comfortable making presentations covering business, technical or sales. Detailed knowledge of; Project phasing and reporting On-site delivery Systems support Operational procedures Batch processing System sizing and capacity planning System backup, contingency and disaster recovery Regional roll outs Education & Preferred Qualifications Third level qualification essential ideally More ❯

business area. Comfortable making presentations covering business, technical or sales. Detailed knowledge of; Project phasing and reporting. On-site delivery. Systems support. Operational procedures. Batch processing. System sizing and capacity planning. System backup, contingency and disaster recovery. Regional roll outs. Education & Preferred Qualifications Third level qualification essential ideally a More ❯

for end-to-end ETL, ELT & reporting solutions using key components like Spark SQL & Spark Streaming. Strong knowledge of multi-threading and high-volume batch processing. Proficiency in performance tuning for Java/Python and Spark. Deep knowledge of AWS products/services and Kubernetes/container technologies and More ❯

to support LLM-based querying, semantic search, and metadata retrieval. Integrate structured (SQL-based) and unstructured (documents, reports) data sources for real-time and batch processing. Maintain and troubleshoot Airflow pipelines for embedding extraction and document processing. Ensure data governance, security, and compliance across all applications. Manage Vector Database More ❯

in a BI leadership role in a global or matrixed organisation. Proven expertise in modern BI architecture (Data Lake, EDW, Streaming, APIs, Real-Time & Batch Processing). Demonstrated experience delivering cloud-based analytics platforms (Azure, AWS, GCP). Strong knowledge of data governance, cataloguing, security, automation, and self More ❯

and debugging code related performance issues, SQL tuning, etc. Experience with AWS services such as S3, RDS, Aurora, NoSQL, MSK (Kafka). Experience with batch processing/ETL using Glue Jobs, AWS Lambda, and Step functions. Experience with designing bespoke & tailored front-end solutions (GUI based) using open More ❯

Service Bus. Design, develop and maintain MS SQL Server relational databases with change tracking/CDC. Keep multiple applications in sync - real time/batch processing. Responsible for the delivery of the non-functional requirements within a project. Work with the architect to help set the development coding standards More ❯

London Based Investment Bank You will: Design and build foundation components that will underpin our data mesh ecosystem. Build enterprise class real-time and batch solutions that support mission critical processes. Build solutions in line with our Digital Principles. Partner with our Product team(s) to create sustainable and … hands-on engineering in large scale complex Enterprise(s), ideally in the banking/financial industry. Worked with modern tech - data streaming, real-time & batch processing and compute clusters. Working knowledge of relational and NoSQL databases, designing and implementing scalable solutions. Experience of working in continuous architecture environment More ❯

business objectives. Manage the technical product backlog, prioritising system enhancements and platform stability. Work closely with engineering and architecture teams to ensure API integrations, batch processing, and data flow across platforms. Lead and oversee broker and partner onboarding, ensuring seamless integration (direct, API, or via Software House solutions More ❯

As a Senior Data Engineer, you will work within a collaborative data team to design, build, and optimise data pipelines for real-time and batch processing. Your work will directly impact business intelligence, analytics, and AI/ML applications. Key Responsibilities Develop and maintain scalable data pipelines using Python … SQL, and Databricks Implement ELT frameworks for transforming, enriching, and processing large datasets Design data models and optimize query performance and cost-efficiency Ensure data security, compliance, and governance (GDPR, privacy regulations) Build and automate CI/CD pipelines for efficient data operations Work with real-time data processing … Skills & Experience Strong expertise in Python and SQL for data engineering Hands-on experience with Databricks, Spark, Delta Lake, Delta Live Tables Experience in batch and real-time data processing Proficiency with cloud platforms (AWS, Azure, Databricks) Solid understanding of data security, privacy regulations, and governance Experience in More ❯

As a Senior Data Engineer, you will work within a collaborative data team to design, build, and optimise data pipelines for real-time and batch processing. Your work will directly impact business intelligence, analytics, and AI/ML applications. Key Responsibilities Develop and maintain scalable data pipelines using Python … SQL, and Databricks Implement ELT frameworks for transforming, enriching, and processing large datasets Design data models and optimize query performance and cost-efficiency Ensure data security, compliance, and governance (GDPR, privacy regulations) Build and automate CI/CD pipelines for efficient data operations Work with real-time data processing … Skills & Experience Strong expertise in Python and SQL for data engineering Hands-on experience with Databricks, Spark, Delta Lake, Delta Live Tables Experience in batch and real-time data processing Proficiency with cloud platforms (AWS, Azure, Databricks) Solid understanding of data security, privacy regulations, and governance Experience in More ❯

to improve and optimize processes Responsibilities: Responsible for the support & integration of Guidewire application to business estates Identify opportunities and propose integration solutions including batch processing and message queues Manage and provide technical for Guidewire applications in production. Liaise & interact with 3rd party vendors Engage with technical teams More ❯

to improve and optimize processes Responsibilities: Responsible for the support & integration of Guidewire application to business estates Identify opportunities and propose integration solutions including batch processing and message queues Manage and provide technical for Guidewire applications in production. Liaise & interact with 3rd party vendors Engage with technical teams More ❯

Files: Handling and editing large multi-layered compositions efficiently. Proficient in file management, ensuring organised and accessible digital assets. Use of Photoshop actions and batch processing for streamlined workflows. Experience in CGI post-production, working with renders and render passes. An understanding of the VRAY 'Back to Beauty More ❯

use of associated tools and applications to complete these tasks. Ability to travel to client site, where required, will be a consideration. Experience in processing large amounts of structured and unstructured data, including integrating data from multiple sources through ingestion and curation functions on AWS cloud using AWS native … or custom programming. Knowledge of data mining, machine learning, natural language processing is an advantage. You enjoy working within cross-functional Agile teams and you are familiar with Scrum ceremonies. Youll be comfortable designing and building for the AWS cloud and will have designed and worked on architectures that … AWS best practices. Your AWS experience spans data engineering, data science and product development projects, plus you will have an understanding of stream and batch processing. Sound interesting? Apply now! Skills: AWS Data engineer Benefits: Annual Bonus/13th Cheque hybrid pension Medical Aid/Health Care More ❯

use of associated tools and applications to complete these tasks. Ability to travel to client site, where required, will be a consideration. Experience in processing large amounts of structured and unstructured data, including integrating data from multiple sources through ingestion and curation functions on AWS cloud using AWS native … or custom programming. Knowledge of data mining, machine learning, natural language processing is an advantage. You enjoy working within cross-functional Agile teams and you are familiar with Scrum ceremonies. Youll be comfortable designing and building for the AWS cloud and will have designed and worked on architectures that … AWS best practices. Your AWS experience spans data engineering, data science and product development projects, plus you will have an understanding of stream and batch processing. Sound interesting? Apply now! Skills: AWS Data engineer Benefits: Annual Bonus/13th Cheque hybrid pension Medical Aid/Health Care More ❯

use of associated tools and applications to complete these tasks. Ability to travel to client site, where required, will be a consideration. Experience in processing large amounts of structured and unstructured data, including integrating data from multiple sources through ingestion and curation functions on AWS cloud using AWS native … or custom programming. Knowledge of data mining, machine learning, natural language processing is an advantage. You enjoy working within cross-functional Agile teams and you are familiar with Scrum ceremonies. Youll be comfortable designing and building for the AWS cloud and will have designed and worked on architectures that … AWS best practices. Your AWS experience spans data engineering, data science and product development projects, plus you will have an understanding of stream and batch processing. Sound interesting? Apply now! Skills: AWS Data engineer Benefits: Annual Bonus/13th Cheque hybrid pension Medical Aid/Health Care More ❯

least 2 years experience of Nice Actimize. - Analyzing current installation of Actimize WLF solution at a client side - Recommend improvements on data ingestion and batch process performance - Recommend optimal Threshold setting and Scoring configuration which adhere to the client requirements - Improve detection and alert generation Must Have: - Minimum More ❯

in mechanical, chemical, or electrical disciplines with minimum 5 years' experience, supporting or developing automated systems. Strong understanding of automation standards and architecture addressing batch process control, such as S88, and related automation systems, including process control systems, manufacturing execution systems, and process data historians. Experience in pharmaceutical/… BACnet, and/or Profibus. Demonstrates skills in requirements gathering, design, configuration, integration, and implementation of process control solutions for the manufacturing industry including batch software, PLC-based control systems, HMI packages, building management systems, SCADA systems, and PC networking technology. Familiarity with FDA GMPs and FDA regulations relating More ❯