1 to 25 of 96 Remote Batch Processing Jobs in the UK

infrastructure. They already have a Kubernetes platform (the Internal Kubernetes Platform or IKP) but this is focused on the support of microservices/mesh applications. They require a large batch processing Kubernetes platform. Role Define a 'Kubernetes batch' stack To review the architecture of our existing IKP with a view to reuse components for Kubernetes batch. To More ❯

and maintain ETL pipelines integrating data from sources like Facebook, Google Analytics, and payment providers. Using tools like AWS Redshift, S3, and Kafka, you'll optimize data models for batch and real-time processing. Collaborating with stakeholders, you'll deliver actionable insights on player behavior and gaming analytics, enhancing experiences and driving revenue with dashboards in Tableau. You'll … include: Design, implement, and maintain robust ETL pipelines to integrate data from diverse sources, including APIs like Facebook, Google Analytics, and payment providers. Develop and optimize data models for batch processing and real-time streaming using tools like AWS Redshift, S3, and Kafka. Lead efforts in acquiring, storing, processing, and provisioning data to meet evolving business requirements. More ❯

Python, SKLearn, XGBoost, SparkML, etc. Experience with deep learning frameworks like TensorFlow or PyTorch. Knowledge of ML model deployment options (e.g., Azure Functions, FastAPI, Kubernetes) for real-time and batch processing. Experience with CI/CD pipelines (e.g., DevOps pipelines, Git actions). Knowledge of infrastructure as code (e.g., Terraform, ARM Template, Databricks Asset Bundles). Understanding of advanced … machine learning techniques, including graph-based processing, computer vision, natural language processing, and simulation modeling. Experience with generative AI and LLMs, such as LLamaIndex and LangChain Understanding of MLOps or LLMOps. Familiarity with Agile methodologies, preferably Scrum We are actively seeking candidates for full-time, remote work within the UK. #J-18808-Ljbffr More ❯

raw data into processed data. You will own the data operations infrastructure, manage and optimise performance, reliability, and scalability of the system to meeting growing demands on ingestion and processing pipelines. To succeed in this data engineering position, you should have strong problem-solving skills and the ability to combine data from different sources. Data engineer skills also include … structured or unstructured solutions. Design, Develop, Deploy and Support data infrastructure, pipelines and architecture. Implement reliable, scalable, and tested solutions to automate data ingestion. Development of systems to manage batch processing and real-time streaming of data Evaluate business needs and objectives. Facilitate pipelines, which prepare data for prescriptive and predictive modelling. Working with domain teams to scale … the processing of data. Combine raw information from different sources. Manage and maintain automated tools for data quality and reliability. Explore ways to enhance data quality and reliability. Collaborate with data scientists, IT and architects on several projects What you'll bring Successful Candidates will have previous experience as a data or software engineer in a similar role. Attributes More ❯

of the best, most efficient use of associated tools and applications to complete these tasks. Ability to travel to client site, where required, will be a consideration. Experience in processing large amounts of structured and unstructured data, including integrating data from multiple sources through ingestion and curation functions on AWS cloud using AWS native or custom programming. Knowledge of … data mining, machine learning, natural language processing is an advantage. You enjoy working within cross-functional Agile teams and you are familiar with Scrum ceremonies. You’ll be comfortable designing and building for the AWS cloud and will have designed and worked on architectures that include Platform-as-a-Service components and perhaps even server-less and container technologies. … lifecycle in line with modern AWS best practices. Your AWS experience spans data engineering, data science and product development projects, plus you will have an understanding of stream and batch processing. We are currently operating a discretionary hybrid working model which is designed to help you plan your work and your life. We want our people to come into More ❯

experience with AWS data and analytics services (or equivalent open-source technologies). Expertise in designing and building data lakes, data warehouses, and ETL pipelines. Strong understanding of data processing techniques, including stream and batch processing. Familiarity with data mining, machine learning, and natural language processing is a plus. Ability to travel to client sites when required. More ❯

S3 to build and deploy cloud-based solutions, ensuring high availability and scalability. Database Management: Design, implement, and maintain database schemas, write complex SQL queries, and optimize database performance. Batch Processing: Develop and manage batch processing systems to handle large volumes of data efficiently. Code Quality: Ensure code quality through code reviews, unit testing, and adherence … APIs. Problem-Solving: Excellent analytical and problem-solving skills. Communication: Strong verbal and written communication skills. Team Player: Ability to work effectively in a collaborative team environment. Preferred Qualifications : Batch Processing experience: Hands-on experience with batch processing frameworks and tools. Python experience: Minimum of 2 years is nice to have. Additional Information Why Version More ❯

Responsibilities: Design, implement, and maintain robust ETL pipelines to integrate data from diverse sources, including APIs like Facebook, Google Analytics, and payment providers. Develop and optimize data models for batch processing and real-time streaming using tools like AWS Redshift, S3, and Kafka. Lead efforts in acquiring, storing, processing, and provisioning data to meet evolving business requirements. More ❯

they operate. Hands on Experience in Java , Spark , Scala ( or Java) Production scale hands-on Experience to write Data pipelines using Spark/any other distributed real time/batch processing. Strong skill set in SQL/Databases Strong understanding of Messaging tech like Kafka, Solace , MQ etc. Writing production scale applications to use the Caching technologies. Understanding of More ❯

Responsibilities: Design, implement, and maintain robust ETL pipelines to integrate data from diverse sources, including APIs like Facebook, Google Analytics, and payment providers. Develop and optimize data models for batch processing and real-time streaming using tools like AWS Redshift, S3, and Kafka. Lead efforts in acquiring, storing, processing, and provisioning data to meet evolving business requirements. More ❯

Responsibilities: Design, implement, and maintain robust ETL pipelines to integrate data from diverse sources, including APIs like Facebook, Google Analytics, and payment providers. Develop and optimize data models for batch processing and real-time streaming using tools like AWS Redshift, S3, and Kafka. Lead efforts in acquiring, storing, processing, and provisioning data to meet evolving business requirements. More ❯

Responsibilities: Design, implement, and maintain robust ETL pipelines to integrate data from diverse sources, including APIs like Facebook, Google Analytics, and payment providers. Develop and optimize data models for batch processing and real-time streaming using tools like AWS Redshift, S3, and Kafka. Lead efforts in acquiring, storing, processing, and provisioning data to meet evolving business requirements. More ❯

Responsibilities: Design, implement, and maintain robust ETL pipelines to integrate data from diverse sources, including APIs like Facebook, Google Analytics, and payment providers. Develop and optimize data models for batch processing and real-time streaming using tools like AWS Redshift, S3, and Kafka. Lead efforts in acquiring, storing, processing, and provisioning data to meet evolving business requirements. More ❯

Responsibilities: Design, implement, and maintain robust ETL pipelines to integrate data from diverse sources, including APIs like Facebook, Google Analytics, and payment providers. Develop and optimize data models for batch processing and real-time streaming using tools like AWS Redshift, S3, and Kafka. Lead efforts in acquiring, storing, processing, and provisioning data to meet evolving business requirements. More ❯

Responsibilities: Design, implement, and maintain robust ETL pipelines to integrate data from diverse sources, including APIs like Facebook, Google Analytics, and payment providers. Develop and optimize data models for batch processing and real-time streaming using tools like AWS Redshift, S3, and Kafka. Lead efforts in acquiring, storing, processing, and provisioning data to meet evolving business requirements. More ❯

company. Responsibilities Partner with stakeholders across Product, Data, and Engineering to translate strategic goals into platform capabilities. Lead and mentor 1-2 squads responsible for experimentation, analytics event logging, batch data platform, real-time infrastructure, data observability and governance. Collaborate closely with stakeholders to define and drive the technical roadmap for Depop's modern data platform, enabling reliable and … Deep understanding of distributed systems and modern data ecosystems - including experience with Databrick, Apache Spark, Apache Kafka and DBT. Demonstrated success in managing data platforms at scale, including both batch processing and real-time streaming architectures. Deep understanding of data warehousing concepts, ETL/ELT processes, and analytics engineering. Strong programming skills, particularly in Python, Scala or Java More ❯

scalability, and maintainability. Data Integration & Pipeline Engineering: Design and implement resilient ETL/ELT workflows using technologies such as BigQuery, Dataflow, Informatica, or IBM DataStage, supporting real-time and batch processing. Governance & Data Quality Management: Establish data governance frameworks, including metadata management and quality assurance, using platforms like Unity Catalog, Alation, Profisee, or DQ Pro. Strategic Advisory & Stakeholder Engagement More ❯

field Experience: Proven experience in ML model lifecycle management Core Competencies: Model lifecycle: You’ve got hands-on experience with managing the ML model lifecycle, including both online and batch processes Statistical Methodology: You have worked with GLMs and other machine learning algorithms and have in-depth knowledge of how they work Python: You have built and deployed production More ❯

and review code, paired programming, and debugging code related performance issues, SQL tuning, etc. Experience with AWS services such as S3, RDS, Aurora, NoSQL, MSK (Kafka). Experience with batch processing/ETL using Glue Jobs, AWS Lambda, and Step functions. Experience with designing bespoke & tailored front-end solutions (GUI based) using open source technology such as React More ❯

a part of our Wellcreator development team. Duties & Responsibilities: Hands-on experience in developing applications using Java EE platforms, Spring Framework, Spring Boot, Hibernate, Java Swing, Java FX, JPA, Batch Processing, Java 8, or higher versions. Expertise in producing RESTful APIs and back-end services. Experience in agile development, GIT, or Maven. Familiarity with continuous integration and build More ❯

As a Senior Data Engineer, you'll build and maintain the robust data systems that power our analytics and operational needs.You'll work across our entire data platform, from batch processing to real-time event streams, ensuring our solutions are scalable, maintainable, and deliver real value to our users. You'll have the opportunity to work with modern More ❯

modelling, data access patterns, and efficient use of databases (MSSQL, PostgreSQL, MySQL). Work with data engineers to enable secure, efficient data movement between systems, including real-time and batch processing pipelines. Support analytics and reporting needs through the engineering of clean, reliable data sources and APIs. Security Engineering & Governance Partner with security engineers to integrate security throughout More ❯

modelling, data access patterns, and efficient use of databases (MSSQL, PostgreSQL, MySQL). Work with data engineers to enable secure, efficient data movement between systems, including real-time and batch processing pipelines. Support analytics and reporting needs through the engineering of clean, reliable data sources and APIs. Security Engineering & Governance Partner with security engineers to integrate security throughout More ❯

modelling, data access patterns, and efficient use of databases (MSSQL, PostgreSQL, MySQL). Work with data engineers to enable secure, efficient data movement between systems, including real-time and batch processing pipelines. Support analytics and reporting needs through the engineering of clean, reliable data sources and APIs. Security Engineering & Governance Partner with security engineers to integrate security throughout More ❯

modelling, data access patterns, and efficient use of databases (MSSQL, PostgreSQL, MySQL). Work with data engineers to enable secure, efficient data movement between systems, including real-time and batch processing pipelines. Support analytics and reporting needs through the engineering of clean, reliable data sources and APIs. Security Engineering & Governance Partner with security engineers to integrate security throughout More ❯