1 to 25 of 32 Batch Processing Jobs in the South East

Responsibilities: Design, implement, and maintain robust ETL pipelines to integrate data from diverse sources, including APIs like Facebook, Google Analytics, and payment providers. Develop and optimize data models for batch processing and real-time streaming using tools like AWS Redshift, S3, and Kafka. Lead efforts in acquiring, storing, processing, and provisioning data to meet evolving business requirements. More ❯

scale data migration and modernization initiatives. Architect end-to-end data platforms using Dataproc, Dataflow, Pub/Sub, BigQuery, Cloud Spanner, and Bigtable. Define and implement real-time and batch processing pipelines for complex use cases involving streaming analytics, ML feature engineering, and automation. Act as a trusted advisor to senior technical and business stakeholders across industries (e.g. … BigQuery, Cloud Spanner, and Bigtable. Hands-on experience with orchestration tools like Apache Airflow or Cloud Composer. Hands-on experience with one or more of the following GCP data processing services: Dataflow (Apache Beam), Dataproc (Apache Spark/Hadoop), or Composer (Apache Airflow). Proficiency in at least one scripting/programming language (e.g., Python, Java, Scala) for data More ❯

both internal and external) and join wider cross-functional teams. You’ll have some experience building distributed systems both on-prem and in public cloud, CI/CD pipelines, batch compute tooling and developer productivity tooling. You will be excited and committed to providing an excellent developer experience, with a constant eye on continuous improvement. Key responsibilities: Understanding the …/components that benefit multiple Software Engineering teams, as well as providing the mechanism for discovering and consuming shared components. Including but not limited to: K8s configuration and usability, batch processing and HPC interactions and hybrid cloud deployments Stay informed on open source and 3rd party tooling that we should consider adopting rather than building in-house Assist … approach to technical problem-solving Exposure to the following would be beneficial: Developer portals and software catalogues Infrastructure-as-code tools such as Ansible or Terraform Experience in using batch compute frameworks, HPC tooling Company Description For almost 50 years, Williams Racing has been at the forefront of one of the fastest sports on the planet, being one of More ❯

high motivated individual to bring a diverse skill set and proactive mindset to help ensure the stability, efficiency, and resilience of applications across esure. What you’ll do: Support batch processing and carry out proactive analysis across squads and services. Review Java code and suggest improvements to enhance quality and performance. Investigate and resolve production incidents, problems, and … change requests. Offer expertise on business processes for projects and production support issues. Maintain core Insurance systems and related infrastructure like Kubernetes and SOLR. Improve batch efficiency and reduce costs, including audit-ready data archiving strategies. Provide data analysis to support business decisions across Operations and Claims. Deliver patching, database fixes, and business-critical data correction services. What we More ❯

on Azure (e.g., Azure Data Lake, Synapse, Data Factory) and AWS (e.g., Redshift, Glue, S3, Lambda) . Strong background in data architecture , including data modeling, warehousing, real-time and batch processing, and big data frameworks. Proficiency with modern data tools and technologies such as Spark, Databricks, Kafka, or Snowflake (bonus). Knowledge of cloud security, networking, and cost More ❯

on Azure (e.g., Azure Data Lake, Synapse, Data Factory) and AWS (e.g., Redshift, Glue, S3, Lambda) . Strong background in data architecture , including data modeling, warehousing, real-time and batch processing, and big data frameworks. Proficiency with modern data tools and technologies such as Spark, Databricks, Kafka, or Snowflake (bonus). Knowledge of cloud security, networking, and cost More ❯

on Azure (e.g., Azure Data Lake, Synapse, Data Factory) and AWS (e.g., Redshift, Glue, S3, Lambda) . Strong background in data architecture , including data modeling, warehousing, real-time and batch processing, and big data frameworks. Proficiency with modern data tools and technologies such as Spark, Databricks, Kafka, or Snowflake (bonus). Knowledge of cloud security, networking, and cost More ❯

on Azure (e.g., Azure Data Lake, Synapse, Data Factory) and AWS (e.g., Redshift, Glue, S3, Lambda) . Strong background in data architecture , including data modeling, warehousing, real-time and batch processing, and big data frameworks. Proficiency with modern data tools and technologies such as Spark, Databricks, Kafka, or Snowflake (bonus). Knowledge of cloud security, networking, and cost More ❯

on Azure (e.g., Azure Data Lake, Synapse, Data Factory) and AWS (e.g., Redshift, Glue, S3, Lambda) . Strong background in data architecture , including data modeling, warehousing, real-time and batch processing, and big data frameworks. Proficiency with modern data tools and technologies such as Spark, Databricks, Kafka, or Snowflake (bonus). Knowledge of cloud security, networking, and cost More ❯

on Azure (e.g., Azure Data Lake, Synapse, Data Factory) and AWS (e.g., Redshift, Glue, S3, Lambda) . Strong background in data architecture , including data modeling, warehousing, real-time and batch processing, and big data frameworks. Proficiency with modern data tools and technologies such as Spark, Databricks, Kafka, or Snowflake (bonus). Knowledge of cloud security, networking, and cost More ❯

on Azure (e.g., Azure Data Lake, Synapse, Data Factory) and AWS (e.g., Redshift, Glue, S3, Lambda) . Strong background in data architecture , including data modeling, warehousing, real-time and batch processing, and big data frameworks. Proficiency with modern data tools and technologies such as Spark, Databricks, Kafka, or Snowflake (bonus). Knowledge of cloud security, networking, and cost More ❯

on Azure (e.g., Azure Data Lake, Synapse, Data Factory) and AWS (e.g., Redshift, Glue, S3, Lambda) . Strong background in data architecture , including data modeling, warehousing, real-time and batch processing, and big data frameworks. Proficiency with modern data tools and technologies such as Spark, Databricks, Kafka, or Snowflake (bonus). Knowledge of cloud security, networking, and cost More ❯

on Azure (e.g., Azure Data Lake, Synapse, Data Factory) and AWS (e.g., Redshift, Glue, S3, Lambda) . Strong background in data architecture , including data modeling, warehousing, real-time and batch processing, and big data frameworks. Proficiency with modern data tools and technologies such as Spark, Databricks, Kafka, or Snowflake (bonus). Knowledge of cloud security, networking, and cost More ❯

on Azure (e.g., Azure Data Lake, Synapse, Data Factory) and AWS (e.g., Redshift, Glue, S3, Lambda) . Strong background in data architecture , including data modeling, warehousing, real-time and batch processing, and big data frameworks. Proficiency with modern data tools and technologies such as Spark, Databricks, Kafka, or Snowflake (bonus). Knowledge of cloud security, networking, and cost More ❯

on Azure (e.g., Azure Data Lake, Synapse, Data Factory) and AWS (e.g., Redshift, Glue, S3, Lambda) . Strong background in data architecture , including data modeling, warehousing, real-time and batch processing, and big data frameworks. Proficiency with modern data tools and technologies such as Spark, Databricks, Kafka, or Snowflake (bonus). Knowledge of cloud security, networking, and cost More ❯

on Azure (e.g., Azure Data Lake, Synapse, Data Factory) and AWS (e.g., Redshift, Glue, S3, Lambda) . Strong background in data architecture , including data modeling, warehousing, real-time and batch processing, and big data frameworks. Proficiency with modern data tools and technologies such as Spark, Databricks, Kafka, or Snowflake (bonus). Knowledge of cloud security, networking, and cost More ❯

on Azure (e.g., Azure Data Lake, Synapse, Data Factory) and AWS (e.g., Redshift, Glue, S3, Lambda) . Strong background in data architecture , including data modeling, warehousing, real-time and batch processing, and big data frameworks. Proficiency with modern data tools and technologies such as Spark, Databricks, Kafka, or Snowflake (bonus). Knowledge of cloud security, networking, and cost More ❯

on Azure (e.g., Azure Data Lake, Synapse, Data Factory) and AWS (e.g., Redshift, Glue, S3, Lambda) . Strong background in data architecture , including data modeling, warehousing, real-time and batch processing, and big data frameworks. Proficiency with modern data tools and technologies such as Spark, Databricks, Kafka, or Snowflake (bonus). Knowledge of cloud security, networking, and cost More ❯

Conduct descriptive and exploratory analysis to uncover patterns, trends, and potential applications across a wide range of datasets. Maintain robust and scalable data infra to support real-time and batch processing workflows. Provide domain expertise to help shape financial knowledge representation in LLM outputs. Develop and apply custom quality metrics to assess and guide model performance. Actively participate … Computer Science, Finance, Economics, Statistics, Engineering). 3+ years of professional experience in Data Science or Quantitative roles within finance. Proficiency in Python and SQL with exp. in data processing libraries such as pandas/numpy. Experience with machine learning frameworks and statistical analysis tools. Familiarity with financial data sources, APIs, and data modelling techniques. Ability to design, implement More ❯

to join our team and help us achieve these goals. Key Responsibilities: Architect and design reference architectures and applications for various cloud hosting patterns (e.g., message-driven, web applications, batch processing). Mentor and guide other engineers, fostering continuous learning and improvement. Develop flexible and customizable solutions for advanced users. Enable rapid provisioning of cloud infrastructure (within More ❯

modelling, data access patterns, and efficient use of databases (MSSQL, PostgreSQL, MySQL). Work with data engineers to enable secure, efficient data movement between systems, including real-time and batch processing pipelines. Support analytics and reporting needs through the engineering of clean, reliable data sources and APIs. Security Engineering & Governance Partner with security engineers to integrate security throughout More ❯

modelling, data access patterns, and efficient use of databases (MSSQL, PostgreSQL, MySQL). Work with data engineers to enable secure, efficient data movement between systems, including real-time and batch processing pipelines. Support analytics and reporting needs through the engineering of clean, reliable data sources and APIs. Security Engineering & Governance Partner with security engineers to integrate security throughout More ❯

modelling, data access patterns, and efficient use of databases (MSSQL, PostgreSQL, MySQL). Work with data engineers to enable secure, efficient data movement between systems, including real-time and batch processing pipelines. Support analytics and reporting needs through the engineering of clean, reliable data sources and APIs. Security Engineering & Governance Partner with security engineers to integrate security throughout More ❯

modelling, data access patterns, and efficient use of databases (MSSQL, PostgreSQL, MySQL). Work with data engineers to enable secure, efficient data movement between systems, including real-time and batch processing pipelines. Support analytics and reporting needs through the engineering of clean, reliable data sources and APIs. Security Engineering & Governance Partner with security engineers to integrate security throughout More ❯

modelling, data access patterns, and efficient use of databases (MSSQL, PostgreSQL, MySQL). Work with data engineers to enable secure, efficient data movement between systems, including real-time and batch processing pipelines. Support analytics and reporting needs through the engineering of clean, reliable data sources and APIs. Security Engineering & Governance Partner with security engineers to integrate security throughout More ❯