12 of 12 Apache Spark Jobs in the South East

Design, build, and maintain cloud-based infrastructure using AWS Develop and manage Infrastructure as Code using Terraform Support and optimise data processing pipelines using Apache Spark and Airflow Collaborate with cross-functional teams to deliver secure and scalable solutions Implement CI/CD pipelines using GitLab Containerise applications … Notebooks for data exploration and development Key Skills & Experience Strong experience with AWS services Expertise in Terraform (Infrastructure as Code) Hands-on experience with: Apache Spark Python Apache Airflow Docker GitLab CI/CD Experience with security scanning tools such as: Trivy Trend Micro Wiz (or similar ...

Warehouses. Design and oversee ingestion frameworks for enterprise data sources, including: Microsoft SQL Server (on-premises and Azure SQL) Oracle Databases Develop and optimize Apache Spark workloads using PySpark or Scala, ensuring performance, scalability, and operational efficiency. Assess existing or partially completed data platform implementations and provide remediation … integration with Databricks SQL Warehouses Experience integrating enterprise data sources such as: Microsoft SQL Server (on-premises and Azure SQL) Oracle Databases Strong Apache Spark expertise using PySpark and/or Scala, including performance tuning, partitioning strategies, and workload orchestration. Experience evaluating and remediating incomplete, underperforming, or legacy ...

Senior Data Engineer: Passion for building scalable, resilient cloud data platforms Strong experience with Databricks or Snowflake on AWS Proven Python skills, including Spark and Airflow expertise Advanced SQL skills and end-to-end data modelling experience Experience building batch and real-time data integrations Hands-on CICD skills ...

with Airflow, Dagster or other orchestration tools Experience with data lakes (Synapse, Snowflake) Experience with Docker Experience with streaming technologies (Kafka, RabbitMQ) Experience with Spark or other big-data tools Hybrid working – 3 days per week in a central London office (Tues - Thurs) If you’re excited ...

capacity planning, skills development, and vendor contributions The individual: Expertise in Azure data services (ADLS, ADF, Databricks, Synapse, Event Hub/Kafka), Python, SQL, Spark/PySpark. Experience with data modeling, metadata and lineage tooling; performance & cost optimization. Proven leadership in engineering teams; Agile/DevOps ways of working ...

through influence with strong collaborator engagement skills. Preferred Experience Digital Twin, asset virtualization, or industrial data platforms Graph technologies (e.g., Neo4j), big data ecosystems (Spark, Hive, Hadoop) Databricks, Python, and modern analytics platforms Delivery of data products at enterprise scale Key Competencies Strategic product thinking with strong execution bias ...

Senior Data Engineer (Scala/Spark) London | Hybrid Working | Initial 12 month contract | £450-550pd PAYE + holiday pay We are partnering with a world-leading technology organisation seeking an experienced Senior Data Engineer to join a high-performing engineering team responsible for building and operating large-scale … machine learning workflows at scale. As a Senior Data Engineer, you will design, build and operate large-scale batch data pipelines using Scala and Apache Spark. Working alongside Machine Learning Engineers, Researchers and Platform Engineers, you will help develop the infrastructure and tooling required to process vast datasets ...

internal knowledge Competencies and Behaviours: 3+ years experience in data platform engineering, cloud engineering, or similar roles Strong hands-on experience with Databricks, including Apache Spark, Delta Lake, Workflows Proven experience designing and deploying data platforms on AWS and/or Azure (e.g. ADLS, S3, VNet/ ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation Knowledge of statistical methods and experimentation (A/B testing) preferred ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation Knowledge of statistical methods and experimentation (A/B testing) preferred ...

Limited (DDIL) communications environments, supporting local processing at the edge Building streaming ingestion, event pipelines, and batch/stream processing workloads using technologies like Apache Flink and/or Apache Spark Implementing metadata, lineage, and catalogue capabilities along with policy-based security and governance Job Requirements: Extensive ...

seasonality analysis) Solid foundation in statistical modelling and machine learning theory Proficiency in Python and experience working with large, complex datasets using SQL and Spark Ability to manage multiple priorities across a portfolio of projects Experience engaging both technical and business stakeholders Experience with libraries such as SciPy, Scikit ...