13 of 13 Contract Apache Spark Jobs in London

Design, build, and maintain cloud-based infrastructure using AWS Develop and manage Infrastructure as Code using Terraform Support and optimise data processing pipelines using Apache Spark and Airflow Collaborate with cross-functional teams to deliver secure and scalable solutions Implement CI/CD pipelines using GitLab Containerise applications … Notebooks for data exploration and development Key Skills & Experience Strong experience with AWS services Expertise in Terraform (Infrastructure as Code) Hands-on experience with: Apache Spark Python Apache Airflow Docker GitLab CI/CD Experience with security scanning tools such as: Trivy Trend Micro Wiz (or similar ...

Warehouses. Design and oversee ingestion frameworks for enterprise data sources, including: Microsoft SQL Server (on-premises and Azure SQL) Oracle Databases Develop and optimize Apache Spark workloads using PySpark or Scala, ensuring performance, scalability, and operational efficiency. Assess existing or partially completed data platform implementations and provide remediation … integration with Databricks SQL Warehouses Experience integrating enterprise data sources such as: Microsoft SQL Server (on-premises and Azure SQL) Oracle Databases Strong Apache Spark expertise using PySpark and/or Scala, including performance tuning, partitioning strategies, and workload orchestration. Experience evaluating and remediating incomplete, underperforming, or legacy ...

understand where traditional patterns remain appropriate - and where modern approaches add measurable value. Data Engineering & Platform Understanding: Strong knowledge of distributed processing (e.g. Apache Spark). Familiarity with Hadoop ecosystem components. Working knowledge of SQL and NoSQL technologies. Understanding of ETL/ELT design trade-offs. Awareness ...

skills and understanding of modern data engineering principles. Desirable Skills Python and PySpark development. AWS services including EMR, S3, Lambda, Athena, and Redshift. Apache Spark and large-scale batch processing. Terraform, CI/CD pipelines, and Infrastructure as Code. Git/GitLab and modern software development practices. ...

experience with SQL and data modelling concepts Strong analytical and problem-solving skills Nice to Have Skills Experience with data pipeline tools (Airflow, Kafka, Spark) Knowledge of cloud platforms (AWS, Azure or GCP) Familiarity with relational and NoSQL databases Additional programming experience (Python, Scala) Essential Requirements Active SC Clearance ...

Senior Data Engineer (Scala/Spark) London | Hybrid Working | Initial 12 month contract | £450-550pd PAYE + holiday pay We are partnering with a world-leading technology organisation seeking an experienced Senior Data Engineer to join a high-performing engineering team responsible for building and operating large-scale … machine learning workflows at scale. As a Senior Data Engineer, you will design, build and operate large-scale batch data pipelines using Scala and Apache Spark. Working alongside Machine Learning Engineers, Researchers and Platform Engineers, you will help develop the infrastructure and tooling required to process vast datasets ...

Prometheus, ElasticSearch , Grafana, OpenTelemetry ) for metrics, logs and traces Platform Technologies & BSS Vendor Platforms - Open source workflow engines (Temporal) - Stream/batch processing (Flink, Spark) - Test automation: Robot Framework and BDD - can review BDD robot files - BSS vendor platforms: Salesforce, Amdocs, Ciena - BSS/OSS proficiency: awareness of Product ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation Knowledge of statistical methods and experimentation (A/B testing) preferred ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation Knowledge of statistical methods and experimentation (A/B testing) preferred ...

seasonality analysis) Solid foundation in statistical modelling and machine learning theory Proficiency in Python and experience working with large, complex datasets using SQL and Spark Ability to manage multiple priorities across a portfolio of projects Experience engaging both technical and business stakeholders Experience with libraries such as SciPy, Scikit ...

work on a Big Data Project in the finance sector. Required Experience: Java Big Data experience On-premises cluster experience Databases Kubernetes Kafka Apache Spark Preffered Experience: Finance sector experience ...

work on a Big Data Project in the finance sector. Required Experience: Java Big Data experience On-premises cluster experience Databases Kubernetes Kafka Apache Spark Preffered Experience: Finance sector experience ...

Analytics Engineer London, hybrid 3 times a week Department: IT Type: 3-month rolling contract Outside IR35 Position: Analytics Engineer The organisation is seeking an experienced Analytics Engineer to join its Data & Analytics team. The ...