51 to 75 of 367 Apache Spark Jobs in England

strategy. What We're Looking For Essential 5+ years' experience in Data Engineering or a similar role. Strong experience with Python, SQL, Apache Spark, and Cloud platforms (AWS, Azure or GCP) Experience designing and operating production-scale data platforms. Desirable Experience working with AWS. Experience supporting machine learning ...

strategy. What We're Looking For Essential 5+ years' experience in Data Engineering or a similar role. Strong experience with Python, SQL, Apache Spark, and Cloud platforms (AWS, Azure or GCP) Experience designing and operating production-scale data platforms. Desirable Experience working with AWS. Experience supporting machine learning ...

design approaches.You understand where traditional patterns remain appropriate – and where modern approaches add measurable value.Data Engineering & Platform Understanding:Strong knowledge of distributed processing (e.g. Apache Spark).Familiarity with Hadoop ecosystem components.Working knowledge of SQL and NoSQL technologies.Understanding of ETL/ELT design trade-offs.Awareness of DevOps practices ...

engineering best practices Skills & Experience * 3+ years' commercial experience in Data Engineering * Strong Java or Python development skills * Experience with Kafka, Flink or Spark * Experience building ETL/ELT or streaming data pipelines * Knowledge of AWS (Azure or GCP also considered) * Experience with Docker, Kubernetes and CI/ ...

Have: Certifications: Google Cloud or AWS Certified Solutions Architect certification. ETL and Data Pipeline Expertise: Experience with ETL tools and Hadoop-based technologies (e.g., Spark) and/or batch/streaming data pipelines such as Beam and Flink. Data Architecture: Proven ability in designing and building data lake ...

Have:Certifications: Google Cloud or AWS Certified Solutions Architect certification.ETL and Data Pipeline Expertise: Experience with ETL tools and Hadoop-based technologies (e.g., Spark) and/or batch/streaming data pipelines such as Beam and Flink.Data Architecture: Proven ability in designing and building data lake and data warehouse ...

e.g., AWS, Azure, GCP) and their AI/ML services.Data Engineering: Proficiency in data manipulation and analysis using SQL and big data technologies (e.g., Spark, Hadoop). Experience with ETL, data pipelines, and automated workflows for moving and processing data.Data Governance: Adherence to data governance, security, and privacy standards ...

Python, or similar languages relevant to data manipulation and scripting.Proficiency in data manipulation and analysis using SQL and big data technologies (e.g., Spark, Hadoop). Experience with ETL/ELT, data warehousing, and automated workflows for moving and processing data.Familiarity with data modelling techniques (e.g., dimensional, relational) and data ...

reconciliation and root-cause analysis.Experience building or supporting production data pipelines in a collaborative engineering environment.Experience working with distributed data processing frameworks such as Apache Spark.Working knowledge of common data formats such as JSON, Avro and Parquet.Technology EnvironmentThe role will involve working with a modern and evolving data stack. … should bring relevant experience and the ability to work across comparable technologies.Examples of technologies in scope include:Data processing and logic: ANSI SQL, Apache Spark, KafkaData formats: JSON, Avro, ParquetPlatforms and storage: Snowflake, Apache Iceberg, Databricks, Hadoop ecosystem technologies, Sybase IQEngineering and deployment: CI/CD tooling ...

engineering and promote their adoption.Drive continuous improvement in data infrastructure and tooling, leveraging cloud platforms (e.g., AWS, GCP, Azure) and big data technologies (e.g., Spark,dbt).Collaborate with PMs to manage project timelines, resource allocation, and deliverables to meet business goals.Work with data governance and compliance teams to ensure … expertise in data pipeline development, ETL/ELT processes, and data warehousing solutions.Proficient with cloud platforms (AWS, GCP, Azure) and big data tools like Apache Spark, Python, etc.Experience with relational and NoSQL databases.Solid understanding of data modeling, data architecture, engineering frameworksProven ability to lead and scale engineering teams ...

Company descriptionCompany DescriptionPublicis Media harnesses the power of modern media through leading agency brands Dysrupt, Infinite Roar, Publicis Collective, Publicis Health Media, Spark Foundry, Starcom and Zenith, as well as global accelerator PMX; immersive experience group PMCI and access to integrated platform-based technologies and offerings from CJ, Epsilon … across 20+ marketing platforms.Build and optimize schemas, transformations, and ETL logic for high-performance, reusable data workflows.Contribute directly to the DFP codebase (Python, SQL, Spark) — developing pipelines and infrastructure alongside the engineering team.Establish data versioning, testing, and governance frameworks ensuring reliability, lineage, and compliance.Data Engineering LeadershipDrive best practices ...

self-service data capabilities. Understanding of emerging technologies including AI, machine learning, real-time analytics, and data lakes. Desirable Experience Experience with Hadoop, Spark, Kafka, or similar big data technologies. Exposure to advanced analytics and machine learning solutions. Relevant certifications such as CDMP, Azure Data Engineer, or AWS Data ...

Learning (ML):• Deep understanding of machine learning principles, algorithms, and techniques.• Experience with popular ML frameworks and libraries like TensorFlow, PyTorch, scikit-learn, or Apache Spark.• Proficiency in data preprocessing, feature engineering, and model evaluation.• Knowledge of ML model deployment and serving strategies, including containerization and microservices.• Familiarity with ...

large request volumes, large data volumes, or both.Deep expertise in distributed data systems: streaming (Kafka or similar), batch and ELT/ETL frameworks (Spark, Flink, dbt, Airflow or Argo Workflows), and modern lakehouse or warehouse technologies.Strong production Python experience and awareness of other relevant languages (Java/Kotlin) sufficient ...

Risk, XVA, CCR, FRTB, portfolio risk, fixed income analytics, securitised products, liquidity risk, or managed risk-as-a-service solutions. Familiarity with Snowflake, Databricks, Spark, AWS, Azure, Confluence, Azure DevOps, Git, Tableau, Power BI, message queues, or comparable tools and platforms. Knowledge of market data vendors, data mastering, golden ...

Market Risk, XVA, CCR, FRTB, portfolio risk, fixed income analytics, securitised products, liquidity risk, or managed risk-as-a-service solutions.Familiarity with Snowflake, Databricks, Spark, AWS, Azure, Confluence, Azure DevOps, Git, Tableau, Power BI, message queues, or comparable tools and platforms.Knowledge of market data vendors, data mastering, golden-source ...

volume, data-intensive environments. Extensive AWS experience including services such as Glue, Lambda, S3, Redshift, and wider cloud-native data tooling. Strong knowledge of Apache Spark and large-scale data processing. Experience working with streaming technologies such as Kafka. Expert SQL and data modelling capabilities. Experience designing modern ...

testable code.Modern Data & Cloud Practices: Experience with modern data engineering tooling and cloud platforms (ideally AWS and Databricks), and familiarity with distributed processing (e.g. Apache Spark). Exposure to infrastructure as code (such as Terraform and Databricks Asset Bundles) is a plus.Engineering Mindset: Understanding of testing, version control ...

What we’re looking for: Strong Python development and software engineering experience Experience building scalable data pipelines and distributed data platforms Knowledge of Kafka, Spark and modern data processing technologies Experience with orchestration tools such as Airflow, Dagster or Prefect Hands-on experience with Docker, Kubernetes and containerised deployments ...

improve overall operational stability of software applications and systems Designs and implements data pipelines, analytics workflows, and reporting solutions using Python, Databricks, and Spark Develops and maintains web-based applications and dashboards using modern web stacks (React, Angular, etc.) Builds and optimises data models, ETL/ELT processes ...

monitoring. Strong background in CI/CD pipeline design, automation, and maintenance. Hands-on experience with data engineering tools and storage solutions (e.g., ADLS, Spark, Databricks, SQL/NoSQL databases). Experience with distributed computing and big data processing frameworks such as PySpark. Knowledge of infrastructure-as-code tools ...

Experience with data warehousing concepts and technologies.Knowledge of other BI tools (e.g., Tableau, Looker) and data modelling techniques.Familiarity with big data technologies (e.g., Hadoop, Spark).Familiarity with modern data architecture (e.g., Data Mesh, Fabric).An understanding of infrastructure as code (e.g., Terraform)Experience with CI/CD pipelines ...

Advanced experience operating Delta tables at scale, covering transaction semantics, schema evolution, partitioning, and compaction, alongside highly optimised analytical query execution using DuckDB or Apache Spark.• Production-grade relational database expertise, particularly PostgreSQL (query tuning, indexing, partitioning, replication, and schema design across OLTP and analytics).• Strong KDB experience ...

GCP. · MSc or equivalent experience in Data Science, Statistics, Mathematics, Computer Science, Artificial Intelligence, Physics, Engineering or a related quantitative discipline. · Experience with Spark or distributed data processing frameworks. · Experience preparing data pipelines or analytical datasets to support machine learning workloads. · Awareness of data engineering, data architecture, data governance ...

data architectures supporting AI/ML, analytics, and real-time data products.Deep hands-on experience with modern data platforms (e.g., Snowflake, Databricks, Spark, Kafka, Elasticsearch, graph databases, AWS/Azure/GCP).Experience designing data quality and assurance frameworks: data lineage, data contracts, reconciliation, and quality measurement at scale.Background ...