101 to 125 of 553 Apache Spark Jobs in the UK

Strong experience in IaC and guidance on deploying infrastructure into different environments. Knowledge of handling and transforming various data types (JSON, CSV, etc.) with Apache Spark, Databricks or Hadoop. Good understanding of modern data system design architectures (Data Warehouse, Data Lakes, Data Meshes). Ability to create data ...

Lake Storage. Advanced SQL development and performance tuning skills, including experience with Azure SQL Database and SQL Server. Proficiency in Python, PySpark, Scala and Spark for data processing and distributed computing. Understanding of data warehousing concepts, dimensional modelling and ETL/ELT frameworks. Knowledge of data quality, data governance ...

with data warehousing concepts and technologies (e.g., Azure Synapse Analytics, Azure Data Lake Storage). Familiarity with big data technologies and frameworks (e.g., Hadoop, Spark) is a plus. Knowledge of data governance and data security best practices. Strong problem-solving skills and attention to detail. Excellent communication and collaboration ...

client engagements. Desirable • AWS certification at Associate or Professional level • Experience with streaming and event-driven data technologies • Experience with technologies such as Spark, Kafka or Databricks • Experience delivering data platforms within public sector, defence or other regulated environments • Experience working within secure environments and understanding associated security ...

large-scale ETL development. Proficiency in SQL and programming languages such as Python, Java, or Scala. Hands‐on experience with big data frameworks (e.g., Spark, Hadoop, Flink) and cloud platforms (AWS, GCP, or Azure). Familiarity with data warehouse technologies (e.g., Snowflake, Redshift, BigQuery) and workflow orchestration tools (e.g. ...

maintain SC clearance. AWS certification at Associate or Professional level Experience with streaming and event‐driven data technologies Experience with technologies such as Spark, Kafka or Databricks Experience delivering data platforms within public sector, defence or other regulated environments Experience working within secure environments and understanding associated security ...

messaging architectures. Data Platforms & Cloud Technologies Hands-on experience with: Azure Data Lake Databricks Snowflake Delta Lake Azure Synapse Analytics/Microsoft Fabric Spark/PySpark Strong understanding of: Data Lake and Lakehouse architecture Hybrid cloud environments Enterprise integration patterns Modern data formats including Parquet, Avro, and ORC DevOps ...

maintain SC clearance. Desirable AWS certification at Associate or Professional level Experience with streaming and event‐driven data technologies Experience with technologies such as Spark, Kafka or Databricks Experience delivering data platforms within public sector, defence or other regulated environments Experience working within secure environments and understanding associated security ...

maintain SC clearance. Desirable AWS certification at Associate or Professional level Experience with streaming and event‐driven data technologies Experience with technologies such as Spark, Kafka or Databricks Experience delivering data platforms within public sector, defence or other regulated environments Experience working within secure environments and understanding associated security ...

following: Python, SQL, JavaCommercial experience in client-facing projects is a plus, especially within multi-disciplinary teamsDeep knowledge of database technologies: Distributed systems (e.g., Spark, Hadoop, EMR)RDBMS (e.g., SQL Server, Oracle, PostgreSQL, MySQL)NoSQL (e.g., MongoDB, Cassandra, DynamoDB, Neo4j)Solid understanding of software engineering best practices – code reviews ...

Nice to Have Hands‐on experience with cloud platforms such as Azure, AWS or GCP. Exposure to modern data platforms such as Snowflake, Databricks, Spark or similar. Experience with MLOps practices such as model deployment, monitoring, versioning or model registries. Exposure to workflow orchestration or deployment tools such ...

architecture (SQL and NoSQL) Experience with cloud technologies Experience with cloud platforms Experience with CI/CD practices Experience with big data tools e.g. Spark, Hadoop, NiFi Experience managing direct reports, providing regular guidance and performance feedback Ability to create delivery plans with estimated timelines for personal and team ...

architecture (SQL and NoSQL) Experience with cloud technologies Experience with cloud platforms Experience with CI/CD practices Experience with big data tools e.g. Spark, Hadoop, NiFi Experience managing direct reports, providing regular guidance and performance feedback Ability to create delivery plans with estimated timelines for personal and team ...

architecture (SQL and NoSQL) Experience with cloud technologies Experience with cloud platforms Experience with CI/CD practices Experience with big data tools e.g. Spark, Hadoop, NiFi Experience managing direct reports, providing regular guidance and performance feedback Ability to create delivery plans with estimated timelines for personal and team ...

similar data‐focused role Strong SQL and Python development skills Experience building and supporting scalable data pipelines and transformation processes Hands‐on experience with Apache Spark, Databricks, dbt and cloud‐based data platforms Good understanding of data modelling principles, data lakes and data warehouse architectures Experience working with … Essential Degree in Computer Science, Software Engineering, Data Engineering, Information Systems, Mathematics or a related discipline; or equivalent practical experience Desirable Relevant certifications in Spark, cloud technologies, data engineering or software engineering practices Why should you apply? We’re not a perfect place – but we’re a people place. ...

basis to support a MoD-owned data platform. The role focuses on building and maintaining secure data pipelines using Python, Azure and Kubernetes, with Apache Airflow and Apache Spark integrated into the stack. You will work within a government-aligned, secure environment, ensuring data remains hosted ...

promote their adoption. Drive continuous improvement in data infrastructure and tooling, leveraging cloud platforms (e.g., AWS, GCP, Azure) and big data technologies (e.g., Spark,dbt).Collaborate with PMs to manage project timelines, resource allocation, and deliverables to meet business goals. Work with data governance and compliance teams to ensure … data pipeline development, ETL/ELT processes, and data warehousing solutions. Proficient with cloud platforms (AWS, GCP, Azure) and big data tools like Apache Spark, Python, etc. Experience with relational and NoSQL databases. Solid understanding of data modeling, data architecture, engineering frameworksProven ability to lead and scale engineering ...

Company Description Company DescriptionPublicis Media harnesses the power of modern media through leading agency brands Dysrupt, Infinite Roar, Publicis Collective, Publicis Health Media, Spark Foundry, Starcom and Zenith , as well as global accelerator PMX ; immersive experience group PMCI and access to integrated platform-based technologies and offerings from … marketing platforms. Build and optimize schemas, transformations, and ETL logic for high-performance, reusable data workflows. Contribute directly to the DFP codebase (Python, SQL, Spark) — developing pipelines and infrastructure alongside the engineering team. Establish data versioning, testing, and governance frameworks ensuring reliability, lineage, and compliance. Data Engineering Leadership Drive ...

Skills MUST HAVE: Strong hands-on experience with Quantexa implementation and development. Proven experience building and supporting big data platforms . Advanced knowledge of Apache Spark/PySpark . Strong programming skills in Scala and/or Python . Expertise in SQL and large-scale data modelling. Strong ...

Skills MUST HAVE: Strong hands‐on experience with Quantexa implementation and development. Proven experience building and supporting big data platforms . Advanced knowledge of Apache Spark/PySpark . Strong programming skills in Scala and/or Python . Expertise in SQL and large-scale data modelling. Strong ...

understand where traditional patterns remain appropriate – and where modern approaches add measurable value. Data Engineering & Platform Understanding: Strong knowledge of distributed processing (e.g. Apache Spark).Familiarity with Hadoop ecosystem components. Working knowledge of SQL and NoSQL technologies. Understanding of ETL/ELT design trade-offs. Awareness of DevOps ...

role 7Dxperts is looking for a Senior Data Engineer to join our UK team. Key Responsibilities Build scalable pipelines using Databricks, Spark and PySpark. Develop ETL/ELT and data transformation workflows using Python and SQL. Create and maintain Databricks notebooks, jobs, workflows and clusters. Work with Delta Lake ...

What we’re looking for: Strong Python development and software engineering experience Experience building scalable data pipelines and distributed data platforms Knowledge of Kafka, Spark and modern data processing technologies Experience with orchestration tools such as Airflow, Dagster or Prefect Hands-on experience with Docker, Kubernetes and containerised deployments ...

Warehousing concepts. Experience of Enterprise ETL tools such as Informatica, Talend, Datastage or Alteryx. Project experience using the any of the following technologies: Hadoop, Spark, Scala, Oracle, Pega, Salesforce. Cross and multi-platform experience. Team building and leading. You must be: Willing to work on client sites, potentially ...

Senior Data Engineer: Passion for building scalable, resilient cloud data platforms Strong experience with Databricks or Snowflake on AWS Proven Python skills, including Spark and Airflow expertise Advanced SQL skills and end-to-end data modelling experience Experience building batch and real-time data integrations Hands-on CICD skills ...