76 to 100 of 415 Apache Spark Jobs in England

able to guide how one could deploy infrastructure into different environments. Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop. Good understanding of possible architectures involved in modern data system design (Data Warehouse, Data Lakes, Data Meshes) Ability to create ...

Azure) Experience with: o Data warehousing and lake architectures o ETL/ELT pipeline development o SQL and NoSQL databases o Distributed computing frameworks (Spark, Kinesis etc) o Software development best practices including CI/CD, TDD and version control. o Containerisation tools like Docker or Kubernetes o Experience ...

working with or within the public sector Familiarity using AI tools to enhance the quality & pace of your output Technologies we love Languages: Python, Spark, SQL Tools: AWS, Azure, Git, Rest APIs, Kubernetes, Docker, Terraform, Datadog Datastores: Databricks, OpenSearch, Postgres Equal Opportunity Statement At Altana, we believe that ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache Spark and knowledge of Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps, ML/ ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache Spark and knowledge of Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps, ML/ ...

Security In-depth knowledge of the financial services industry and their IT systems Practical cloud native experience Preferred qualifications, capabilities, and skills Advanced Apache Spark experience(PySpark/Spark SQL), including performance tuning (partitioning, shuffle optimization, caching), troubleshooting, and designing scalable batch/stream processing patterns. Deep ...

Management Stakeholder Collaboration Leadership & Mentorship Technology Stack The successful candidate will have experience working with some or all of the following technologies: Data Processing: Apache Spark/PySpark Workflow Orchestration: Apache Airflow Containerisation: Kubernetes Cloud Platform: Microsoft Azure Data APIs & Semantic Layer: CubeJS Programming Languages: Python ...

cleansed, conformed business layers) while securely handling identifiable data. Scalable Data Processing: Develop, tune, and monitor large-scale batch processing pipelines using Azure Databricks (Spark-based processing) and PySpark. Orchestration & Workflow: Build automated data workflows using Microsoft Fabric Data Factory and Azure Data Factory to move data efficiently across … Microsoft Azure, with practical exposure to the Microsoft Fabric ecosystem (OneLake, Data Factory, Fabric Lakehouse). Big Data Processing: Deep technical expertise with Apache Spark/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL ...

solutions. Pipeline Development : Develop and configure meta-data driven data pipelines using data orchestration tools such as Azure Data factory and engineering tools like Apache Spark to ensure seamless data flow. Monitoring and Failure Recovery: Implement monitoring procedures to detect failures or unusual data profiles and establish recovery … using SQL to query and manipulate data. Proficiency in cloud data tooling such as Synapse Analytics, Microsoft Fabric, Azure Data Factory or AWS equivalents. Spark/Pyspark or Python skills. Excellent problem-solving skills and attention to detail. Strong communication and collaboration skills. MS Fabric interest, skills or experience ...

batch and streaming data pipelines using Kafka, Flink, and ETL technologies Design and build unified analytics engine designed for processing large-scale data using Apache Spark and related tools Develop and optimize analytical data models for time-series, financial metrics, and trading activity Implement and manage analytical databases ...

Lake Storage. Advanced SQL development and performance tuning skills, including experience with Azure SQL Database and SQL Server. Proficiency in Python, PySpark, Scala and Spark for data processing and distributed computing. Understanding of data warehousing concepts, dimensional modelling and ETL/ELT frameworks. Knowledge of data quality, data governance ...

with data warehousing concepts and technologies (e.g., Azure Synapse Analytics, Azure Data Lake Storage). Familiarity with big data technologies and frameworks (e.g., Hadoop, Spark) is a plus. Knowledge of data governance and data security best practices. Strong problem-solving skills and attention to detail. Excellent communication and collaboration ...

large-scale ETL development. Proficiency in SQL and programming languages such as Python, Java, or Scala. Hands‐on experience with big data frameworks (e.g., Spark, Hadoop, Flink) and cloud platforms (AWS, GCP, or Azure). Familiarity with data warehouse technologies (e.g., Snowflake, Redshift, BigQuery) and workflow orchestration tools (e.g. ...

maintain SC clearance. AWS certification at Associate or Professional level Experience with streaming and event‐driven data technologies Experience with technologies such as Spark, Kafka or Databricks Experience delivering data platforms within public sector, defence or other regulated environments Experience working within secure environments and understanding associated security ...

/CD platform (Github Actions and Admin), Password/Secret Management (hashicorp vault).Strong Data related programming skills SQL/Python/Spark/Scala.Database technologies in relation to Data Warehousing/Data Lake/Lake housing patterns and relevant experience when handling structured and non-structured data.(Information ...

maintain SC clearance. Desirable AWS certification at Associate or Professional level Experience with streaming and event‐driven data technologies Experience with technologies such as Spark, Kafka or Databricks Experience delivering data platforms within public sector, defence or other regulated environments Experience working within secure environments and understanding associated security ...

maintain SC clearance. Desirable AWS certification at Associate or Professional level Experience with streaming and event‐driven data technologies Experience with technologies such as Spark, Kafka or Databricks Experience delivering data platforms within public sector, defence or other regulated environments Experience working within secure environments and understanding associated security ...

architecture (SQL and NoSQL) Experience with cloud technologies Experience with cloud platforms Experience with CI/CD practices Experience with big data tools e.g. Spark, Hadoop, NiFi Experience managing direct reports, providing regular guidance and performance feedback Ability to create delivery plans with estimated timelines for personal and team ...

automation & configuration management; • Ansible (plus Puppet, Saltstack), Terraform, CloudFormation; • NodeJS, REACT/MaterialUI (plus Angular), Python, JavaScript; • Big data processing and analysis, e.g. Apache Hadoop (CDH), Apache Spark; • RedHat Enterprise Linux, CentOS, Debian or Ubuntu. • Java 8, Spring framework (preferably Spring boot), AMQP - RabbitMQ, • Open source technologies ...

similar data‐focused role Strong SQL and Python development skills Experience building and supporting scalable data pipelines and transformation processes Hands‐on experience with Apache Spark, Databricks, dbt and cloud‐based data platforms Good understanding of data modelling principles, data lakes and data warehouse architectures Experience working with … Essential Degree in Computer Science, Software Engineering, Data Engineering, Information Systems, Mathematics or a related discipline; or equivalent practical experience Desirable Relevant certifications in Spark, cloud technologies, data engineering or software engineering practices Why should you apply? We’re not a perfect place – but we’re a people place. ...

basis to support a MoD-owned data platform. The role focuses on building and maintaining secure data pipelines using Python, Azure and Kubernetes, with Apache Airflow and Apache Spark integrated into the stack. You will work within a government-aligned, secure environment, ensuring data remains hosted ...

root-cause analysis. Experience building or supporting production data pipelines in a collaborative engineering environment. Experience working with distributed data processing frameworks such as Apache Spark. Working knowledge of common data formats such as JSON, Avro and Parquet. For More Experienced Candidates Stronger ownership of technical design across multiple … should bring relevant experience and the ability to work across comparable technologies. Examples Of Technologies In Scope Include Data processing and logic: ANSI SQL, Apache Spark, Kafka Data formats: JSON, Avro, Parquet Platforms and storage: Snowflake, Apache Iceberg, Databricks, Hadoop ecosystem technologies, Sybase IQ Engineering and deployment ...

Company Description Company DescriptionPublicis Media harnesses the power of modern media through leading agency brands Dysrupt, Infinite Roar, Publicis Collective, Publicis Health Media, Spark Foundry, Starcom and Zenith , as well as global accelerator PMX ; immersive experience group PMCI and access to integrated platform-based technologies and offerings from … marketing platforms. Build and optimize schemas, transformations, and ETL logic for high-performance, reusable data workflows. Contribute directly to the DFP codebase (Python, SQL, Spark) — developing pipelines and infrastructure alongside the engineering team. Establish data versioning, testing, and governance frameworks ensuring reliability, lineage, and compliance. Data Engineering Leadership Drive ...

role 7Dxperts is looking for a Senior Data Engineer to join our UK team. Key Responsibilities Build scalable pipelines using Databricks, Spark and PySpark. Develop ETL/ELT and data transformation workflows using Python and SQL. Create and maintain Databricks notebooks, jobs, workflows and clusters. Work with Delta Lake ...

Senior Data Engineer: Passion for building scalable, resilient cloud data platforms Strong experience with Databricks or Snowflake on AWS Proven Python skills, including Spark and Airflow expertise Advanced SQL skills and end-to-end data modelling experience Experience building batch and real-time data integrations Hands-on CICD skills ...