76 to 100 of 479 Apache Spark Jobs in the UK

able to guide how one could deploy infrastructure into different environments. Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop. Good understanding of possible architectures involved in modern data system design (Data Warehouse, Data Lakes, Data Meshes) Ability to create ...

engineering teams. Strong Python and SQL skills, with experience working with large-scale datasets and distributed data processing environments. Experience with technologies such as Apache Spark and modern data engineering frameworks. Experience with cloud platforms and data services, ideally AWS, including technologies such as Redshift and S3. Experience ...

Azure) Experience with: o Data warehousing and lake architectures o ETL/ELT pipeline development o SQL and NoSQL databases o Distributed computing frameworks (Spark, Kinesis etc) o Software development best practices including CI/CD, TDD and version control. o Containerisation tools like Docker or Kubernetes o Experience ...

working with or within the public sector Familiarity using AI tools to enhance the quality & pace of your output Technologies we love Languages: Python, Spark, SQL Tools: AWS, Azure, Git, Rest APIs, Kubernetes, Docker, Terraform, Datadog Datastores: Databricks, OpenSearch, Postgres Equal Opportunity Statement At Altana, we believe that ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache Spark and knowledge of Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps, ML/ ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache Spark and knowledge of Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps, ML/ ...

Security In-depth knowledge of the financial services industry and their IT systems Practical cloud native experience Preferred qualifications, capabilities, and skills Advanced Apache Spark experience(PySpark/Spark SQL), including performance tuning (partitioning, shuffle optimization, caching), troubleshooting, and designing scalable batch/stream processing patterns. Deep ...

Management Stakeholder Collaboration Leadership & Mentorship Technology Stack The successful candidate will have experience working with some or all of the following technologies: Data Processing: Apache Spark/PySpark Workflow Orchestration: Apache Airflow Containerisation: Kubernetes Cloud Platform: Microsoft Azure Data APIs & Semantic Layer: CubeJS Programming Languages: Python ...

Management Stakeholder Collaboration Leadership & Mentorship Technology Stack The successful candidate will have experience working with some or all of the following technologies: Data Processing: Apache Spark/PySpark Workflow Orchestration: Apache Airflow Containerisation: Kubernetes Cloud Platform: Microsoft Azure Data APIs & Semantic Layer: CubeJS Programming Languages: Python ...

cleansed, conformed business layers) while securely handling identifiable data. Scalable Data Processing: Develop, tune, and monitor large-scale batch processing pipelines using Azure Databricks (Spark-based processing) and PySpark. Orchestration & Workflow: Build automated data workflows using Microsoft Fabric Data Factory and Azure Data Factory to move data efficiently across … Microsoft Azure, with practical exposure to the Microsoft Fabric ecosystem (OneLake, Data Factory, Fabric Lakehouse). Big Data Processing: Deep technical expertise with Apache Spark/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL ...

Strong experience in IaC and guidance on deploying infrastructure into different environments. Knowledge of handling and transforming various data types (JSON, CSV, etc.) with Apache Spark, Databricks or Hadoop. Good understanding of modern data system design architectures (Data Warehouse, Data Lakes, Data Meshes). Ability to create data ...

batch and streaming data pipelines using Kafka, Flink, and ETL technologies Design and build unified analytics engine designed for processing large-scale data using Apache Spark and related tools Develop and optimize analytical data models for time-series, financial metrics, and trading activity Implement and manage analytical databases ...

troubleshoot issues, improve performance and work closely with engineers, analysts, data scientists and technical leads. Key Responsibilities You will: Build scalable pipelines using Databricks, Spark and PySpark. Develop ETL/ELT and data transformation workflows using Python and SQL. Create and maintain Databricks notebooks, jobs, workflows and clusters. Work ...

Lake Storage. Advanced SQL development and performance tuning skills, including experience with Azure SQL Database and SQL Server. Proficiency in Python, PySpark, Scala and Spark for data processing and distributed computing. Understanding of data warehousing concepts, dimensional modelling and ETL/ELT frameworks. Knowledge of data quality, data governance ...

with data warehousing concepts and technologies (e.g., Azure Synapse Analytics, Azure Data Lake Storage). Familiarity with big data technologies and frameworks (e.g., Hadoop, Spark) is a plus. Knowledge of data governance and data security best practices. Strong problem-solving skills and attention to detail. Excellent communication and collaboration ...

client engagements. Desirable • AWS certification at Associate or Professional level • Experience with streaming and event-driven data technologies • Experience with technologies such as Spark, Kafka or Databricks • Experience delivering data platforms within public sector, defence or other regulated environments • Experience working within secure environments and understanding associated security ...

large-scale ETL development. Proficiency in SQL and programming languages such as Python, Java, or Scala. Hands‐on experience with big data frameworks (e.g., Spark, Hadoop, Flink) and cloud platforms (AWS, GCP, or Azure). Familiarity with data warehouse technologies (e.g., Snowflake, Redshift, BigQuery) and workflow orchestration tools (e.g. ...

maintain SC clearance. AWS certification at Associate or Professional level Experience with streaming and event‐driven data technologies Experience with technologies such as Spark, Kafka or Databricks Experience delivering data platforms within public sector, defence or other regulated environments Experience working within secure environments and understanding associated security ...

messaging architectures. Data Platforms & Cloud Technologies Hands-on experience with: Azure Data Lake Databricks Snowflake Delta Lake Azure Synapse Analytics/Microsoft Fabric Spark/PySpark Strong understanding of: Data Lake and Lakehouse architecture Hybrid cloud environments Enterprise integration patterns Modern data formats including Parquet, Avro, and ORC DevOps ...

maintain SC clearance. Desirable AWS certification at Associate or Professional level Experience with streaming and event‐driven data technologies Experience with technologies such as Spark, Kafka or Databricks Experience delivering data platforms within public sector, defence or other regulated environments Experience working within secure environments and understanding associated security ...

maintain SC clearance. Desirable AWS certification at Associate or Professional level Experience with streaming and event-driven data technologies Experience with technologies such as Spark, Kafka or Databricks Experience delivering data platforms within public sector, defence or other regulated environments Experience working within secure environments and understanding associated security ...

/CD platform (Github Actions and Admin), Password/Secret Management (hashicorp vault). Strong Data related programming skills SQL/Python/Spark/Scala. Database technologies in relation to Data Warehousing/Data Lake/Lake housing patterns and relevant experience when handling structured and non-structured ...

Nice to Have Hands‐on experience with cloud platforms such as Azure, AWS or GCP. Exposure to modern data platforms such as Snowflake, Databricks, Spark or similar. Experience with MLOps practices such as model deployment, monitoring, versioning or model registries. Exposure to workflow orchestration or deployment tools such ...

architecture (SQL and NoSQL) Experience with cloud technologies Experience with cloud platforms Experience with CI/CD practices Experience with big data tools e.g. Spark, Hadoop, NiFi Experience managing direct reports, providing regular guidance and performance feedback Ability to create delivery plans with estimated timelines for personal and team ...

similar data‐focused role Strong SQL and Python development skills Experience building and supporting scalable data pipelines and transformation processes Hands‐on experience with Apache Spark, Databricks, dbt and cloud‐based data platforms Good understanding of data modelling principles, data lakes and data warehouse architectures Experience working with … Essential Degree in Computer Science, Software Engineering, Data Engineering, Information Systems, Mathematics or a related discipline; or equivalent practical experience Desirable Relevant certifications in Spark, cloud technologies, data engineering or software engineering practices Why should you apply? We’re not a perfect place – but we’re a people place. ...