26 to 48 of 48 Apache Spark Jobs in the UK excluding London

environments Strong understanding of data security, governance, and access controls Desirable Experience Experience working within UK defence, intelligence, or national security programmes Exposure to Apache Kafka, Spark, Hadoop, or similar technologies Experience supporting data science or advanced analytics use cases Background working with classified datasets Contract Details Apply ...

promote DevOps. Support internal and external stakeholders, including customers and regulatory authorities. Tools, Systems & Continuous Improvement Manage and optimise the use of tools (e.g. Apache Spark, AWS Glue, Databricks or equivalent). Support system enhancements of data processing and transformation, migrations, and process improvements. Contribute to the development ...

driven decision making. What youll be doing Building and optimising scalable data pipelines using Java (primary) and technologies like Kafka, Flink, or Spark Developing real-time streaming solutions and event-driven architectures using AWS services (Kinesis, MSK, Lambda, Glue) Contributing to data architecture, modelling, and best practices across modern ...

source models or integrating with managed foundation model platforms. Hands-on experience building data pipelines and model workflows using tools such as Python, SQL, Spark or similar data processing technologies. Experience deploying and operating AI systems in cloud environments using containerisation, managed ML services or serverless architectures. Understanding ...

teams to drive outcomes Core skills (indicative) Production pipeline design and deployment experience Strong engineering capability with Python, SQL, plus big data tooling (e.g., Spark, and Java/Scala where relevant AWS, Azure, GCP Working pattern Hybrid working, with the team on client site or in the office ...

translate business needs into technical roadmaps What were looking for Strong experience in data architecture & large-scale systems design Expertise across Python, SQL, Spark, Scala Deep knowledge of AI/ML frameworks and modern data platforms Experience with cloud-native architecture and DevOps (Kubernetes, Terraform, CI/CD) Proven ...

manage multiple priorities and respond calmly to issues. • Self-motivated with a collaborative approach. Preferred: • Familiarity with AWS data tooling such as S3, Athena, Spark or notebooks. • Experience supporting data products or client-facing datasets. • Understanding of release testing, UAT or production support processes. • Exposure to Jira/Agile ...

expand their Devops team (multiple roles). Key requirements: Strong experience with AWS services and infrastructure as code using Terraform Data engineering expertise using Apache Spark and Python, including development in Jupyter Notebooks Workflow orchestration and containerisation experience with Apache Airflow and Docker Version control ...

Proven experience with modern data platforms such as Snowflake (AI Data Cloud) and cloud‐native services. Good understanding of open‐source table formats, specifically Apache Iceberg , to enable interoperability, schema evolution, and high‐performance analytics across multiple engines. Programming: Expert‐level proficiency in Java , Python and SQL . … Data & Orchestration: Hands‐on experience with Spark, Kafka, and orchestration tools like Apache Airflow, Dagster, or dbt. Data Modeling: Deep understanding of data warehousing and modern data lakehouse architecture. Leadership & Soft Skills Mentorship: Proven track record of upskilling junior and senior engineers. Communication: Ability to explain complex technical ...

large-scale analytics platforms Expert knowledge of Quantexa, Entity Resolution (ER), Scoring Framework, Contextual Network Generation Expertise in Big Data technologies, including: Scala, Apache Spark, Hadoop, ElasticSearch Strong stakeholder management and communication skills - capable of interfacing with both business and technical audiences. The work is heavily weighted toward ...

applying emerging ML techniques. Desirable: Experience with cloud platforms (AWS) and containerisation (such as Docker, Podman, Kubernetes). Exposure to big data technologies (Spark, Hadoop) and Apache tools. Knowledge of NLP, computer vision, and deep learning architectures. Familiarity with Agile and DevOps practices. STEM degree or equivalent experience ...

with Git, Python, and Ansible Familiarity with containers (Docker, Kubernetes, LXC) Experience with monitoring tools and SQL databases Exposure to big data systems (e.g., Spark, HDFS, Airflow) Summary A great opportunity for someone who enjoys solving complex infrastructure challenges, automating systems, and working closely with engineering teams. Spectrum ...

team Exposure to sectors such as defence, public sector and consumer/retail Familiarity with modern data and AI tooling such as Python, Spark, Databricks, Azure cloud, and Generative AI platforms (experience with specific tools is helpful but not essential) A mix of technical, product and operational experience - comfortable ...

optimise scalable data pipelines (batch and real-time) Work primarily with Java , with exposure to Python and streaming frameworks such as Kafka, Flink, or Spark Develop event-driven architectures and streaming integrations Integrate data from multiple sources using AWS services (e.g. Kinesis, Lambda, Glue, MSK) Architecture & Data Management Contribute … experience in Data Engineering or large-scale data systems Strong programming skills in Java (preferred) or Python Hands-on experience with Kafka, Flink, or Spark (streaming focus is a strong advantage) Solid understanding of streaming concepts (event time, state, backpressure) Experience building ETL/ELT or real-time data ...

Data Engineer Salary up to £70,000 + discretionary bonus Hybrid working model with collaborative office environment Work with Databricks, Spark Structured Streaming, Kafka (MSK), and AWS Apply online or contact Chelsea Hackett via ABOUT THE CLIENT Due to continued growth, we're seeking a skilled Data Engineer ...

across complex engagements. What You’ll Do Lead design and implementation of enterprise-scale data pipelines and lakehouse architectures Provide technical leadership across Databricks, Spark and cloud data platforms Drive delivery of data engineering workstreams within Agile programmes Engage with client stakeholders to translate business objectives into technical solutions … architecture documentation, design artefacts and delivery roadmaps Must-Have Requirements 7+ years experience in data engineering or data platform development Strong experience with Databricks, Spark and distributed data processing Experience with Unity Catalog, Delta Lake and streaming architectures Advanced Python and Microsoft SQL Server development Experience designing enterprise data ...

systems, and modern cloud architectures . What youll be doing Design and build scalable data pipelines (batch & streaming) Work with technologies like Kafka, Flink, Spark, AWS services Develop real-time, event-driven data solutions Contribute to data architecture and modelling best practices Implement CI/CD pipelines and Infrastructure … deliver clean datasets What were looking for Strong programming skills in Java (preferred) or Python Experience with streaming technologies (Kafka/Flink/Spark) Knowledge of distributed systems and large-scale data processing Experience with cloud platforms (AWS preferred) Familiarity with DevOps tools and containerisation (Docker/Kubernetes ...

Developing and maintaining scalable data pipelines using Python - Automating manual Excel/VBA workflows into reliable, production-grade systems - Working with large datasets using Spark and Azure (Databricks/Synapse) - Supporting data modelling and analytics use cases (Crossing over to more Data Science driven work) - Driving best practices including … code reviews - Collaborating with stakeholders to deliver actionable data solutions What they’re looking for: - Strong Python and SQL skills - Experience or exposure to Spark, big data tools, or cloud platforms (ideally Azure) What’s on offer: - The chance to join a business at an exciting stage of growth ...

Educated to degree level in a relevant IT subject. Expertise in designing and implementing data pipelines using Azure services (Azure Data Factory, Data storage), Spark and Databricks. Expertise in data modelling, database design and designing enterprise data architecture . Azure Data Stack SQL server, NoSQL, Nanobricks, ADT, Spark ...

teams and setting best practice standards Designing data models, ETL pipelines, warehousing solutions and distributed systems Building and supporting real-time streaming platforms using Apache Flink and related technologies Ensuring compliance with industry standards and government regulations Tracking delivery against agreed designs and managing design deviations Supporting pre-sales … data architecture on complex, secure programmes Strong hands-on AWS experience (EC2, S3, RDS, Lambda, Redshift); Azure or GCP a bonus Proven experience using Apache Flink for real-time, high-throughput data processing Experience leading, mentoring and developing technical teams Confident engaging with senior technical and non-technical stakeholders ...

Aker Systems was founded in 2017 by a team of experienced technology professionals who recognised an opportunity to provide highly secure enterprise data platforms to large organisations. We build and operate ground-breaking, ultra-secure ...

backend services using Java Work within cloud-native AWS environments Contribute to containerised workloads (Docker/Kubernetes) Build and support data-intensive systems (including Spark where applicable) Collaborate with architects and engineers on system design and delivery Participate in agile delivery, code reviews, and technical discussions Required Skills & Experience … background Experience working in distributed systems Conditional (must meet at least 2 of the below) AWS 3+ years hands-on experience Big Data/Spark 3+ years Containerisation Docker and/or Kubernetes, 3+ years Payments domain 23 years experience Nice to Have Infrastructure as Code (Terraform) Event-driven ...

Senior DataEngineer-RemoteWorking LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions. Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions. Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures DevelopingandmaintainingETL/ELTprocessesusingmoderntools Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms Ensuringdataquality,reliability,andoperationalexcellence Leadingthedesignanddeliveryofnewdataproductsandpipelines Collaboratingcloselywithstakeholdersandtechnicalteams Providingtechnicalleadershipandmentoringwithindeliveryteams Whattherelookingfor: Strongexperienceindataengineeringroles Provenleadershiporlinemanagementexperience ...