26 to 48 of 48 Apache Spark Jobs in the UK excluding London

teams to drive outcomes Core skills (indicative) Production pipeline design and deployment experience Strong engineering capability with Python, SQL, plus big data tooling (e.g., Spark, and Java/Scala where relevant AWS, Azure, GCP Working pattern Hybrid working, with the team on client site or in the office ...

unstructured datasets. Strong stakeholder management and communication skills. Desirable Insurance or financial services experience, ideally within pricing, underwriting, fraud, or claims. Experience with Spark, Hadoop, NoSQL, or modern big data platforms. Agile delivery experience and collaborative software development practices. Experience leading analytics teams and driving organisational change. Robert Walters ...

Experience with: Relational databases (SQL Server, PostgreSQL, Oracle, MySQL) Data visualisation tools (Power BI, Tableau, Spotfire) Cloud platforms, particularly Microsoft Azure Big data technologies (Spark, Elasticsearch, Hadoop) Statistical modelling and advanced analytics Machine learning and pattern recognition techniques Web technologies such as HTML and JavaScript ...

Limited (DDIL) communications environments, supporting local processing at the edge Building streaming ingestion, event pipelines, and batch/stream processing workloads using technologies like Apache Flink and/or Apache Spark Implementing metadata, lineage, and catalogue capabilities along with policy-based security and governance Job Requirements: Extensive ...

large-scale analytics platforms Expert knowledge of Quantexa, Entity Resolution (ER), Scoring Framework, Contextual Network Generation Expertise in Big Data technologies, including: Scala, Apache Spark, Hadoop, ElasticSearch Strong stakeholder management and communication skills - capable of interfacing with both business and technical audiences. The work is heavily weighted toward ...

ongoing basis. Contribute to the planning, research, recommendation, and implementation of current and future IT strategies. PERSON SPECIFICATION: Essential Demonstrable experience of Spark (PySpark) and SQL. Demonstrable experience of Fabric components. Demonstrable experience of working on enterprise environments. Experience of designing architectural Fabric processes. Experience of Azure Data Factory. ...

with large-scale and unstructured datasets. Python development experience, including technologies such as PySpark, Pandas and PyArrow. Expertise in distributed data processing frameworks, particularly Apache Spark. Experience building and maintaining ETL pipelines using technologies such as Apache Airflow, AWS Step Functions or Apache NiFi. Cloud platform experience ...

Swindon (we could consider twice per fortnight etc). Required Skills & Experience Strong experience testing data pipelines/ETL processes Hands-on experience with Apache Spark Experience working with Databricks and/or AWS Glue Proven ability to build automation or testing frameworks (Python preferred) Experience with large ...

with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well ...

insights that drive the AI product roadmap forward. Tech Stack Includes : AWS: EC2, Athena, S3, Kinesis, Lambda, Step Functions, RDS, QuickSight Languages & Frameworks: Python, Spark, TensorFlow, LLMs & LLM coding tools Databases & Search: MySQL, PostgreSQL, SOLR Requirements Experience: Minimum 3 years of post-degree commercial experience in a data science ...

Senior Data Engineer - Elastic Stack/Apache NiFi Worcester (Onsite) £70-£75 per hour | Outside IR35 12-Month Rolling Contract Compressed Working Available - Work 3 Long Days, Paid for 5 We are seeking an experienced Senior Data Engineer to join a high-profile programme delivering secure, scalable data platforms … full five-day week). Key Responsibilities Design, develop, and maintain secure, scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi Build and manage data ingestion, transformation, and integration processes Ensure data quality, integrity, security, and compliance with governance standards Collaborate with data architects ...

delivery of secure, scalable and high-performance data solutions within a regulated environment. This role will focus heavily on the Elastic Stack and Apache NiFi, supporting enterprise-scale data ingestion, transformation and analytics capabilities across complex secure systems. The successful candidate will play a key role in architecting resilient … compliance with strict governance and security standards. KEY RESPONSIBILITIES Data Architecture & Engineering - Design and implement scalable, resilient data pipeline architectures using Elastic Stack and Apache NiFi - Develop secure frameworks for data ingestion, transformation and integration - Build and maintain batch and real-time data processing pipelines - Ensure high standards ...

platform role Strong hands-on experience with AWS (EC2, S3, RDS, Lambda, Redshift, Glue, Athena) Experience with real-time and batch data processing - Apache Flink, Kafka, Spark, or similar Proficiency in one or more of: Python, Java, Scala Experience with data modelling , ETL processes, and distributed systems Comfortable ...

Deep experience with cloud platforms (Azure, AWS, or GCP) and cloud-native architecture Strong background in data engineering and large-scale data platforms (e.g., Spark, Databricks) Experience implementing MLOps frameworks, including CI/CD and model governance Experience integrating AI into microservices, APIs, and enterprise systems Exposure to advanced … similar programming languages) TensorFlow, PyTorch, or similar ML frameworks Cloud AI services (Azure AI, AWS SageMaker, GCP AI Platform) Data technologies (SQL/NoSQL, Spark, Databricks) API and microservices architecture DevOps/MLOps tooling Desirable Experience Experience with Generative AI solutions (e.g. Azure OpenAI, GPT-based models) Microsoft stack ...

Purpose of position Data sits at the heart of the company. This role ensures that Awin can fully leverage the data available across the group, enabling the development of robust reporting capabilities that support stronger ...

Senior Data Engineer 6 Month contract with extension Full-time Inside IR35 Hybrid 1 day every two weeks in the office Nottingham - NG1 4BX Join our client in embarking on an ambitious data transformation journey ...

skills, including query optimization and schema design. Programming experience in Python or Scala for data processing. Experience with big data processing tools such as Spark or Databricks. Experience with cloud data platforms such as Azure, AWS, or GCP, especially data services. Responsibilities Develop and implement ETL and ELT data … pipelines. Optimize SQL queries and support schema design and data modeling. Use Python or Scala to build and maintain data processing solutions. Work with Spark or Databricks for big data processing. Support data engineering work across cloud data platforms such as Azure, AWS, or GCP. Technologies AWS Azure ...

Join DWP Digital and lead the future of data engineering in one of the UK's largest and most impactful digital environments As a Senior Data Engineer, you'll join our Data Platform and Operations ...

Join DWP Digital and lead the future of data engineering in one of the UK's largest and most impactful digital environments As a Senior Data Engineer, you'll join our Data Platform and Operations ...

Join DWP Digital and lead the future of data engineering in one of the UK's largest and most impactful digital environments As a Senior Data Engineer, you'll join our Data Platform and Operations ...

Join DWP Digital and lead the future of data engineering in one of the UK's largest and most impactful digital environments As a Senior Data Engineer, you'll join our Data Platform and Operations ...

part of a data science team to analyse data, build insight, and support the development of predictive, analytical, and decisioning solutions. Use Python, Spark, and relevant data tools to clean, transform, explore, and analyse datasets. Support the development, testing, and validation of machine learning models, business metrics, dashboards … cleaning, feature engineering, exploratory analysis) Exposure to machine learning model development and evaluation Familiarity with cloud platforms or distributed data tools (e.g. Azure, AWS, Spark) Understanding of software engineering practices (e.g. Git, testing) Experience communicating insights through visualisation or presentations Interest in applying data science to customer or decisioning ...

Aker Systems was founded in 2017 by a team of experienced technology professionals who recognised an opportunity to provide highly secure enterprise data platforms to large organisations. We build and operate ground-breaking, ultra-secure ...