101 to 106 of 106 Apache Spark Jobs in the UK

Developing and maintaining scalable data pipelines using Python - Automating manual Excel/VBA workflows into reliable, production-grade systems - Working with large datasets using Spark and Azure (Databricks/Synapse) - Supporting data modelling and analytics use cases (Crossing over to more Data Science driven work) - Driving best practices including … code reviews - Collaborating with stakeholders to deliver actionable data solutions What they’re looking for: - Strong Python and SQL skills - Experience or exposure to Spark, big data tools, or cloud platforms (ideally Azure) What’s on offer: - The chance to join a business at an exciting stage of growth ...

match rates, reduce false positives/negatives, and support downstream fraud and KYC models. Build and maintain scalable data pipelines and feature stores in Spark/PySpark (or Scala), including data normalization, deduplication, and feature computation across large PII datasets in AWS/Databricks environments. Lead A/… preferred. Strong proficiency in Python (preferred) or Scala, including experience with ML libraries such as scikit‐learn, XGBoost, TensorFlow or PyTorch. Extensive experience with Spark or PySpark and distributed data systems (e.g., AWS EMR, Databricks) working on very large, messy datasets. Deep understanding of supervised and unsupervised learning, feature ...

Design, build, and maintain scalable data pipelines and workflows to support analytics, fraud detection, model development, and ongoing data monitoring (e.g., using Spark, Airflow, or similar distributed systems). Leverage and build agentic AI and LLM-powered systems to automate data exploration, anomaly detection, vendor evaluation, and investigative workflows … both batch and near‐real‐time use cases. Experience building and maintaining data pipelines and workflows in distributed or large‐scale environments (e.g., Spark, Airflow, Databricks, or similar technologies). Demonstrated ability to evaluate and work with third‐party data vendors or external datasets, including designing tests for data ...

Aker Systems was founded in 2017 by a team of experienced technology professionals who recognised an opportunity to provide highly secure enterprise data platforms to large organisations. We build and operate ground-breaking, ultra-secure ...

backend services using Java Work within cloud-native AWS environments Contribute to containerised workloads (Docker/Kubernetes) Build and support data-intensive systems (including Spark where applicable) Collaborate with architects and engineers on system design and delivery Participate in agile delivery, code reviews, and technical discussions Required Skills & Experience … background Experience working in distributed systems Conditional (must meet at least 2 of the below) AWS 3+ years hands-on experience Big Data/Spark 3+ years Containerisation Docker and/or Kubernetes, 3+ years Payments domain 23 years experience Nice to Have Infrastructure as Code (Terraform) Event-driven ...

Senior DataEngineer-RemoteWorking LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions. Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions. Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures DevelopingandmaintainingETL/ELTprocessesusingmoderntools Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms Ensuringdataquality,reliability,andoperationalexcellence Leadingthedesignanddeliveryofnewdataproductsandpipelines Collaboratingcloselywithstakeholdersandtechnicalteams Providingtechnicalleadershipandmentoringwithindeliveryteams Whattherelookingfor: Strongexperienceindataengineeringroles Provenleadershiporlinemanagementexperience ...