26 to 47 of 47 Apache Spark Jobs in London

record designing, scaling, and maintaining distributed systems and high-volume data pipelines. Strong background in cloud-native architectures and modern data stacks such as Spark or Databricks. Experience working closely with Data Science teams, including delivering production-grade ML models and pipelines. Solid understanding of modern frontend engineering practices ...

/CD , Git-based workflows, and automation Strong problem-solving skills and a collaborative mindset Bonus Terraform or other IaC tools Slurm, Kueue, Ray, Spark, or similar systems GPU tooling (CUDA, Nvidia operators, schedulers) Experience supporting ML training or data science teams ...

complex, data-driven platform used by enterprise customers. You will set technical direction and guide architecture across distributed systems, large-scale data processing (Scala, Spark, Databricks), machine learning pipelines, and cloud-native infrastructure on AWS. A core focus is driving high-quality execution through strong engineering practices including ...

innovative thinking process and encourage it in others. Working within the agile framework at Chambers Skills and Experience Strong proficiency in SQL , including Spark SQL and MS SQL Server , for querying, data manipulation, and performance optimization. Hands-on experience with Python and PySpark for data processing, transformation, and automation … design, build, and maintain scalable data pipelines for batch and streaming data using modern frameworks. Experience working with Databricks for big data processing, Spark-based transformations, and collaborative analytics workflows. Familiarity with Azure Data Factory (ADF) for data orchestration and integration across cloud environments. Skilled in version control using ...

please only apply if you hit this criteria. Required Experience Strong background in data engineering within distributed data environments Hands-on expertise with AWS , Spark , Glue , and Snowflake Experience building and optimising data pipelines & reporting workflows Ability to work closely with internal engineering and controls teams Experience upgrading … modernising existing workflows and frameworks For Lead-level: prior experience leading engineering teams or workstreams Tech Stack AWS (Glue, S3, Lambda, Step Functions) Apache Spark Snowflake Java if you are interested in this role then please apply here or email me your most recent and up to date ...

Data Platform Engineer – London (AWS, Apache Spark, AWS Glue, Iceberg, S3, RDS, Redshift, Kafka/MSK, Python, Terraform, Ansible, CI/CD, Jenkins, GitLab, Snowflake, Databricks) Working with an established FinTech client in London who is looking for a Data Platform Engineer to play a key role ...

team Exposure to sectors such as defence, public sector and consumer/retail Familiarity with modern data and AI tooling such as Python, Spark, Databricks, Azure cloud, and Generative AI platforms (experience with specific tools is helpful but not essential) A mix of technical, product and operational experience - comfortable ...

team Exposure to sectors such as defence, public sector and consumer/retail Familiarity with modern data and AI tooling such as Python, Spark, Databricks, Azure cloud, and Generative AI platforms (experience with specific tools is helpful but not essential) A mix of technical, product and operational experience - comfortable ...

experience with Python (2+ Years) Experience working with REST Microservices Strong SQL Experience working with very large data sets. Knowledge of big data tools (Spark, Kafka etc) Experience working in finance (Preferred) Strong formal education - ideally in Computer Science If this sounds of interest, then please do not hesitate ...

ensure A-Level grades are included on your CV) Basic scripting knowledge in Python or Bash Excellent customer-facing skills You have a sales spark - while this role isn't a focused sales role, this is required due to the nature of the role A motivated self-starter with ...

power, and deep knowledge of just about every industry. We believe a mix of data, analytics, automation, and responsible AI can do almost anything—spark digital metamorphoses, widen the range of what humans can do, and breathe life into smart products and services. Want to join our crew ...

power, and deep knowledge of just about every industry. We believe a mix of data, analytics, automation, and responsible AI can do almost anything—spark digital metamorphoses, widen the range of what humans can do, and breathe life into smart products and services. Want to join our crew ...

lead embedded data science teams, partnering effectively with senior stakeholders across multiple domains Hands-on technical expertise with tools such as Databricks, Python, Spark, and GCP/BigQuery Engineering mindset, with experience moving teams toward machine learning engineering best practices Credibility to lead long-tenured individual contributors while providing ...

team engineering organisations Strong background in distributed systems, large-scale data pipelines and ML model deployment Deep understanding of modern data stacks such as Spark, Databricks or cloud-native architectures Solid understanding of modern frontend engineering, DevOps knowledge and experience delivering machine learning systems Strong communication, organisation and stakeholder ...

production-ready PySpark code. Strong understanding of data and data warehousing concepts, including: ETL/ELT, Data models, Dimensions and facts, Data marts, SCDs Spark Performance & Optimisation Strong knowledge of Spark execution concepts, including partitioning, optimisation, and performance tuning. Experience troubleshooting and improving distributed data processing pipelines. Python ...

Position Summary The Senior AI Architect is responsible for designing and implementing advanced AI and cloud architectures, with a strong emphasis on integrating generative AI technologies. This role requires a broad understanding of AI and ...

orchestrate data pipelines across Snowflake and AWS environments. Apply data modeling, warehousing, and architecture principles (Kimball/Inmon). Develop pipeline programming using Python, Spark, and SQL; integrate APIs for seamless workflows. Support Machine Learning and AI initiatives, including NLP, Computer Vision, Time Series, and LLMs. Implement MLOps … data pipeline development and orchestration. Proficient with cloud platforms (Snowflake, AWS fundamentals). Solid understanding of data architecture, warehousing, and modeling. Programming expertise: Python, Spark, SQL, API integration. Knowledge of ML/AI frameworks, MLOps, and advanced analytics concepts. Experience with CI/CD, data testing frameworks, and versioning ...

architecture, and network stack optimization. Data Infrastructure: Familiarity with the modern data stack, including time-series databases, object stores, and streaming frameworks like Apache Flink or Spark Streaming . Engineering Rigor: You write clean, testable, and reliable code. You understand that in this environment, a system failure ...

scale Hadoop clusters across on-prem environments Supporting and optimising our HDFS-based Data Lake and adjacent platform components Working with technologies including Hadoop, Spark, HDFS, YARN, Hive, Oozie , and related tooling Partnering with engineering teams to deliver scalable, high-quality data solutions Contributing to our cloud migration strategy … objectives Ensuring stability, performance, and security of critical data services What We're Looking For Strong hands-on experience with Hadoop ecosystem tools (HDFS, Spark, Hive, Oozie, YARN, etc.) Experience operating and supporting large, complex, distributed clusters Understanding of data engineering principles and data platform design Experience with ...

Head of Engineering - Remote - UK - £80,000-£150,000 GenAI, ML, Spark Why Join This Company? This is a leadership role for someone who wants to work in a truly AI-first engineering environment, not a business experimenting at the edges. The company is already well ahead … solving rather than legacy ways of working. You'll Work With Cross-functional teams spanning: Web Engineering (React/TypeScript) Data Engineering (distributed systems, Spark-based platforms) Data Science & Machine Learning A highly technical CTO who is deeply invested in AI adoption and modern engineering Product leadership following significant ...

Senior DataEngineer-RemoteWorking LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions. Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions. Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures DevelopingandmaintainingETL/ELTprocessesusingmoderntools Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms Ensuringdataquality,reliability,andoperationalexcellence Leadingthedesignanddeliveryofnewdataproductsandpipelines Collaboratingcloselywithstakeholdersandtechnicalteams Providingtechnicalleadershipandmentoringwithindeliveryteams Whattherelookingfor: Strongexperienceindataengineeringroles Provenleadershiporlinemanagementexperience ...

DataEngineer-RemoteWorking LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions. Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions. Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures DevelopingandmaintainingETL/ELTprocessesusingmoderntools Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms Ensuringdataquality,reliability,andoperationalexcellence Leadingthedesignanddeliveryofnewdataproductsandpipelines Collaboratingcloselywithstakeholdersandtechnicalteams Providingtechnicalleadershipandmentoringwithindeliveryteams Whattherelookingfor: Strongexperienceindataengineeringroles Provenleadershiporlinemanagementexperience Hands ...