51 to 75 of 227 Apache Spark Jobs in London

Data Engineer, Data Cloud

Location
Greater London, England, United Kingdom
services, a good understanding of cloud architecture, and proficiency with Terraform for provisioning and managing cloud infrastructure Experience building streaming data pipelines using Apache Flink, Apache Kafka, Spark Structured Streaming, or similar technologies Knowledge of modern data lakehouse architectures and ecosystem such as Kafka, Flink, Spark … with a SaaS or FinTech company Technologies we use: AWS Services: MSK, EC2, VPC, IAM, S3, SQS, Managed Flink, EKS Data Lakehouse: Kafka, Flink, Spark, Databricks, DBT, Debezium #J-18808-Ljbffr ...

Azure Data Engineer (Fabric & Cloud Data Platform)

Location
Greater London, England, United Kingdom
cleansed, conformed business layers) while securely handling identifiable data. Scalable Data Processing: Develop, tune, and monitor large-scale batch processing pipelines using Azure Databricks (Spark-based processing) and PySpark. Orchestration & Workflow: Build automated data workflows using Microsoft Fabric Data Factory and Azure Data Factory to move data efficiently across … Microsoft Azure, with practical exposure to the Microsoft Fabric ecosystem (OneLake, Data Factory, Fabric Lakehouse). Big Data Processing: Deep technical expertise with Apache Spark/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL ...

Lead Data Engineer

Hiring Organisation
Hexwired Recruitment Limited
Location
London, United Kingdom
Employment Type
Permanent
Salary
£80000 - £115000/annum
PostgreSQL (or similar relational databases) and NoSQL databases. Experience with vector databases such as Qdrant, Milvus or pgvector . Familiarity with technologies such as Apache Spark, Airflow, Kafka, Elasticsearch/OpenSearch, Pandas and Polars . Strong understanding of data modelling, storage architecture and performance optimisation. Desirable Experience working ...

Senior Data Scientist

Location
Greater London, England, United Kingdom
equivalent apprenticeship/work experience in a subject such as Mathematics, Statistics, Computer Science, or Geography DESIRABLE Exposure to Big Data Analytics (BigQuery, Apache Spark), Parallel Computing (Kubernetes, Databricks), Cloud Engineering (AWS, GCP, Azure), Spatial Query Optimisation, Graph Computing, or Microservices Relevant industry sector knowledge across retail, property ...

Senior Data & AI Architect

Location
Greater London, England, United Kingdom
platforms using AWS, Azure or Google Cloud. Expertise in data modelling, SQL, NoSQL and modern database technologies. Experience with big data technologies, such as Apache Spark, Hadoop or distributed computing frameworks. Programming knowledge in Python, Scala, Java or similar languages. Experience architecting AI and machine learning solutions, including ...

Senior Developer Advocate - Data Observability

Location
Greater London, England, United Kingdom
into a coherent operational story. Data engineering & processing systems: You have built, operated, or supported production data pipelines using distributed processing systems such as Apache Spark or Databricks, and understand common failure modes, performance tradeoffs, and operational challenges in batch and hybrid pipelines. Data platforms & analytics systems ...

Python Engineer Data Domains

Location
Greater London, England, United Kingdom
grade code Distributed Data Processing Experience building and operating: batch processing systems distributed data systems near real-time streaming pipelines Familiarity with: Kafka Flink Spark Databricks experience is a strong plus Databases & Data Warehousing Comfortable working with: RDBMS data warehousing solutions Strong understanding of data modelling aligned with business ...

Principal Data Consultant - AWS / Databricks

Location
Greater London, England, United Kingdom
Designed and delivered modern data platforms end to end on AWS and/or Databricks Still writes production-grade code: strong Python and SQL, Spark, dbt or equivalent, infrastructure-as-code (Terraform preferred) Track record in a consultancy, systems integrator or other client-facing setting — comfortable being accountable ...

Senior Consultant - Enterprise Data & Analytics - AI Factory

Location
Greater London, England, United Kingdom
into business processes and decision-making.**Technical Skills:**Hands-on experience in one or more of the following areas:Programming & Analytics – Python, SQL, R, Spark, machine learning and advanced analyticsCloud & Data Platforms – Microsoft Fabric, Databricks, Snowflake, Azure, AWS and GCPAI Technologies – GenAI frameworks, Agentic AI, vector databases, semantic search ...

Senior EMEA AI Solutions Architect

Location
Greater London, England, United Kingdom
migration strategies as well as with machine learning, LLMs, and agentic AI workflows. Proficient in programming, debugging, and problem‐solving using SQL, Python, Apache Spark, cloud platforms, and Lakehouse architectures and with AI tools. Experience collaborating with Global System Integrators (GSIs) and third‐party consulting organizations to drive ...

Senior Data Engineer, Proprietary Research

Location
Greater London, England, United Kingdom
years of strong experience in Python development Experience with system design and leading multi-person development efforts Experience with distributed data processing frameworks like Spark Experience with building scalable APIs, and software applications w/focus on backend infrastructure Experience with cloud platforms and services for data storage ...

The Core Engineering - London - Vice President - Software Engineering

Location
City Of London, England, United Kingdom
scale datasets Preferred Qualifications Knowledge or interestin investment banking or financial instruments Experience with big data concepts, such as Lakehouse, ETL data pipelines (e.g. Spark, AWS Glue) Experience with near real time and event‐based systems like Kafka ABOUT GOLDMAN SACHS At Goldman Sachs, we commit our people, capital ...

Senior Tech, Data & AI Auditor

Location
Greater London, England, United Kingdom
including data quality, data lineage, model testing, prompt testing, access controls, monitoring, explainability, human oversight and governance. Strong data analytics capability, including Python, SQL, Spark, Power BI, Tableau or equivalent tools. Ability to translate complex technical concepts into practical business risks, control implications and clear audit conclusions. Internal Audit ...

Senior Data Engineer, Fabric & Analytics (Remote)

Location
Greater London, England, United Kingdom
support comprehensive data analysis and reporting. Implement and uphold best practices in data governance, security, and compliance, using tools like Azure Purview, UnityCatalog, and Apache Atlas tomaintaindata integrity and trust. Ensure data quality and integrity through meticulous attention to detail and rigorous QA processes, continually refining and optimising data … ease. Understanding of Data Lakehouse Architecture:You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions:Youpossessexperience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or evenon-premiseSpark clusters, usingPySparkor Spark SQL to manage ...

Data Engineer

Location
Greater London, England, United Kingdom
into actionable insights, and power high-impact trading and AI decisions. Ideal Candidate: 5+ years of experience in data engineering with MPP databases or Apache Spark Strong skills in Python and SQL, with solid data modeling experience Extensive knowledge of AWS/cloud services (ECS, EKS, Lambda, Kinesis ...

Lead Data Engineer

Location
Greater London, England, United Kingdom
languages like Python, or functional programming languages like Scala Demonstrated experience in design and development of big data applications using tech stacks like Databricks, Apache Spark, HDFS, HBase and Snowflake Commendable skills in building data products, by integrating large sets of data from hundreds of internal and external ...

Principal Engineer

Location
Greater London, England, United Kingdom
hands-on engineering experience delivering enterprise data platforms or analytics solutions. Strong commercial experience building and delivering Databricks solutions in production. Deep expertise in Apache Spark, including performance tuning and optimisation at scale. Expert-level Python and SQL, with a track record of writing production-quality, well-tested ...

Databricks Lakehouse Platform Engineer (Hybrid, UK)

Location
Greater London, England, United Kingdom
internal knowledge Competencies and Behaviours: 3+ years experience in data platform engineering, cloud engineering, or similar roles Strong hands-on experience with Databricks, including Apache Spark, Delta Lake, Workflows Proven experience designing and deploying data platforms on AWS and/or Azure (e.g. ADLS, S3, VNet/ ...

Data Engineer Vice President

Hiring Organisation
Hackajob Ltd
Location
South West London, South East, United Kingdom
Employment Type
Permanent Contract, On-site
Experience with non-relational database technologies (for example, MongoDB) and associated query patterns Familiarity with large-scale data processing and analytics platforms (for example, Apache Spark or Hadoop) Experience implementing, or partnering closely on, data quality, metadata, and governance controls to improve reliability and auditability Understanding of modern ...

Senior Application Architect

Location
Greater London, England, United Kingdom
/ML offerings. Strong programming skills (Python, Java, C++) and experience with DevOps practices (CI/CD). Knowledge of big data technologies (Hadoop, Spark, Flink). Knowledge of NoSQL databases (MongoDB, Cassandra, DynamoDB), message queueing systems (Kafka, RabbitMQ), and version control systems (Git). Preferred Skills: Hugging Face ...

Senior Data Scientist

Location
Greater London, England, United Kingdom
libraries Strong background in applied machine learning, model development and data engineering Experience with cloud environments (Azure, AWS, GCP) and tools such as Spark, Hive, Redshift Demonstrated ability to lead cross-functional teams and mentor junior practitioners Ability to communicate complex technical concepts clearly to non-technical audiences Bonus ...

Senior Backend Engineer

Location
Greater London, England, United Kingdom
Company description Publicis Media harnesses the power of modern media through leading agency brands Dysrupt, Infinite Roar, Publicis Collective, Publicis Health Media, Spark Foundry, Starcom and Zenith, as well as global accelerator PMX; immersive experience group PMCI and access to integrated platform-based technologies and offerings from CJ, Epsilon ...

Data & AI Senior Manager

Location
London, United Kingdom
power, and deep knowledge of just about every industry. We believe a mix of data, analytics, automation, and responsible AI can do almost anything spark digital metamorphoses, widen the range of what humans can do, and breathe life into smart products and services. Want to join our crew ...

Data & AI Senior Manager

Hiring Organisation
Hackajob Ltd
Location
South West London, South East, United Kingdom
Employment Type
Permanent Contract, On-site
power, and deep knowledge of just about every industry. We believe a mix of data, analytics, automation, and responsible AI can do almost anything—spark digital metamorphoses, widen the range of what humans can do, and breathe life into smart products and services. Want to join our crew ...

Senior Machine Learning Engineer, Ranking

Location
Greater London, England, United Kingdom
work. Technologies and Tools Python Machine learning frameworks (e.g. PyTorch, TensorFlow, scikit-learn) ML/MLOps tooling (e.g. SageMaker, MLflow, TFServing) Spark and Databricks AWS services (e.g. IAM, S3, Redis, ECS) CI/CD tooling and best practices Streaming and batch data systems (e.g. Kafka, Airflow, RabbitMQ) Additional Information ...