1 to 25 of 48 Apache Spark Jobs in London

Lead Python Data Engineer - Leading Technology AI Brand

Hiring Organisation
MLR Associates
Location
London, United Kingdom
Employment Type
Permanent
Salary
GBP 70,000 - 100,000 Annual
acceptable). Experience with NoSQL databases. Experience with vector databases used in modern AI systems. Strong programming experience in Python. Frameworks/Infrastructure:- Apache Spark Apache Airflow Kafka Elasticsearch/OpenSearch ...

Java Developer Contract

Hiring Organisation
Atrium (EMEA)
Location
London Area, United Kingdom
flexibility to attend key meetings onsite. Required: Good knowledge and experience using programming languages such as Python, Java or SQL and tools such as Apache Spark or in this instance LakeFS, for data processing and management. Useful: R, Apache Spark Very Useful: Python, SQL, JSON, LakeFS ...

Java Developer

Hiring Organisation
Experis
Location
City of London, London, United Kingdom
Employment Type
Contract
Contract Rate
£468 - £469/day
data sets can be effectively combined and analysed. They can also utilise programming languages such as Python, Java or SQL and tools such as Apache Spark or in this instance LakeFS, for data processing and management. Useful - R, Apache Spark Very useful - Python, SQL, JSON, LakeFS. ...

Data Engineer

Hiring Organisation
Experis
Location
London Area, United Kingdom
Skills & Experience Strong experience with Python and/or SQL in a production environment. Hands-on experience with modern data engineering frameworks such as: Apache Spark, Databricks Airflow/Prefect/Dagster Kafka/EventHub/Kinesis Experience building data solutions on at least one major cloud platform ...

Data Engineer - Databricks

Hiring Organisation
TXP Technology x People
Location
London, South East, England, United Kingdom
Employment Type
Contractor
Contract Rate
£0 per annum
downstream data services across the organisation. Key Responsibilities Design, build, and maintain data pipelines using Databricks Develop ETL/ELT processes with PySpark and Spark SQL Transform and model structured and semi-structured datasets Improve performance, reliability, and cost efficiency of data workloads Ensure compliance with data governance, security … practice routes Required Skills & Experience Active SC Clearance - mandatory and non-negotiable Strong hands-on experience with Databricks in production environments Expert knowledge of Apache Spark , including PySpark and Spark SQL Proficiency with Python for data engineering Experience delivering solutions on cloud data platforms ( AWS preferred , Azure ...

Senior Data Engineer

Hiring Organisation
Randstad Digital
Location
City of London, London, United Kingdom
logic into high-quality code using Python, Scala, or Java. Innovation: Evaluate and implement new technologies across Cloud (Azure, AWS, GCP) and Big Data (Spark, Kafka) stacks. Collaboration: Work closely with business teams to define data flows and ensure security protocols (encryption/authentication) are met. Technical Toolkit Languages ...

Data Engineer - Apache Airflow

Hiring Organisation
Norton Blake
Location
Greater London, England, United Kingdom
data platform and engineering capability as the business continues to scale its trading operations. They are looking for a Data Engineer with strong Apache Airflow experience to help design and build the pipelines that power trading, analytics and commercial decision-making. This is a great opportunity to join … platform and the opportunity to work with modern open-source technologies . What You’ll Be Doing Designing and building data pipelines orchestrated with Apache Airflow Developing ETL/ELT workflows in Python and SQL to process trading and operational data Building reliable, scalable data infrastructure to support analytics ...

SENIOR CLOUD ENGINEER / AI ENGINEER (CONTRACT)

Hiring Organisation
Secure Recruitment Ltd
Location
North London, London, United Kingdom
Employment Type
Contract, Work From Home
Contract Rate
Up to £1,200 per day
Production Hands-On Experience with Deep Learning Frameworks (TensorFlow, PyTorch, XGBoost) Deep Knowledge of Data Warehousing, ETL/ELT Architectures & Heavy Analytic Environments (Apache Beam, Hadoop, Spark, Hive) Responsibilities: Leading Customer-Facing AI Deployments across Europe, from the Initial Consulting Phase right through to the Live Implementation … Production ML Systems Architecting & Optimising Scalable Data Pipelines, ETL/ELT Solutions & Distributed ML Environments (using Hadoop, Spark & Apache Beam). Deploying Recommendation Engines & Deep Learning Models (TensorFlow, PyTorch) to solve specific Client Problems. Running Technical Design Sessions, Tailoring ML Ops Architectures using Vertex AI & Gen AI. Writing ...

Data Engineer

Hiring Organisation
Youngs Employment Services
Location
Hammersmith and Fulham, Greater London, Notting Barns, United Kingdom
Employment Type
Permanent
Salary
£60000 - £70000/annum Excellent Benefits
Responsibilities will include; Design, implement, and optimize end-to-end solutions using Fabric components: o Data Factory (pipelines, orchestration) o Data Engineering (Lakehouse, notebooks, Apache Spark) o Data Warehouse (SQL endpoints, schemas, MPP performance tuning) o Real-Time Analytics (KQL databases, event ingestion) o Manage and enhance OneLake ...

Lead Data Engineer - Databricks

Hiring Organisation
ARCA Resourcing Ltd
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
£62,000 - £74,000 per annum
quality and reliable deliverables. Optionally contribute to on-call platform support where required. What We’re Looking For Technical Expertise Strong experience with Databricks, Apache Spark, and cloud-based data architectures (preferably Azure). Excellent programming capability in Python and SQL. Experience orchestrating pipelines with Azure Data Factory ...

Contract Azure Data Engineer - £600/pd Outside IR35

Hiring Organisation
Tenth Revolution Group
Location
London, South East, England, United Kingdom
Employment Type
Contractor
Contract Rate
£550 - £600 per day
/CD (Azure DevOps/GitHub). Excellent communication with technical and non-technical stakeholders. Desired Skills Power BI/Tableau Stakeholder management Spark/Hadoop Unit testing frameworks Azure advanced analytics Data warehousing experience Azure Data Engineer certification Agile/Scrum/Kanban exposure This is a London ...

Python Developer

Hiring Organisation
Radley James
Location
City of London, London, United Kingdom
commodities markets or broader financial markets. Knowledge of quantitative modeling, risk management, or algorithmic trading. Familiarity with big data technologies like Kafka, Hadoop, Spark, or similar. Why Work With Us? Impactful Work: Directly influence the profitability of the business by building technology that drives trading decisions. Innovative Culture ...

AI Engineer - all seniorities

Hiring Organisation
Datatech Analytics
Location
London Area, United Kingdom
data pipelines • Engineering ML systems on modern cloud platforms • Working with architects and data engineers on enterprise AI platforms Technology • Python • SQL • Spark/PySpark • TensorFlow, PyTorch or similar frameworks • AWS, Azure or GCP • Databricks or Snowflake • Exposure to GenAI or LLMs Background We’re interested in engineers with ...

Data Architect - SC Cleared

Hiring Organisation
Sanderson Government and Defence
Location
London, United Kingdom
Employment Type
Contract
Contract Rate
£700 - £750 per day
able to engage technical and non-technical stakeholders Nice to Have: Experience with modern analytics and BI platforms Knowledge of big data technologies (e.g. Spark, Hadoop) Experience with real-time or event-driven architectures Familiarity with data cataloguing and lineage tools Experience working in regulated or data-sensitive environments ...

Data Delivery Lead

Hiring Organisation
Coforge
Location
Greater London, England, United Kingdom
uplift practices and delivery maturity. Required Technical Skills & Expertise AWS Data & Analytics (Hands-On Expertise) AWS Glue (ETL/ELT development, PySpark, Glue Catalog) Apache Spark AWS Redshift (modelling, optimisation) AWS Lambda AWS RDS AWS Step Functions Experience with Open Table Formats: Delta Lake Apache Hudi Strong ...

Data Engineer

Hiring Organisation
Searchability NS&D
Location
City of London, London, United Kingdom
security controls. Monitor, maintain and optimise data platforms to ensure performance and reliability. Skills and Experience Required: ETL development using Python or similar languages Apache Spark, NiFi, or Kafka Relational (PostgreSQL, MySQL) and NoSQL (MongoDB, Cassandra) databases AWS and cloud-based data services Palantir Foundry Stakeholder Engagement ...

Senior Data Engineer

Hiring Organisation
PRIMA Partners Global
Location
City of London, London, United Kingdom
integration and API development (REST, SOAP) Informatica or Oracle Data Integrator SQL Server Integration Services (SSIS) Azure Data Factory or Azure Synapse Databricks or Apache Spark Amazon Redshift SQL Server or Oracle Database Oracle Integration Cloud or Business Objects Data Services Desirable Skills: Business intelligence tools such ...

Data Scientist

Hiring Organisation
Searchability NS&D
Location
City of London, London, United Kingdom
preparation. Machine learning and statistical methods, including model validation. Data analytics and visualisation techniques. Processing large datasets using batch or streaming tools such as Apache Spark. Data acquisition and fusion approaches. Cloud platforms (ideally AWS) and cloud-based data science solutions. DataOps principles and practices. Structured and unstructured databases. ...

Jr Machine Learning Engineer

Hiring Organisation
Information Tech Consultants
Location
Greater London, England, United Kingdom
extracting, manipulating, and preparing data from relational databases. Experience with NoSQL databases is a plus. Big Data/Cloud: Experience with big data tools ( Spark, Hadoop ) and cloud computing platforms ( AWS, Azure, or GCP ) for scalable ML workflows. Visualization: Ability to create clear, compelling data visualizations using tools like ...

Data Scientist - UKIC DV Clearance Required

Hiring Organisation
Matchtech
Location
London, United Kingdom
Employment Type
Contract
Contract Rate
GBP Annual
statistical techniques, including validation. Understanding of data analytics and data visualisation techniques. Ability to process large datasets via batch or stream processing using Apache Spark or similar tools. Exposure to techniques used for acquiring and fusing data. Experience with cloud platforms (preferably AWS) or implementing cloud-based data ...

Lead Data Engineer

Hiring Organisation
Canada Life
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
Salary negotiable
influence within the team and communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous). Experience contributing … Engineer or Solutions Architect certification (desirable). Databricks Certified Data Engineer or Machine Learning Associate (desirable). Experience with streaming solutions (Kafka, Event Hubs, Spark Streaming) (desirable). Knowledge of machine learning and AI on Databricks (desirable). ...

Lead Data Engineer

Hiring Organisation
Canada Life UK
Location
City of London, London, United Kingdom
Employment Type
Part Time
influence within the team and communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous). Experience contributing … Engineer or Solutions Architect certification (desirable). Databricks Certified Data Engineer or Machine Learning Associate (desirable). Experience with streaming solutions (Kafka, Event Hubs, Spark Streaming) (desirable). Knowledge of machine learning and AI on Databricks (desirable). Benefits of working at Canada Life We believe in recognising ...

Data Engineer

Hiring Organisation
Hexegic
Location
London Area, United Kingdom
data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

Senior Solutions Architect

Hiring Organisation
HCLTech
Location
City of London, London, United Kingdom
RNNs, Transformers) and when to apply them Strong SQL and data manipulation capabilities Experience working with large-scale datasets and distributed compute frameworks (e.g., Spark) Demonstrated ability to independently lead technical ML solution design Experience working in client-facing delivery environments Exposure to cloud-based ML platforms (AWS, Azure ...

Data Scientist

Hiring Organisation
Formula Recruitment
Location
City of London, London, United Kingdom
data science or statistical role. Expertise in advanced analytics techniques including machine learning, optimisation and segmentation. Strong proficiency in Python, SQL and Spark SQL. Experience working with large datasets and distributed computing tools such as Azure Synapse and Snowflake. Familiarity with data transformation tools (Databricks, Azure Data Factory ...