76 to 100 of 342 PySpark Jobs in the UK

Analytics Engineer

Location
Greater London, England, United Kingdom
Comfortable with Git-based workflows and pull requests. Able to work with stakeholders to understand requirements and explain technical trade-offs. Desirable: Python/PySpark, CI/CD for BI, and experience in retail, hospitality or multi-site businesses, experience with Microsoft Fabric or a similar lakehouse platform (Delta ...

Tech Lead Data & AI London

Location
Leeds, England, United Kingdom
Generation), Serverless, DWH models, Data Mesh, Data Fabric, Medallion and Data Products. Hands on mindset : Fluent in the language of data, whether SQL, Python, PySpark, R or Scala. Open formats including Parquet, Iceberg and JSON. Agile Ways of Working Run and contribute to agile ceremonies, helping to shape ...

Senior Data Engineer

Location
City Of London, England, United Kingdom
strong problem solving, communication, and stakeholder management skills. Requirements 4-6 years of experience in data engineering with a strong focus on Databricks (SQL & PySpark). Strong hands-on experience with Azure data services, including ADF, ADLS, and Azure Functions. Proven experience designing and implementing metadata-driven data pipelines. ...

Senior Data Engineer

Location
Greater London, England, United Kingdom
problem solving, communication, and stakeholder management skills.RequirementsCandidates should have:4-6 years of experience in data engineering with a strong focus on Databricks (SQL & PySpark).Strong hands-on experience with Azure data services, including ADF, ADLS, and Azure Functions.Proven experience designing and implementing metadata-driven data pipelines.Strong understanding ...

Junior Data Engineer

Location
Manchester, England, United Kingdom
come from employment, education, an apprenticeship or personal projects. Experience with Microsoft Fabric or another cloud data platform. Familiarity with data warehouses, Lakehouses , PySpark or Delta Lake. Awareness of testing and CI/CD practices. Candidates do not need to meet every desirable criterion to apply. We are proud ...

Data Engineer

Location
Swindon, England, United Kingdom
record of development in Databricks* Proven track record of working with IaC* Proven track record of working with git* Strong working knowledge of Python (Pyspark) and SQL* Strong working knowledge of data management inside a datalake* Proven ability to explain technical issues and solutions to peers and customers* Proven ...

Data Engineer

Location
Swindon, England, United Kingdom
record of development in Databricks* Proven track record of working with IaC* Proven track record of working with git* Strong working knowledge of Python (Pyspark) and SQL* Strong working knowledge of data management inside a datalake* Proven ability to explain technical issues and solutions to peers and customers* Proven ...

Data Engineer

Location
Swindon, England, United Kingdom
record of development in Databricks* Proven track record of working with IaC* Proven track record of working with git* Strong working knowledge of Python (Pyspark) and SQL* Strong working knowledge of data management inside a datalake* Proven ability to explain technical issues and solutions to peers and customers* Proven ...

Data Engineer

Location
Swindon, England, United Kingdom
record of development in Databricks Proven track record of working with IaC Proven track record of working with git Strong working knowledge of Python (Pyspark) and SQL Strong working knowledge of data management inside a datalake Proven ability to explain technical issues and solutions to peers and customers Proven ...

Data & AI Pre-Sales Consultant

Location
East Midlands, England, United Kingdom
/planning context Query language familiarity - T-SQL, Power Query M, DAX - sufficient for scoping conversations and engaging credibly with technical stakeholders; Python or PySpark awareness useful, though the ability to position these skills in a customer team context matters more than hands-on proficiency Awareness of compliance ...

Data Engineer

Location
United Kingdom
data ingestion processes from clinical, operational, financial and external systems, including relational databases, REST APIs and file-based sources. Build and optimise SQL and PySpark transformations across Bronze, Silver and Gold data layers. Work with technologies including Databricks, Delta Lake, Lakeflow, dbt and Unity Catalog. Implement automated data quality … similar data engineering role. Strong SQL skills, including performance tuning, CTEs, window functions and data modelling. Good Python programming skills, particularly PySpark, for data transformation, API integrations and automation. Hands-on experience with Databricks, including Delta Lake, Spark and Unity Catalog, or a comparable cloud data platform such ...

Data Engineer IT · London ·

Location
Greater London, England, United Kingdom
data from D365 F&O, our eCommerce platform and other source systems. Developing and maintaining well-structured data models in OneLake using Delta tables, PySpark and SQL, following a medallion approach and dimensional modelling principles. Working closely with analytics engineers, data analysts, data scientists and business stakeholders to understand … working across areas such as data ingestion, transformation, modelling, data quality and performance optimisation. Strong hands-on proficiency across the core data engineering languages: PySpark : experience building and optimising Spark transformations at scale, ideally in notebook-based environments such as Fabric or Databricks. Python : experience with scripting and automation ...

Senior Data Engineer

Location
Manchester, England, United Kingdom
responsibilities. Ensure solutions are secure, scalable, maintainable, and aligned with organisational standards and governance requirements. Design, build, and optimise data pipelines on Databricks using PySpark and Spark SQL, applying Delta Lake and a medallion/Lakehouse architecture. Implement data validation and reconciliation checks, and automated testing across pipelines … duties as required. Skills and Experience Strong understanding of data engineering principles, data architecture, and cloud-based data platforms. Proficient in Python (PySpark) and SQL for building and optimising data transformations at scale. Strong hands-on Databricks experience. Experience with Azure data storage technologies (Azure SQL, Data Lakes, Blob ...

Senior Data Engineer

Location
Manchester, England, United Kingdom
At UBDS, we help organisations transform through cloud, cyber security, AI and digital innovation. We're looking for an experienced Senior Data Engineer to support the design, development and delivery of modern data solutions within ...

Senior Data Engineer

Location
Greater London, England, United Kingdom
At UBDS, we help organisations transform through cloud, cyber security, AI and digital innovation. We're looking for an experienced Senior Data Engineer to support the design, development and delivery of modern data solutions within ...

Python Developer with Data experience

Hiring Organisation
Hackajob Ltd
Location
South West London, South East, United Kingdom
Employment Type
Permanent Contract, On-site
development of scalable ETL/ELT pipelines. Proven experience working with SQL, relational databases and large datasets. Experience with data processing technologies such as PySpark, Spark or Pandas. Strong understanding of data quality, transformation and integration. Experience with Git, CI/CD and automated testing. Comfortable working in Linux ...

Python Developer with Data experience

Location
Greater London, England, United Kingdom
development of scalable ETL/ELT pipelines. Proven experience working with SQL, relational databases and large datasets. Experience with data processing technologies such as PySpark, Spark or Pandas. Strong understanding of data quality, transformation and integration. Experience with Git, CI/CD and automated testing. Comfortable working in Linux ...

Lead Data Engineer - Perm up to £100k

Hiring Organisation
Lorien Resourcing Limited
Location
Manchester, North West, United Kingdom
Employment Type
Permanent Contract, Hybrid
looking for: Previous experience as a Technical Lead (hands-on) Deep, hands-on AWS data experience: Glue, Lambda, S3, Redshift and EMR Advanced Python, PySpark and SQL Production experience with Kafka or Amazon MSK, including Spark Structured Streaming Hands-on experience with Apache Iceberg or Delta Lake Strong data ...

Senior Solution Architect

Location
Manchester, England, United Kingdom
competitive ETL solutions Experience with value-based sales motion, especially through MEDDPPICC and Command of the Message Knowledge of big data technologies (Spark, PySpark), data lakes, and MPP databases (Teradata, Vertica, Netezza) Familiarity with version control tools such as Git, and experience with Python Degree in Computer Science ...

Senior Specialist Solutions Engineer (Infrastructure & Platform)

Location
Greater London, England, United Kingdom
Azure Resource Manager/Bicep, or Google Cloud Deployment Manager) to build, maintain, and scale complex cloud environments Strong coding proficiency in Python and PySpark/Spark: you must demonstrate live coding, debugging, and solution‐building skills Solid grounding in distributed data systems architecture: scalable pipelines, streaming architectures, lakehouse ...

Data Engineer: Build AI-Ready Data Pipelines on Fabric

Location
Southampton, England, United Kingdom
readiness. The role focuses on secure, reliable data delivery and reusable datasets for analysts. The ideal candidate will have strong skills in Python, PySpark, SQL, and Power BI, with hands-on experience in Fabric or Databricks, and a good grounding in data governance and CI/CD practices. #J ...

Senior Data Engineer – Fabric & AI Platform Lead

Location
Chilworth Old Village, England, United Kingdom
using Fabric components and lead the migration to Microsoft Fabric, ensuring secure, governed data for analytics and AI readiness. You will leverage your Python, PySpark and SQL skills to ingest data from core systems, design robust data models and enable self-service reporting with Power BI. A collaborative, flexible ...

AWS Data Engineer

Location
Basildon, England, United Kingdom
Data Lakehouse solutions AWS services including S3, Glue, EMR, Athena, Redshift, Lake Formation, EventBridge & Step Functions Advanced Python and SQL skills Strong experience with PySpark and large-scale data processing Hands-on experience with dbt or modern data transformation frameworks Data modelling experience (Kimball, Data Vault, Inmon) Strong understanding ...

Data QA - Test Engineer

Location
City Of London, England, United Kingdom
calculation logic (cash flows, commitments, valuations, NAV, and return metrics) across data pipelines. Automated Data Quality Frameworks: Develop automated test suites using Python (PyTest) , PySpark , and advanced SQL to catch discrepancies, nulls, and duplicate records. CI/CD Quality Gates: Integrate automated testing and data validation checks into Azure ...

Databricks Architect

Location
Greater London, England, United Kingdom
Support security hardening and governance requirements appropriate for banking and financial-services environments. Key Requirements Databricks & Lakehouse Strong hands-on experience with: Databricks Notebooks PySpark DataFrame API SQL transformations Delta Lake Databricks Jobs, Job Flows, Workflows Declarative Pipelines (dpx files, pipeline definitions) Experience with Unity Catalog Databricks Autoloader ...