1 to 25 of 102 PySpark Jobs

Senior Data Engineer

Hiring Organisation
83zero Limited
Location
London, United Kingdom
Employment Type
Permanent
Salary
£80,000
contribute to team capability building Technical Stack Core: Azure Data Factory Azure Synapse Analytics Azure Data Lake Storage Gen2 SQL Server Databricks Enhancements: Python (PySpark, Pandas) CI/CD (Azure DevOps) Infrastructure as Code (Terraform, Bicep) REST APIs GitHub Actions Desirable: Microsoft Fabric Delta Live Tables Power BI dataset ...

Data Engineer

Hiring Organisation
Peregrine
Location
United Kingdom
Employment Type
Permanent
Criteria Extensive experience with Azure services including Azure Databricks, Azure Data Lake Storage, and Azure Data Factory. Advanced proficiency in SQL, Python, and Spark (PySpark), with a strong focus on performance optimization and distributed processing. Proven experience in CI/CD practices using industry-standard tools (e.g., GitHub Actions ...

Senior Data Engineer - £62,000 - Hybrid

Hiring Organisation
Hays Specialist Recruitment Limited
Location
Bridgend, Mid Glamorgan, Wales, United Kingdom
Employment Type
Full-Time
Salary
£55,000 - £62,000 per annum
system implementations improve service quality and accessibility for customers; What you'll need to succeed Knowledge/Qualifications - Expert-level SQL and Python (ideally PySpark).- A strong understanding of DataSecOps; experience with PII handling, data obfuscation, and RBAC (Role-Based Access Control).- Proven ability to design data ...

Data Engineer - SC Cleared

Hiring Organisation
VIQU IT Recruitment
Location
London, South East, England, United Kingdom
Employment Type
Contractor
Contract Rate
£550 - £650 per day
needed: • Extensive experience with Azure services including Azure Databricks, Azure Data Lake Storage, and Azure Data Factory.• Advanced proficiency in SQL, Python, and Spark (PySpark), with a strong focus on performance optimization and distributed processing.• Proven experience in CI/CD practices using industry-standard tools (e.g., GitHub Actions ...

Junior Data Engineer

Hiring Organisation
Chambers and Partners
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
Competitive salary
Strong proficiency in SQL , including Spark SQL and MS SQL Server , for querying, data manipulation, and performance optimization. Hands-on experience with Python and PySpark for data processing, transformation, and automation tasks. Proven ability to design, build, and maintain scalable data pipelines for batch and streaming data using modern ...

Data Architect

Hiring Organisation
IO Associates
Location
London, United Kingdom
Employment Type
Permanent
cost estimation, effort sizing, and client presentations Support technical deal qualification and proposal development Deliver scalable data pipelines using Databricks, Spark, Delta Lake, and PySpark Own end-to-end architecture decisions across delivery programmes Work closely with consulting, commercial, and engineering teams Required Experience Strong recent hands-on Databricks ...

Databricks Architect

Hiring Organisation
Tenth Revolution Group
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
£100,000 - £125,000 per annum
working with flexibility and autonomy Key Experience Deep end-to-end experience delivering enterprise Databricks analytics solutions Expertise in Lakehouse Architecture, Delta Lake, Spark, PySpark Strong data modelling skills (3NF, Kimball, Data Vault) Experience exposing Delta Lakehouse to Power BI or Tableau Skilled in Python, Scala ...

Databricks Architect

Hiring Organisation
Tenth Revolution Group
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
£100,000 - £120,000 per annum
Experience* Proven experience delivering enterprise-scale Databricks solutions end-to-end.* Strong background in Lakehouse Architecture, including structured and unstructured data.* Expertise in Spark, PySpark, Delta Lake, and Databricks workflows.* Experience building scalable ETL/ELT pipelines, including Delta Live Tables.* Strong programming skills in Python, Scala, or SQL. ...

Databricks Architect

Hiring Organisation
Tenth Revolution Group
Location
Edinburgh, Midlothian, Scotland, United Kingdom
Employment Type
Full-Time
Salary
£110,000 - £120,000 per annum
Experience* Proven experience delivering enterprise-scale Databricks solutions end-to-end.* Strong background in Lakehouse Architecture, including structured and unstructured data.* Expertise in Spark, PySpark, Delta Lake, and Databricks workflows.* Experience building scalable ETL/ELT pipelines, including Delta Live Tables.* Strong programming skills in Python, Scala, or SQL. ...

Data Engineer

Hiring Organisation
TXP
Location
London, United Kingdom
Employment Type
Contract
Contract Rate
£550 - £600/day
Demonstrable experience building data pipelines in production environments. Proficiency with Microsoft Fabric, including Data Factory, lakehouses, warehouses, notebooks and semantic models. Strong skills in PySpark, Spark SQL and T-SQL for data transformation. Experience with medallion architecture or equivalent layered data processing patterns. Working knowledge of OneLake, delta tables ...

Data Engineer TV Advertising Data (FAST)

Hiring Organisation
Datatech Analytics
Location
London, United Kingdom
Employment Type
Permanent
Salary
£85,000
Engineer or in a similar role Proven experience with cloud-based data platforms (Azure, AWS, SQL, Snowflake, Springserv); Strong proficiency in Spark SQL and PySpark , including complex transformations Experience building ETL/ELT pipelines using tools such as Azure Data Factory or equivalent Ability to write efficient, reusable scripts ...

Data Architect

Hiring Organisation
Damia Group Ltd
Location
Chiswick, London, United Kingdom
Employment Type
Permanent
Salary
£70000 - £90000/annum
Proficiency in AWS Glue, Qlik Talend, DBT, Airflow, and modern data integration tools. Excellent knowledge of Python, SQL, PowerQuery (M), and preferably Scala or PySpark Working knowledge with enterprise architecture frameworks (e.g., TOGAF), MLOps, and BI tools like Power BI and QuickSight Experience of generative AI platforms (e.g., Amazon ...

Data Engineer

Hiring Organisation
Artis Recruitment
Location
Bristol, Avon, England, United Kingdom
Employment Type
Full-Time
Salary
£50,000 - £55,000 per annum
concept initiatives About You To be considered, you should have: Proven experience in a Data Engineering role Strong proficiency in SQL, Python and Spark (PySpark) Experience designing data models and implementing reliable pipeline patterns Knowledge of orchestration tools such as Airflow Experience with low-code or no-code integration ...

Data Engineer

Hiring Organisation
Blatchford Ltd
Location
Basingstoke, Hampshire, South East, United Kingdom
Employment Type
Work From Home
with various databases e.g. MS SQL, Azure Cosmos DB. Skilled at optimizing large and more complicated SQL statements. Proficiency in Python and experience with PySpark Experience of REST API standards. Proficiency in writing clean, readable, and secure code for Azure backend services, i.e. functions, utility functions etc. Experience ...

Senior BI Developer

Hiring Organisation
Proactive Appointments
Location
Luton, Bedfordshire, England, United Kingdom
Employment Type
Full-Time
Salary
£50,000 - £75,000 per annum
data workflows using Data Lakes and Logic Apps Write and maintain complex T-SQL queries, stored procedures, and optimised database logic Use Python and PySpark for distributed data processing and large-scale transformation within Fabric and cloud BI pipelines Ensure data accuracy, quality, and adherence to BI governance standards ...

Microsoft Fabric Data Engineer

Hiring Organisation
FBI &TMT
Location
Warrington, Cheshire, North West, United Kingdom
Employment Type
Permanent
modern data platform development. Deep hands-on experience with Microsoft Fabric, including Lakehouse, Data Warehouse, OneLake, Pipelines and Data Factory. Strong SQL, Spark/PySpark and data modelling capability across analytical and operational datasets. Proven experience delivering complex end-to-end data platforms in Azure or Fabric environments. Strong ...

Azure Data Engineer

Hiring Organisation
Opus Recruitment Solutions Ltd
Location
Bristol, Avon, England, United Kingdom
Employment Type
Contractor
Contract Rate
£400 - £500 per day
legacy warehouse assets into a modern Azure environment Contribute to cloud architecture decisions, data standards and best-practice engineering patterns Develop reliable Python and PySpark code to support data ingestion, transformation, and end-to-end processing. What you’ll bring Strong hands-on experience across Azure Data Services ...

Data Engineer Azure Data Platform

Hiring Organisation
Synextra
Location
Birchwood Park, Birchwood, Warrington, Cheshire, England, United Kingdom
Employment Type
Full-Time
Salary
£55,000 - £70,000 per annum
code, with a genuine passion for software engineering in a data context. You'll work across the modern Azure data stack - ADF, ADLS Gen2, PySpark, Delta Lake - with increasing exposure to Microsoft Fabric as the platform matures. You'll collaborate closely with customers and internal teams to ensure data … architectural decisions, and help mentor junior engineers as the team scales. Responsibilities Architect and write production-grade ELT/ETL data pipelines using PySpark and Python within Azure ecosystem. Build custom, reusable data processing frameworks and libraries in Python/Scala to streamline ingestion and transformation tasks across ...

Palantir Foundry Data Engineer X2

Hiring Organisation
TXP
Location
London, United Kingdom
Employment Type
Contract
Contract Rate
£400 - £500/day
experience required Strong experience in data engineering Proven, hands-on experience working with Palantir Foundry in a production environment. Strong proficiency in Python, SQL, PySpark and Spark SQL Experience delivering production pipelines in AWS, Azure, or GCP environments. Solid understanding of data modeling, schema design, metadata management, and governance. ...

Palantir Foundry Data Engineer X2

Hiring Organisation
TXP Technology x People
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
£400 - £500 per day
experience required Strong experience in data engineering Proven, hands-on experience working with Palantir Foundry in a production environment. Strong proficiency in Python, SQL, PySpark and Spark SQL Experience delivering production pipelines in AWS, Azure, or GCP environments. Solid understanding of data modeling, schema design, metadata management, and governance. ...

Data Engineering Manager

Hiring Organisation
McGregor Boyall Associates Limited
Location
Manchester, North West, United Kingdom
Employment Type
Permanent
business needs Ensure data quality, governance and best practice Key Requirements Strong leadership experience (5+ engineers) Solid Data Engineering/DataOps background Python/PySpark for data pipelines AWS experience (design & deployment) Strong data modelling & warehousing knowledge Agile (Jira/Azure DevOps) experience Get in touch for more details ...

Lead Data Engineer

Hiring Organisation
Hays Technology
Location
Kingston Upon Hull, East Riding of Yorkshire, United Kingdom
Employment Type
Permanent
Salary
£70000 - £80000/annum Up to £80,000
Customer Experience, Finance and Operations can trust. Responsibilities Build, optimise and operate ELT/ETL pipelines into our data platform using SQL and Python (PySpark), with a focus on reliability, performance and maintainability. Develop and maintain core data models (curated layers, dimensional models, shared definitions) that enable consistent ...

Senior Data Modeller (London-based)

Hiring Organisation
Itransition
Location
West London, London, United Kingdom
Employment Type
Permanent
Data bricks). Experience with Power BI or other Azure Data Services is a significant advantage. Strong experience with SQL. Familiarity with Python/PySpark is a plus. Deep knowledge of modeling approaches such as Kimball, Inmon, and Data Vault 2.0, with the ability to select the most appropriate ...

Azure Cosmos DB Developer

Hiring Organisation
Stackstudio Digital Ltd
Location
London, United Kingdom
Employment Type
Contract, Work From Home
Contract Rate
From £450 to £500 per day
CDF. High technical proficiency with Cosmos DB SDKs (e.g., .NET, Java, Python, Node.js) Thorough understanding of Spark distributed computing concepts. High technical proficiency in PySpark and Spark Concepts. Experience with concurrency patterns, CLR, and scalable application design. Deep understanding of Azure services including Azure Functions, App Services ...

Technical Architect

Hiring Organisation
Harvey Nash
Location
Chester, Cheshire, England, United Kingdom
Employment Type
Full-Time
Salary
£80,000 - £90,000 per annum
sizing Fabric initiatives, including capacity needs, effort estimates, infrastructure requirements, team resourcing, and delivery timelines. Extensive knowledge in data modelling, DAX, Python/PySpark, SQL/KQL. Proven record of technical leadership and effective stakeholder engagement. Lead architectural design, capacity planning, and performance optimisation efforts. Implement governance, security ...