1 to 25 of 28 PySpark Jobs in the South East

Data Engineer - SC Cleared

Hiring Organisation
VIQU IT Recruitment
Location
London, South East, England, United Kingdom
Employment Type
Contractor
Contract Rate
£550 - £650 per day
needed: • Extensive experience with Azure services including Azure Databricks, Azure Data Lake Storage, and Azure Data Factory.• Advanced proficiency in SQL, Python, and Spark (PySpark), with a strong focus on performance optimization and distributed processing.• Proven experience in CI/CD practices using industry-standard tools (e.g., GitHub Actions ...

Junior Data Engineer

Hiring Organisation
Chambers and Partners
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
Competitive salary
Strong proficiency in SQL , including Spark SQL and MS SQL Server , for querying, data manipulation, and performance optimization. Hands-on experience with Python and PySpark for data processing, transformation, and automation tasks. Proven ability to design, build, and maintain scalable data pipelines for batch and streaming data using modern ...

Databricks Architect

Hiring Organisation
Tenth Revolution Group
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
£100,000 - £125,000 per annum
working with flexibility and autonomy Key Experience Deep end-to-end experience delivering enterprise Databricks analytics solutions Expertise in Lakehouse Architecture, Delta Lake, Spark, PySpark Strong data modelling skills (3NF, Kimball, Data Vault) Experience exposing Delta Lakehouse to Power BI or Tableau Skilled in Python, Scala ...

Senior Data Engineer (Managed Services)

Hiring Organisation
Coeo
Location
Reading, England, United Kingdom
team supporting Azure based Production Data & Analytics ETL/integration solutions (Fabric, Databricks, Synapse, Data Factory, Function Apps, Power BI etc) against SLAs Python, PySpark, Scala, Spark SQL Ability to identify and investigate complex problems and implement resolutions Direct contact with customers/end-users on the phone ...

Data Engineer

Hiring Organisation
Blatchford Ltd
Location
Basingstoke, Hampshire, South East, United Kingdom
Employment Type
Work From Home
with various databases e.g. MS SQL, Azure Cosmos DB. Skilled at optimizing large and more complicated SQL statements. Proficiency in Python and experience with PySpark Experience of REST API standards. Proficiency in writing clean, readable, and secure code for Azure backend services, i.e. functions, utility functions etc. Experience ...

Palantir Foundry Data Engineer X2

Hiring Organisation
TXP Technology x People
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
£400 - £500 per day
experience required Strong experience in data engineering Proven, hands-on experience working with Palantir Foundry in a production environment. Strong proficiency in Python, SQL, PySpark and Spark SQL Experience delivering production pipelines in AWS, Azure, or GCP environments. Solid understanding of data modeling, schema design, metadata management, and governance. ...

Lead Azure Databricks Engineer

Hiring Organisation
Tenth Revolution Group
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
£80,000 - £95,000 per annum
Lloyd's of London or London Market insurance sector Deep understanding of Delta Lake, Medallion architecture, Lakehouse patterns and data modelling. Strong Python, PySpark and Spark SQL capability Solid understanding of data governance Proven track record of ownership, delivery and operating effectively in fast-paced environments. What ...

Senior Data Engineer

Hiring Organisation
IO Associates
Location
South East, United Kingdom
Employment Type
Permanent, Work From Home
Need to Have Hands-on experience with Databricks or Microsoft Fabric Azure Data platform skills (ADF, Synapse, Function Apps, Power BI) Python/PySpark/Scala/Spark SQL Experience in Managed Services or BAU support Customer-facing experience, including ticketing systems Why Apply Mostly remote working Funded certifications ...

Data Delivery Lead

Hiring Organisation
VIQU IT Recruitment
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
£90,000 - £100,000 per annum
Hands-on experience with Microsoft Purview • Strong Azure Data Platform experience: ADLS Gen2, Azure Databricks, Data Factory• Hands-on data engineering experience (Spark/PySpark, Delta Lake)• Knowledge of security and compliance controls, GDPR, PII/DLP, RBAC/ABAC• Experience leading Agile teams and managing risks, dependencies ...

Data Analyst

Hiring Organisation
True North Group
Location
London, South East, England, United Kingdom
Employment Type
Temporary
Salary
£45,000 - £65,000 per annum
Matplotlib) Data visualisation using Power BI or Tableau ideal Familiarity with Agile delivery methodologies Microsoft Fabric Power Apps, Power Automate, Dataverse Apache Spark/PySpark ...

Senior BI Developer

Hiring Organisation
Tenth Revolution Group
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
£65,000 - £80,000 per annum
design and optimisation within Microsoft Fabric Lakehouse, including Data Pipelines, Notebooks, and cloud-based ingestion/transformation layers. Deliver complex SQL, Python, and PySpark solutions for large-scale distributed processing. Coordinate workloads across BI developers and mentor junior and mid-level team members. Ensure BI quality, governance, documentation ...

Data Engineer - SC Cleared

Hiring Organisation
Lorien
Location
London, South East, England, United Kingdom
Employment Type
Contractor
Contract Rate
Competitive salary
scalable, secure data solutions using Azure Databricks, Data Factory and Data Lake Storage. Develop and optimise advanced data pipelines with Python, SQL, Spark/PySpark and Delta Lake. Champion strong data quality, governance and observability practices. Modernise legacy systems and support large-scale migrations into Azure. Provide technical leadership … Criteria Extensive experience with Azure services including Azure Databricks, Azure Data Lake Storage, and Azure Data Factory. Advanced proficiency in SQL, Python, and Spark (PySpark), with a strong focus on performance optimization and distributed processing. Proven experience in CI/CD practices using industry-standard tools (e.g., GitHub Actions ...

Lead Data Engineer

Hiring Organisation
Canada Life
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
Salary negotiable
data modelling for Finance and Enterprise data products, working closely with architecture where required. Implement and maintain data pipelines and ETL workflows in Databricks (PySpark, Delta Lake). Contribute to CI/CD pipelines for data applications using Azure DevOps and infrastructure-as-code (Terraform) in line with established … influence within the team and communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous). Experience contributing ...

Lead Data Engineer

Hiring Organisation
Canada Life UK
Location
Potters Bar, Hertfordshire, South East, United Kingdom
Employment Type
Part Time
data modelling for Finance and Enterprise data products, working closely with architecture where required. Implement and maintain data pipelines and ETL workflows in Databricks (PySpark, Delta Lake). Contribute to CI/CD pipelines for data applications using Azure DevOps and infrastructure-as-code (Terraform) in line with established … influence within the team and communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous). Experience contributing ...

Cloud Data Engineer

Hiring Organisation
Accenture
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
Competitive salary
data for critical client projects. How to develop and enhance your knowledge of agile ways of working and working in open source stack (PySpark/PySql). Quality engineering professionals utilise Accenture delivery assets to plan and implement quality initiatives to ensure solution quality throughout delivery. As a Data ...

Data Engineer

Hiring Organisation
Michael Page Technology
Location
London, South East, England, United Kingdom
Employment Type
Temporary
Salary
Salary negotiable
Implement role-based access, secure data handling, and privacy-aware processing Apply row-level security and dataset-level client isolation Develop SQL, Python, and PySpark scripts for AI/ML feature and model datasets Deliver work through Jira and collaborate with cross-functional teams Use GitHub for version control … data architectures and cost optimisation Advanced SQL and relational data modelling skills Hands-on experience with Snowflake and Microsoft Fabric Proficiency in Python and PySpark, including notebook-based development Experience with workflow orchestration, CI/CD, and GitHub-based delivery Strong knowledge of data security, encryption, and GDPR compliance ...

Data Engineer - Databricks

Hiring Organisation
TXP Technology x People
Location
London, South East, England, United Kingdom
Employment Type
Contractor
Contract Rate
£0 per annum
reporting, and downstream data services across the organisation. Key Responsibilities Design, build, and maintain data pipelines using Databricks Develop ETL/ELT processes with PySpark and Spark SQL Transform and model structured and semi-structured datasets Improve performance, reliability, and cost efficiency of data workloads Ensure compliance with data … Skills & Experience Active SC Clearance - mandatory and non-negotiable Strong hands-on experience with Databricks in production environments Expert knowledge of Apache Spark , including PySpark and Spark SQL Proficiency with Python for data engineering Experience delivering solutions on cloud data platforms ( AWS preferred , Azure acceptable) Understanding of data lake ...

Databricks Architect

Hiring Organisation
Tenth Revolution Group
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
£110,000 - £125,000 per annum
Data Architect - Databricks - Hybrid - Up to £125,000 A leading Microsoft partnered consultancy is seeking an experienced Data Architect to design and deliver modern data solutions built on Databricks Lakehouse Architecture. You'll work with ...

Data Engineer

Hiring Organisation
Tenth Revolution Group
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
£60,000 - £75,000 per annum
maintain scalable ETL/ELT data pipelines within Microsoft Fabric Lakehouse environments Develop and optimise data ingestion/transformation workflows Develop Python and PySpark processes for data transformation and large-scale processing Support the development and optimisation of Power BI datasets, reports and dashboards Build and maintain scalable semantic … data engineering solutions Skills and Experience Hands-on expertise with Microsoft Fabric, Lakehouse architecture and Data Pipelines Experience developing Python or PySpark solutions for large-scale data processing Understanding of ETL/ELT design patterns, data modelling and modern data platform architecture Experience developing Power BI datasets, semantic models ...

Senior BI Developer

Hiring Organisation
Tenth Revolution Group
Location
Southampton, Hampshire, England, United Kingdom
Employment Type
Full-Time
Salary
£60,000 - £75,000 per annum
maintain scalable ETL/ELT data pipelines within Microsoft Fabric Lakehouse environments Develop and optimise data ingestion/transformation workflows Develop Python and PySpark processes for data transformation and large-scale processing Support the development and optimisation of Power BI datasets, reports and dashboards Build and maintain scalable semantic … data engineering solutions Skills and Experience Hands-on expertise with Microsoft Fabric, Lakehouse architecture and Data Pipelines Experience developing Python or PySpark solutions for large-scale data processing Understanding of ETL/ELT design patterns, data modelling and modern data platform architecture Experience developing Power BI datasets, semantic models ...

Senior BI Developer

Hiring Organisation
Tenth Revolution Group
Location
Milton Keynes, Buckinghamshire, England, United Kingdom
Employment Type
Full-Time
Salary
£60,000 - £75,000 per annum
maintain scalable ETL/ELT data pipelines within Microsoft Fabric Lakehouse environments Develop and optimise data ingestion/transformation workflows Develop Python and PySpark processes for data transformation and large-scale processing Support the development and optimisation of Power BI datasets, reports and dashboards Build and maintain scalable semantic … data engineering solutions Skills and Experience Hands-on expertise with Microsoft Fabric, Lakehouse architecture and Data Pipelines Experience developing Python or PySpark solutions for large-scale data processing Understanding of ETL/ELT design patterns, data modelling and modern data platform architecture Experience developing Power BI datasets, semantic models ...

Lead PySpark Engineer

Hiring Organisation
Randstad Technologies
Location
London, South East, England, United Kingdom
Employment Type
Contractor
Contract Rate
£281 - £292 per day
PySpark Engineer Lead As the Technical Lead, you will drive the high-stakes migration of legacy SAS analytics to a modern, cloud-native PySpark ecosystem on AWS. This isn't just a lift and shift you will refactor complex procedural logic into scalable, production-ready distributed pipelines … Tier-1 financial services environment. Core Responsibilities Engineering Leadership: Design and develop complex ETL/ELT pipelines and Data Marts using PySpark, EMR, and Glue. Legacy Modernisation: Architect the conversion of SAS Base/Macros into modular, testable Python code using SAS2PY and manual refactoring. Performance Tuning: Optimise Spark ...

AI Architect

Hiring Organisation
CODEVERSE LIMITED
Location
Marlow, Buckinghamshire, England, United Kingdom
Employment Type
Full-Time
Salary
£70,000 - £80,000 per annum
value from commercial data across Foundry. Key Responsibilities: Architect end-to-end AI solutions using Foundry, including machine learning models, large-scale data pipelines (PySpark), and production-grade deployment architectures within Foundry’s ecosystem. Evaluate and select optimal ML and data technologies, frameworks, and platform components within Foundry … degree in Computer Science, Engineering, or related field. Proven experience as an AI Architect, ML Engineer, or similar role implementing solutions on Foundry, PySpark, and cloud-based big data environments. Deep understanding of AI/ML algorithms, model optimization, and enterprise-grade model deployment (preferably via Foundry). Experience ...

Senior BI Developer

Hiring Organisation
Tenth Revolution Group
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
£60,000 - £75,000 per annum
translating ERP data into scalable BI models Building robust Power BI datasets, reports, and semantic models Delivering complex data transformations using Python and PySpark Writing advanced T-SQL queries and performance-optimised stored procedures Acting as a senior escalation point for BI developers and coordinating workload distribution Ensuring governance … skills across modelling, DAX, optimisation, and dataset design Strong SQL capabilities (T-SQL, performance tuning, stored procedures) Experience with Python and/or PySpark for data transformation Background coordinating or leading BI developers Experience delivering BI in enterprise, multi-region environments Excellent communication skills, analytical mindset, and ability ...

Lead Data Platform Engineer

Hiring Organisation
Harnham - Data & Analytics Recruitment
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
£75,000 - £85,000 per annum
Azure Developing and deploying the Lakehouse platform using Databricks Building automation for cluster management, integration runtimes, and infrastructure components Leading implementation with Databricks, PySpark, Spark SQL, and Azure Data Factory Creating APIs for integration and automation Driving CI/CD, automated testing, and DevOps best practice Your Skills … Experience You will have strong capability in: Azure and Databricks engineering PySpark, SQL, Python, and Spark SQL DevOps best practices including CI/CD, automation, and testing Infrastructure as code such as Terraform Stakeholder communication and cross-team collaboration How to Apply If you are a strong Data Platform ...