76 to 100 of 143 PySpark Jobs in the UK

Technical Architect - SC Cleared

Hiring Organisation
TrueNorth®
Location
England, United Kingdom
Strong experience designing modern cloud-based data platforms. Hands-on architectural experience with: AWS (essential): S3, EMR, Glue Kafka/event streaming architectures Python & PySpark-based data processing Experience designing data ingestion pipelines (batch and real-time). Proficiency in Infrastructure as Code (Terraform). Experience with GitHub-based ...

AI Solution Architect

Hiring Organisation
Randstad Digital
Location
London, United Kingdom
Employment Type
Contract
Contract Rate
£500 - £550 per day
processing Handson in Prompt Engineering Azure Open AI Form Recognizer Cognitive Search Vector Databases Develop and deliver upskilling sessions to the customer Python and PySpark Randstad Technologies is acting as an Employment Business in relation to this vacancy. ...

CV Technical Pricing Manager

Hiring Organisation
Vermelo
Location
Haywards Heath, West Sussex, South East, United Kingdom
Employment Type
Permanent
differences between different packages for some of these model types would be an advantage. Experience in statistical and data science programming languages (e.g. Python, PySpark, R, SAS, SQL) Experience of WTW's Radar software is preferred. Proficient at communicating results in a concise manner both verbally and written. ...

Data Engineering Lead

Hiring Organisation
ECS
Location
Newark, Nottinghamshire, East Midlands, United Kingdom
Employment Type
Permanent, Work From Home
Salary
£90,000
frameworks Engineering Delivery Design and evolve scalable data pipelines (batch and incremental watermark-based ingestion) using: Microsoft Fabric Pipelines Azure Data Factory PySpark/Python Fabric Notebooks Implement and maintain a medallion lakehouse architecture using OneLake and Fabric Warehouses Ensure Direct Lake-ready semantic models for analytics Lead incident … Hands-on experience with Microsoft Fabric, including: Workspaces, OneLake, Fabric Warehouses, Fabric Notebooks and Data Pipelines and Direct Lake semantic models Strong Python and PySpark development capability Solid SQL skills for warehouse development and metadata layers Proven ownership of production data systems end-to-end Wider Azure data platform ...

Azure Data Engineer

Hiring Organisation
Emporia Consulting Group Limited
Location
South West London, London, United Kingdom
Employment Type
Permanent, Work From Home
A leading financial services company is hiring an Azure Data Engineer who has extensive Azure, ADLS, ADF, MS Fabric, CI/CD. Hybrid position based in London. Paying up to £100,000 per annum. Experience ...

Sr Python Developer – Trading / Risk Platforms

Hiring Organisation
Crisil
Location
City of London, London, United Kingdom
/PnL/Trade lifecycle/Reconciliation Understanding of financial products such as: Derivatives (swaps, futures, options), bonds Good-to-Have Exposure to PySpark or large-scale data processing Experience working with quants or pricing models Basic scripting (Unix/Bash) Prior experience in global banking environments Experience ...

Data Engineer

Hiring Organisation
Gold Group
Location
City, London, United Kingdom
Employment Type
Permanent
Salary
GBP 50,000 - 65,000 Annual
ownership of data pipelines and automation of data workflows. Finally have 5 years Strong programming experience at an advanced level in Python, PySpark, and SQL scripting. Benefits Salary: 50,000 - 65,000 per annum 25 day's holiday Company car/Allowance Variable annual bonus based 5-15% Pension … leveraging tools such as Foundry Code Workbooks, Pipelines, and Object Explorer for efficient data processing and integration. Strong coding skills in Python (Pandas) and PySpark: Essential experience in using Python (Pandas) and PySpark for data manipulation, transformation, and pipeline development within Palantir Foundry, ensuring high performance and scalability ...

Data Engineer

Hiring Organisation
Searchability®
Location
Greater Manchester, England, United Kingdom
Collaborative office environment with regular social and charity events And Much More!!! DATA ENGINEER – ESSTENTIAL SKILLS Strong experience with Databricks and Apache Spark (PySpark or Scala) Experience building streaming and batch pipelines using Spark Structured Streaming Hands-on experience with Kafka (MSK) and real-time data ingestion Strong understanding … submit (subject to required skills) your application to our client in conjunction with this vacancy only. KEY SKILLS Data Engineering, Databricks, Apache Spark, PySpark, SQL, Python, Kafka, AWS, Delta Lake, ETL, Data Pipelines, CI/CD, Big Data ...

Data Engineer

Hiring Organisation
Tenth Revolution Group
Location
London Area, United Kingdom
Azure or AWS Developing modern lakehouse architectures using: Databricks (Spark, SQL, Delta Lake) Azure Data Lake Storage or Amazon S3 Writing high-quality PySpark, Python, and SQL for ingestion, transformation, and modelling Implementing Bronze/Silver/Gold data layers Embedding data quality, validation and monitoring within pipelines Supporting … Data Engineering roles Strong experience working with Azure or AWS cloud platforms Commercial, hands-on experience using Databricks in production Proven capability across: PySpark/Python/SQL Cloud data storage (ADLS or S3) ETL/ELT pipeline development Solid understanding of data architecture, modelling, and governance Experience building ...

Data Engineer

Hiring Organisation
Gleeson Recruitment Group
Location
Birmingham, West Midlands, England, United Kingdom
Employment Type
Full-Time
Salary
£65,000 - £75,000 per annum
scalable, high-quality data platform from the ground up. The Role Design and build scalable ETL/ELT pipelines in Microsoft Fabric using PySpark and SQL Develop transformation logic within Fabric Lakehouses using Delta Lake and medallion architecture principles Integrate data from ecommerce, ticketing, and marketing platforms into … best practices in data engineering, documentation, and version control Requirements Strong hands-on experience with Microsoft Fabric (Lakehouse, Warehouse, pipelines) Advanced SQL and strong PySpark experience Proficiency in Python for data processing and API integration Experience building Power BI models and dashboards for business reporting Strong understanding ...

Cloud Data Architect - AWS

Hiring Organisation
Quantiphi
Location
United Kingdom
data loss. Enable CDC-based replication and minimal downtime migrations across heterogeneous systems. Design and optimize ETL/data pipelines using AWS Glue, PySpark, or EMR. Architect and implement scalable, secure, and highly available data platforms on AWS. Drive schema conversion and modernization strategies (heterogeneous migrations where applicable). … cloud or cloud to cloud). Experience designing and implementing data validation and reconciliation frameworks. Strong experience in ETL/data pipeline development (Glue, PySpark, EMR, or equivalent). Deep proficiency in SQL and data modeling concepts. Experience working with S3-based data lakes and Glue Data Catalog. Strong ...

Cloud Data Architect

Hiring Organisation
Quantiphi
Location
United Kingdom
data loss Enable CDC-based replication and minimal downtime migrations across heterogeneous systems Design and optimize ETL/data pipelines using AWS Glue, PySpark, or EMR Architect and implement scalable, secure, and highly available data platforms on AWS Drive schema conversion and modernization strategies (heterogeneous migrations where applicable) Optimize … data integration), Amazon Redshift (data warehousing) Experience designing and implementing data validation and reconciliation frameworks Strong experience in ETL/data pipeline development (Glue, PySpark, EMR, or equivalent) Deep proficiency in SQL and data modeling concepts Experience working with S3-based data lakes and Glue Data Catalog Strong understanding ...

Databricks SC Cleared Data Engineer

Hiring Organisation
IO Associates
Location
London, United Kingdom
Employment Type
Permanent, Work From Home
Salary
£550 - £650 per day
national scale. You must already hold active SC clearance to be considered. Responsibilities Design, build and maintain Databricks data pipelines using Apache Spark (PySpark & Spark SQL) Implement Delta Lake/Lakehouse architectures (Bronze, Silver, Gold) Develop scalable ELT/ETL workflows for batch and streaming data Optimise performance, reliability … delivery environment Contribute to CI/CD pipelines and automated testing practices Essential Skills & Experience Proven commercial experience as a Databricks Data Engineer Strong PySpark and SQL capability Hands-on delivery within AWS, Azure or GCP Experience with Delta Lake , data lakes and data warehouses Strong understanding of data ...

GenAI Data Engineer

Hiring Organisation
Stott & May Professional Search Limited
Location
London, United Kingdom
Employment Type
Contract
Contract Rate
£500 - £525 per day
focus on designing scalable data platforms and integrating Generative AI capabilities into enterprise systems. Key Responsibilities Design, build and maintain scalable data pipelines using PySpark, Python and distributed computing frameworks Architect and optimise AWS-based data and AI infrastructure for secure, high-performance data processing Develop, fine-tune, benchmark … functional teams to integrate GenAI solutions into production environments Ensure data quality, governance and operational reliability across systems Essential Skills & Experience Strong experience with PySpark and large-scale distributed data processing (ETL/ELT pipelines) Advanced SQL expertise, including schema design (star/snowflake), indexing and optimisation techniques Proven ...

Data Engineer

Hiring Organisation
Hays
Location
Newport, Gwent, Wales, United Kingdom
Employment Type
Permanent, Work From Home
Salary
£55,000
accurate documentation, lineage and operational run books to support ongoing platform support Key Responsibilities Develop and maintain data pipelines using Azure Databricks (Python, PySpark and SQL) Implement ingestion and transformation across Bronze, Silver and Gold layers using Delta Lake Support and operate Azure Databricks and Azure Synapse platforms, including … pipelines and code reviews What you'll need to succeed Essential Proven experience as a Data Engineer Strong hands-on experience with AzureDatabricks, Python, PySpark and SQL Experience supporting Azure data platforms in production Understanding of medallion architecture and Delta Lake Desirable Experience with Azure Synapse Analytics Experience with ...

Senior Data Engineer

Hiring Organisation
Novatus Global
Location
London Area, United Kingdom
Novatus Global is a Series B scale-up RegTech SaaS provider and boutique advisory firm, helping financial institutions manage their most complex regulatory requirements. We combine deep consulting expertise with cutting-edge SaaS solutions, enabling ...

Lead Data Platform Engineer - Databricks - IAC - Terraform - Azure Data Factory - Data Lakehouse

Hiring Organisation
Nexere Consulting Limited
Location
London, United Kingdom
Employment Type
Permanent
Salary
GBP 80,000 - 85,000 Annual
solutions, producing technical documentation and securing approvals through governance bodies such as Architecture Review Boards. Build and deliver robust data solutions using Databricks, PySpark, Spark SQL, Azure Data Factory, and Azure services. Develop APIs and write efficient Python, PySpark, and SQL code to support data integration, processing … Data Engineering, or a related field. Proven experience designing and building cloud-based data platforms, ideally within Azure. Strong hands-on expertise with Databricks, PySpark, Spark SQL, and Azure Data Factory. Solid understanding of Data Lakehouse architecture and modern data platform design. Proficiency in Python for data engineering, automation ...

Data Engineer

Hiring Organisation
Tenth Revolution Group
Location
City of London, London, United Kingdom
properly, you’ll fit right in. You’ll Work With Azure Data Factory, Data Lake, Key Vault, Azure Functions Databricks, Delta Lake, Unity Catalog, PySpark, Spark SQL Medallion architecture, Lakehouse patterns, distributed compute CI/CD pipelines using Azure DevOps Cross‐functional teams in underwriting, actuarial, delegated authority, exposure … have: Strong hands‐on experience with Azure Data Services and Databricks Solid understanding of Delta Lake, Medallion architecture and Lakehouse patterns Proficiency in Python, PySpark and Spark SQL Experience building CI/CD pipelines using Azure DevOps Knowledge of data governance, lineage, access controls and FinOps Confident communication skills ...

Senior FullStack Developer

Hiring Organisation
Adecco
Location
City of London, London, United Kingdom
Employment Type
Contract
Contract Rate
£600 - £650/day
native architectures, with experience in the financial sector. Experience with CI/CD , microservices , and event-driven architectures is essential, and knowledge of Python (pyspark) for data-driven workflows is advantageous Key Responsibilities Design, develop, and maintain full stack applications using React JS , C#.NET Core , and supporting technologies. Build … C#, REST APIs, Web APIs, and microservices-based development. Experience in transforming monolithic applications into composable microservices, or modular monoliths is a plus. Python (pyspark): Some hands-on experience with distributed data processing, data pipelines, and analytic transformations. Power BI: Some experience developing dashboards, data models, DAX calculations ...

Senior FullStack Developer

Hiring Organisation
Adecco
Location
London, South East, England, United Kingdom
Employment Type
Contractor
Contract Rate
£600 - £650 per day
native architectures, with experience in the financial sector. Experience with CI/CD , microservices , and event-driven architectures is essential, and knowledge of Python (pyspark) for data-driven workflows is advantageous Key Responsibilities Design, develop, and maintain full stack applications using React JS , C#.NET Core , and supporting technologies. Build … C#, REST APIs, Web APIs, and microservices-based development. Experience in transforming monolithic applications into composable microservices, or modular monoliths is a plus. Python (pyspark): Some hands-on experience with distributed data processing, data pipelines, and analytic transformations. Power BI: Some experience developing dashboards, data models, DAX calculations ...

Data Engineering Lead

Hiring Organisation
Novatus Global
Location
City of London, London, United Kingdom
Novatus Global is a Series B scale-up RegTech SaaS provider and boutique advisory firm, helping financial institutions manage their most complex regulatory requirements. We combine deep consulting expertise with cutting-edge SaaS solutions, enabling ...

Forward Deployed Engineer - Databricks

Hiring Organisation
DecisionForest
Location
United Kingdom
code, documenting patterns and helping customers adopt Databricks technologies. Responsibilities Embed with customer teams to deliver production Databricks solutions. Build data pipelines using Databricks, PySpark, SQL, Delta Lake, and Databricks Workflows. Design and implement Lakehouse architecture patterns across bronze, silver and gold layers. Implement Unity Catalog structures, including catalogs … analytics engineering or technical consulting. Demonstrable experience owning at least 2 end-to-end Databricks implementations in a production environment. Strong production experience with PySpark and SQL. Experience with Databricks Lakeflow, notebooks, repos, SQL Warehouses, and cluster/serverless compute patterns. Solid understanding of distributed computing concepts (partitioning, shuffling ...

SAP Data Architect

Hiring Organisation
HCLTech
Location
Greater London, England, United Kingdom
Role Overview The role is responsible for designing, building & maintaining robust data architectures that integrate SAP and non-SAP systems to support enterprise-wide data management, analytics, and reporting needs. This role ensures seamless data ...

Lead Platform Engineer

Hiring Organisation
Synapri
Location
London, United Kingdom
Employment Type
Permanent
Salary
£75000 - £85000/annum
solutions, producing technical documentation and securing approvals through governance bodies such as Architecture Review Boards. Build and deliver robust data solutions using Databricks, PySpark, Spark SQL, Azure Data Factory, and Azure services. Develop APIs and write efficient Python, PySpark, and SQL code to support data integration, processing … Data Engineering, or a related field. Proven experience designing and building cloud-based data platforms, ideally within Azure. Strong hands-on expertise with Databricks, PySpark, Spark SQL, and Azure Data Factory. Solid understanding of Data Lakehouse architecture and modern data platform design. Proficiency in Python for data engineering, automation ...

Senior Data Engineer - Microsoft Fabric

Hiring Organisation
Roc Search Europe Limited
Location
Leeds, West Yorkshire, United Kingdom
Employment Type
Permanent
Salary
£65000 - £70000/annum Excellent Benefits & Holidays
maintain ETL/ELT pipelines and data models in Fabric (Data Factory, Notebooks, Spark) Write high-performance Spark SQL, T-SQL, Python/PySpark Manage ingestion, transformation, and loading from multiple sources Translate stakeholder requirements into scalable technical solutions Mentor team members and establish engineering standards, security, and governance … Leverage AI-assisted development tools like GitHub Copilot, ChatGPT, and Fabric Copilot Essential Experience: Microsoft Fabric & Azure Data ecosystem Lakehouse architectures & Data Factory Python, PySpark, Spark SQL Proven hands-on delivery in this stack What's on Offer: Salary: £70,000 Excellent benefits & annual leave package Strong progression & development ...