101 to 125 of 342 PySpark Jobs in the UK

Lead Data Engineer

Location
City Of London, England, United Kingdom
best practices and driving a culture of continuous improvement. Requirements 8+ years of experience in data engineering with a strong focus on Databricks (SQL & PySpark). Extensive experience with Azure data services, including ADF, ADLS, and Azure Functions. Proven track record of designing and implementing metadata‐driven data pipelines. ...

Data & AI Principal Consultant UK

Location
Greater London, England, United Kingdom
Microsoft Fabric, Databrick, Snowflake, Azure Data Factory, Azure Synapse Analytics, Azure Data Explorer, Azure SQL, Data lakes and lake houses, Azure DevOps, Python and PySpark A strong understanding of data platform architecture, data integration, data processing at scale, analytics, and data governance Experience applying AI/GenAI/Agentic ...

Data Engineer

Location
Manchester, England, United Kingdom
party data providers, delivering reliable and auditable data at scale. Key Responsibilities Build and maintain ingestion and transformation pipelines on Microsoft Fabric (Data Pipelines, PySpark notebooks, Dataflows Gen2) across the Landing, Curate and Enrich layers of the medallion architecture. Onboard new source systems by setting up the team … Snowflake and SQL Server, change data capture, file and API loads). Do this through configuration rather than one-off code. Write and tune PySpark and T-SQL for Lakehouse and Warehouse workloads, including fact and dimension models in the enterprise data warehouse. Implement data quality rules and validation ...

Microsoft Fabric Data Engineer

Location
Greater London, England, United Kingdom
growth across regulated industries.ResponsibilitiesImplement end-to-end Microsoft Fabric solutions based on approved architecture, including OneLake, Lakehouse, Warehouse, Data Factory pipelines, Dataflows Gen2, Notebooks (PySpark/Spark SQL) and Real-Time IntelligenceBuild medallion architectures (Bronze/Silver/Gold), including incremental loads, change data capture, slowly changing dimensions … ingestion from diverse sources (SQL databases, APIs, files, SaaS applications, SAP, streaming sources) using pipelines, shortcuts and mirroringWrite clean, reusable and tested code in PySpark, Spark SQL, T-SQL and KQL, optimizing Delta tablesAdd data quality checks, validation rules, logging and alerting to data pipelinesBuild enterprise semantic models (star ...

Data Engineer

Location
Southampton, England, United Kingdom
from the Head of Data Architecture.Day to day, you’ll:Build and run data pipelines and notebooks in Fabric, using Data Factory, Lakehouse, Python, PySpark and SQLIngest data from our core systems, including Workday, Dynamics 365, Business World and SharePointBuild and maintain data structures in Fabric Warehouse and Lakehouse … minimum criteria you’ll need is practical experience with Microsoft Fabric (Data Factory, Lakehouse, Warehouse and Notebooks) or Databricks, working proficiency in Python, PySpark and SQL for data engineering, an understanding of data modelling, warehousing and ETL/ELT principles, working knowledge of Power BI and version control ...

Data Engineer

Location
Chilworth Old Village, England, United Kingdom
Head of Data Architecture. Day to day, you’ll: Build and run data pipelines and notebooks in Fabric, using Data Factory, Lakehouse, Python, PySpark and SQL Ingest data from our core systems, including Workday, Dynamics 365, Business World and SharePoint Build and maintain data structures in Fabric Warehouse … minimum criteria you’ll need is practical experience with Microsoft Fabric (Data Factory, Lakehouse, Warehouse and Notebooks) or Databricks, working proficiency in Python, PySpark and SQL for data engineering, an understanding of data modelling, warehousing and ETL/ELT principles, working knowledge of Power BI and version control ...

Senior Data Engineer: Python, Cloud, Pipelines & Reporting

Location
West of England, England, United Kingdom
across Financial Services, Fintech, Public Sector and Utilities. You will work on modern data engineering, reporting, and cloud data solutions with Python, Spark, SQL, PySpark and Power BI. The role emphasizes client-facing advisory, Agile delivery, and experience with Snowflake/Databricks in a dynamic data practice. #J ...

Senior Software Engineer (Python)

Location
United Kingdom
/CD workflows, GitLab Experience with Rust and pyO3, or similar low-level languages Experience with SQL database technologies (PostgreSQL, MySQL) Experience with PySpark, Databricks, Delta Lake or other data warehousing technologies Experience with distributed systems engineering, message/event buses such as RabbitMQ, and RESTful API design Understanding ...

Senior Data Engineer — Pipelines, Lakes & AI Enablement (Remote)

Location
West of England, England, United Kingdom
London area will be required, with a flexible hybrid approach that supports remote work. Candidates should have hands-on experience with Python, Spark, SQL, PySpark, and PowerBI, and be comfortable working across AWS/Azure/GCP and Snowflake/Data Bricks. SC clearance eligibility is required. #J ...

Data Scientist

Location
United Kingdom
data analysis Strong Databricks experience including notebooks, modules, jobs, workflows, Delta tables, Unity Catalog data, model registry, and production scoring Hands‐on Spark or PySpark capability for analytical pipelines and large-scale data processing Git and Azure DevOps including branching, pull requests, code review, testing, and controlled deployments Strong ...

Customer Loyalty/ Transaction Lead Data Scientist

Location
United Kingdom
payment, loyalty and transaction analysis. Strong Python for statistical analysis, matching, weighting, cohort analysis and reproducible data products. Hands-on Databricks/Spark or PySpark experience for notebooks, Delta tables, workflows and scheduled analytical pipelines. Working proficiency with Git/Azure DevOps, pull requests and controlled production changes. Strong ...

Senior Data Scientist

Hiring Organisation
Data Careers
Location
South East London, South East, United Kingdom
Employment Type
Permanent Contract, Remote
Salary
£70,000
investigate complex problems and take ownership of technical solutions. Strong communication skills and a collaborative approach. Desirable Experience Data engineering technologies such as Databricks, PySpark or Spark. Cloud platforms, particularly AWS. Python frameworks such as FastAPI and PyTorch. Experience developing APIs and data pipelines. Infrastructure as Code, CI/ ...

Java Lead Software Engineer

Location
Glasgow, Scotland, United Kingdom
services, and scripting and automation using common libraries and frameworks relevant to the role Experience with data engineering and analytics platforms such as PySpark, Databricks, Airflow, or similar orchestration tools Experience with relational databases and SQL; familiarity with NoSQL databases and data modeling concepts Knowledge across the data lifecycle ...

Data Engineer – Databricks – Contract Role

Location
Milton Keynes, England, United Kingdom
optimise pipelines for large-scale data processing. Hands-on experience withDatabricksandAzure. Strong stakeholder communication and problem-solving skills. Tech Stack Required: Databricks DBT Python PySpark SQL Azure Bonus: Experience ineCommerceenvironments. #J-18808-Ljbffr ...

Senior Manager – Palantir Foundry

Hiring Organisation
Staffworx Limited
Location
London, England, United Kingdom
agentic capabilities, able to build production-grade applications not just prototypes.Strong proficiency in at least one relevant programming language such as Python or PySpark, Java, Typescript or SQL.Solid understanding of data engineering, data modelling, security and governance in enterprise settings.Experience with software engineering best practices including Git-based development ...

Data Engineer — Secure Health Data Pipelines

Location
Greater London, England, United Kingdom
experience: Proven experience building production-grade data pipelines and ETL processes. Excellent Python skills with experience developing scalable data solutions. Experience working with PySpark or distributed data processing frameworks. Experience using GitHub and modern software engineering practices. Understanding of cloud-based data platforms, ideally Azure. Experience working with sensitive ...

Senior Data Engineer

Location
Greater London, England, United Kingdom
Lake and Lakehouse architecture for efficient data management; experience with ETL processes and optimizing data pipelines for performance Be strong in Python, SQL, and PySpark Be passionate about building data products for stakeholders and customers, ensuring they are stable, scalable, secure, accurate, observable, and performant Enjoy collaborating closely with ...

Senior Data Scientist / ML Engineer (Forecasting) | NDA

Location
Nottingham, England, United Kingdom
audiences Essential knowledge, skills & experience (must-have): 4+ years of commercial experience in Data Science/Machine Learning Hands-on experience with: Databricks Notebooks PySpark Workflows Deployment through Asset Bundles Proven experience building, deploying, and maintaining production ML solutions Broad experience across multiple ML domains, including: Forecasting/Time ...

Fabric Engineer

Location
Birmingham, England, United Kingdom
Gen2), Purview, Cosmos DB, OpenAI, Azure ML, and AI Foundry Solid understanding of software engineering concepts, with hands‐on experience using Python, Scala or PySpark, data modelling techniques, database technologies, and SQL Confident communicator, able to explain technical concepts to non‐technical audiences and mentor junior colleagues For Senior ...

Fabric Engineer

Location
City Of London, England, United Kingdom
Gen2), Purview, Cosmos DB, OpenAI, Azure ML, and AI Foundry Solid understanding of software engineering concepts, with hands‐on experience using Python, Scala or PySpark, data modelling techniques, database technologies, and SQL Confident communicator, able to explain technical concepts to non‐technical audiences and mentor junior colleagues For Senior ...

Senior AI Engineer

Location
Greater London, England, United Kingdom
What will you need to succeed? Strong production‐level Python engineering experience, with the ability to build maintainable, tested, and supportable services. Experience with PySpark or equivalent large‐scale data processing frameworks. Hands‐on experience designing and operating CI/CD pipelines using Azure DevOps or similar tooling. Experience ...

Senior Data Engineer

Location
Horsham, England, United Kingdom
grow your career while helping customers unlock the power of their data. Responsibilities Design, build and optimise scalable data pipelines using Databricks, Apache Spark, PySpark and Delta Lake. Develop and implement modern Lakehouse architectures, applying Medallion (Bronze, Silver, Gold) design principles to support trusted and reusable data products. Build … ongoing evolution of Envitia’s data engineering capability. Essential Skills & Experience Proven hands‐on experience building and supporting production‐grade data pipelines using Databricks, PySpark, Spark SQL and Delta Lake. Experience designing and implementing Lakehouse architectures, including Medallion (Bronze, Silver, Gold) data models. Expertise in developing scalable ETL/ ...

Senior Data Engineer - Hybrid (Python / Spark / Databricks)

Location
Greater London, England, United Kingdom
building scalable data pipelines, Databricks lakehouse architectures, and robust data platforms in a modern cloud environment. Strong expertise in Python, Databricks, Spark/PySpark and SQL is required, with hands-on delivery across ETL/ELT, data quality and CI/CD practices. Banking domain experience is a plus ...

Data Scientist Lead- Python, ML & Gen AI

Location
Glasgow, Scotland, United Kingdom
Science, Mathematics, or a related field. Applied experience in data science, machine learning, or related areas. Strong programming skills in Python with experience in PySpark, Spark SQL, and manipulating data. Experience LLM Orchestration and Agentic AI libraries Proficiency in using GenAI models (OpenAI or similar) to solve business problems. ...

Senior Developer (Python)

Location
Greater London, England, United Kingdom
chain, content operations, or omnichannel marketing technology Experience with cloud platforms (AWS, Azure, GCP) and containerization (Docker, Kubernetes) Knowledge of data engineering tools: pandas, PySpark, Airflow, or equivalent pipeline frameworks Exposure to ML frameworks or LLM APIs (Anthropic, OpenAI, Hugging Face) for applied use cases Experience in pharmaceutical, life ...