23 of 23 Remote PySpark Jobs in the North of England

the transformation pipeline from start to finish, guaranteeing that datasets are robust, tested, secure, and business-ready. Our data platform is built using Databricks, with data pipelines written in PySpark and orchestrated using Airflow. You will be expected to challenge and improve current transformations, ensuring they meet our performance, scalability, and data governance needs. This includes work with complex … days per year for meaningful collaboration in either Leeds or Thame. Key Responsibilities Ensure end-to-end data quality, from raw ingested data to business-ready datasets Optimise PySpark-based data transformation logic for performance and reliability Build scalable and maintainable pipelines in Databricks and Airflow Implement and uphold GDPR-compliant processes around PII data Collaborate with stakeholders to … management, metadata management, and wider data governance practices Help shape our approach to reliable data delivery for internal and external customers Skills & Experience Required Extensive hands-on experience with PySpark, including performance optimisation Deep working knowledge of Databricks (development, architecture, and operations) Proven experience working with Airflow for orchestration Proven track record in managing and securing PII data, with More ❯

commercial, and operations - and this role will have a big say in what we build next. You'll be responsible for designing and building robust, scalable data pipelines using PySpark, SQL and Databricks - enabling our analytics, BI and data science colleagues to unlock real value across the business. This is a brilliant opportunity for someone who's passionate about … further - especially with tools like Databricks. Here's what will help you thrive in this role: Essential experience: 2-5 years in data engineering or a related field Strong PySpark and advanced SQL skills Practical experience building and maintaining ETL/ELT pipelines in Databricks Familiarity with CI/CD pipelines and version control practices Nice to have: Experience More ❯

experience across full project lifecycles, DevOps implementation, and working with RDBMS (SQL Server, Oracle, MongoDB). Hands-on with tools like Redgate SQL Toolbelt, Visual Studio, Team Foundation Services, PySpark/Python, and Snowflake. Resilient, self-managing, and aligned with UKMU IT’s strategic direction Benefits Our benefits are designed to make health happen for our people. Viva is More ❯

data engineering and reporting. Including storage, data pipelines to ingest and transform data, and querying & reporting of analytical data. You've worked with technologies such as Python, Spark, SQL, Pyspark, PowerBI etc. You’ve got a background in software engineering, including Front End technologies like JavaScript. You’re a problem-solver, pragmatically exploring options and finding effective solutions. An More ❯

Specification Strong SQL knowledge Proven experience working with Azure Git/DevOps Repos experience Demonstration of problem solving ability Synapse Analytics or similar experience - desirable Visual Files experience – desirable PySpark/Python experience – desirable Powershell experience – desirable What We Offer We aim to reward your hard work generously. You’ll be greeted in our offices with great coffee, fruit More ❯

needs Supporting best practices in data governance, quality, and security What We’re Looking For: Strong experience with AWS Glue , S3, Lambda, and related services Proficiency in Python or PySpark for data processing Familiarity with data lake, data warehouse, and cloud-first environments Comfortable working in agile teams and hybrid work settings Location: Hybrid – 2–3 days a week More ❯

security What We're Looking For: Strong experience with AWS Glue , S3, Lambda, and related services Proven track record in data engineering or ETL development Proficiency in Python or PySpark for data processing Familiarity with data lake, data warehouse, and cloud-first environments Comfortable working in agile teams and hybrid work settings Location: Hybrid - 2-3 days a week More ❯

Knowledge of the technical differences between different packages for some of these model types would be an advantage. Experience in statistical and data science programming languages (e.g. R, Python, PySpark, SAS, SQL) A good quantitative degree (Mathematics, Statistics, Engineering, Physics, Computer Science, Actuarial Science) Experience of WTW’s Radar software is preferred Proficient at communicating results in a concise More ❯

Knowledge of the technical differences between different packages for some of these model types would be an advantage. Experience in statistical and data science programming languages (e.g. R, Python, PySpark, SAS, SQL) A good quantitative degree (Mathematics, Statistics, Engineering, Physics, Computer Science, Actuarial Science) Experience of WTW's Radar software is preferred Proficient at communicating results in a concise More ❯

with large-scale datasets using Azure Data Factory (ADF) and Databricks. Strong proficiency in SQL (T-SQL, Spark SQL) for data extraction, transformation, and optimisation. Proficiency in Azure Databricks (PySpark, Delta Lake, Spark SQL) for big data processing. Knowledge of data warehousing concepts and relational database design, particularly with Azure Synapse Analytics. Experience working with Delta Lake for schema … evolution, ACID transactions, and time travel in Databricks. Strong Python (PySpark) skills for big data processing and automation. Experience with Scala (optional but preferred for advanced Spark applications). Experience working with Databricks Workflows & Jobs for data orchestration. Strong knowledge of feature engineering and feature stores, particularly in Databricks Feature store for ML training and inference. Experience with data More ❯

Location: Guernsey, Haywards Heath, Home Office (Remote) or Manchester Salary: £50,000 - £77,500 - depending on experience Department: Technology and Data We’re 1st Central, a market-leading insurance company utilising smart data and technology at pace. Rapid growth has More ❯

presentation skills, with the ability to engage both technical and non-technical stakeholders Desirable Criteria Experience with Matillion products and competitive ETL solutions Knowledge of big data technologies (Spark, PySpark), data lakes, and MPP databases (Teradata, Vertica, Netezza) Familiarity with version control tools such as Git, and experience with Python Degree in Computer Science or related field (or equivalent More ❯

Analytics department. The Successful Applicant The successful Lead Data Engineer will be able to demonstrate: A background in building data pipelines and data analysis Skilled with Azure, AWS, Python, PySpark, SQL Knowledge of data architecture and database design.. Excellent leadership and communication skills. What's on Offer We are based in Stockport and our office is easily commutable by More ❯

two years experience working with moderately complex software systems in production and have a curiosity and interest in learning more. In this role, you will be using python (specifically pyspark) and Node.js for processing data, backed by various Hadoop stack technologies such as HDFS and HBase. MongoDB and Elasticsearch are used for indexing smaller datasets. Airflow & Nifi are used … have at least two years experience working with moderately complex software systems in production and have a curiosity and interest in learning more. You will be using Python (specifically pyspark) and Node.js for processing data You will be using Hadoop stack technologies such as HDFS and HBase Experience using MongoDB and Elasticsearch for indexing smaller datasets would be beneficial More ❯

Press Tab to Move to Skip to Content Link Select how often (in days) to receive an alert: For airports, for partners, for people. We are CAVU. At CAVU our purpose is to find new and better ways to make More ❯

solutions and enjoys working with modern tech in a collaborative environment. What You’ll Do: Design, build, and maintain scalable data pipelines Optimize and automate data workflows Work with PySpark, Python, and SQL to process and manage large datasets Collaborate in a cloud-based environment to deliver efficient and reliable data solutions What We’re Looking For: Proven experience … with Python, PySpark, and SQL Strong understanding of data engineering principles and cloud infrastructure Ability to work collaboratively and communicate technical concepts clearly A passion for clean, efficient, and scalable code Why Join Us? Supportive team environment with a strong focus on innovation Opportunities for career growth and development Flexible working arrangements Interested? Please send us your CV and More ❯

development of our Configurable Integrated Processing & Assurance System (CIPAS) and its associated management application. This work will involve using a range of modern open-source tools, such as: Python PySpark R Flask CSS Familiarity with all of these tools is not required to get the role, but Python and PySpark will be the main technologies you use day More ❯

predictive modelling techniques; Logistic Regression, GBMs, Elastic Net GLMs, GAMs, Decision Trees, Random Forests, Neural Nets and Clustering Experience in statistical and data science programming languages (e.g. R, Python, PySpark, SAS, SQL) A good quantitative degree (Mathematics, Statistics, Engineering, Physics, Computer Science, Actuarial Science) Experience of WTW’s Radar and Emblem software is preferred Proficient at communicating results in More ❯

predictive modelling techniques; Logistic Regression, GBMs, Elastic Net GLMs, GAMs, Decision Trees, Random Forests, Neural Nets and Clustering Experience in statistical and data science programming languages (e.g. R, Python, PySpark, SAS, SQL) A good quantitative degree (Mathematics, Statistics, Engineering, Physics, Computer Science, Actuarial Science) Experience of WTW's Radar and Emblem software is preferred Proficient at communicating results in More ❯

predictive modelling techniques; Logistic Regression, GBMs, Elastic Net GLMs, GAMs, Decision Trees, Random Forests, Neural Nets and Clustering Experience in statistical and data science programming languages (e.g. R, Python, PySpark, SAS, SQL) A good quantitative degree (Mathematics, Statistics, Engineering, Physics, Computer Science, Actuarial Science) Experience of WTW’s Radar and Emblem software is preferred Proficient at communicating results in More ❯

Press Tab to Move to Skip to Content Link Select how often (in days) to receive an alert: For airports, for partners, for people. We are CAVU. At CAVU our purpose is to find new and better ways to make More ❯

scientific discipline, backed by minimum A A A grades at A-level You have commercial Data Engineering experience working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You have a good understanding of modern data engineering best practices Ideally you will also have experience with Azure and Data Bricks You're collaborative with excellent More ❯

Azure Data Engineer/Architect | Customer Facing Experience Essential | Azure, Synapse | £550p/day Outside IR35 - Remote (UK) Role: Azure Data Engineer/Architect with Customer-Facing Experience Duration: 3 Months Initially Location: Remote (UK) Start Date: ASAP Rate: £550p More ❯