PySpark Jobs with Work from Home Options

PySpark - Spark Python API
UK > Work from Home

The table below provides summary statistics for permanent job vacancies with an option to work from home (WFH) and a requirement for PySpark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited PySpark over the 6 months to 17 May 2024 with a comparison to the same period in the previous 2 years.

6 months to
17 May 2024
Same period 2023 Same period 2022
Rank 453 523 630
Rank change year-on-year +70 +107 -86
Permanent jobs citing PySpark 144 141 241
As % of all permanent jobs with a WFH option 0.43% 0.35% 0.34%
As % of the Libraries, Frameworks & Software Standards category 1.60% 1.03% 0.76%
Number of salaries quoted 88 122 81
10th Percentile £52,500 £52,500 £48,250
25th Percentile £55,000 £62,500 £50,000
Median annual salary (50th Percentile) £67,500 £85,000 £65,000
Median % change year-on-year -20.59% +30.77% +13.04%
75th Percentile £83,125 £105,000 £80,000
90th Percentile £100,000 £160,000 £85,000
UK median annual salary £70,000 £80,000 £70,000
% change year-on-year -12.50% +14.29% +7.69%

All Software Libraries and Frameworks
Work from Home

PySpark is in the Software Libraries and Frameworks category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a WFH option and a requirement for technical specification, industry standards, software libraries and framework skills.

Permanent vacancies with a requirement for technical specification, industry standards, software libraries and framework skills 9,026 13,738 31,783
As % of all permanent jobs with a WFH option 26.68% 33.97% 45.28%
Number of salaries quoted 7,528 10,307 17,107
10th Percentile £38,750 £41,250 £38,750
25th Percentile £47,500 £51,250 £46,250
Median annual salary (50th Percentile) £60,000 £65,000 £62,500
Median % change year-on-year -7.69% +4.00% +13.64%
75th Percentile £79,500 £86,250 £81,250
90th Percentile £100,000 £103,750 £95,000
UK median annual salary £60,000 £67,500 £63,750
% change year-on-year -11.11% +5.88% +15.91%

PySpark
Trend for Jobs with a WFH Option

Job vacancies with a work from home option citing PySpark as a proportion of all IT jobs advertised.

PySpark trend for jobs with a WFH option

PySpark
Salary Trend for Jobs with a WFH Option

3-month moving average salary quoted in job vacancies with a work from home option citing PySpark.

PySpark salary trend for jobs with a WFH option

PySpark
Salary Histogram for Jobs with a WFH Option

Salary distribution for jobs with a work from home option citing PySpark over the 6 months to 17 May 2024.

PySpark salary histogram for jobs with a WFH option

PySpark
Co-occurring Skills and Capabilities with WFH Options by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 19 (13.19%) Apache Spark
2 11 (7.64%) Apache
2 11 (7.64%) Apache Airflow
3 2 (1.39%) Luigi
Applications
1 7 (4.86%) Microsoft Excel
Business Applications
1 4 (2.78%) Primavera
2 1 (0.69%) SAP ERP
Cloud Services
1 79 (54.86%) Azure
2 31 (21.53%) Azure Data Factory
2 31 (21.53%) Azure Synapse Analytics
3 23 (15.97%) AWS
4 22 (15.28%) Power Platform
5 18 (12.50%) Azure DevOps
6 17 (11.81%) GCP
7 16 (11.11%) Azure Cosmos DB
8 12 (8.33%) GitHub
8 12 (8.33%) Snowflake
9 8 (5.56%) Power Automate
9 8 (5.56%) PowerApps
10 6 (4.17%) AWS Lambda
10 6 (4.17%) Azure AKS
10 6 (4.17%) Azure Machine Learning
10 6 (4.17%) Serverless
11 5 (3.47%) Amazon S3
11 5 (3.47%) SaaS
12 4 (2.78%) AWS CloudFormation
12 4 (2.78%) dbt
Communications & Networking
1 2 (1.39%) Network Security
Database & Business Intelligence
1 45 (31.25%) Tableau
2 41 (28.47%) Data Lake
3 33 (22.92%) Data Warehouse
4 22 (15.28%) Big Data
4 22 (15.28%) Power BI
5 16 (11.11%) SQL Server
6 15 (10.42%) Azure SQL Database
7 13 (9.03%) Spark SQL
8 10 (6.94%) Delta Lake
9 7 (4.86%) Data Mining
9 7 (4.86%) NoSQL
10 5 (3.47%) Amazon Redshift
10 5 (3.47%) MySQL
11 4 (2.78%) BigQuery
11 4 (2.78%) Data Mart
11 4 (2.78%) Semantic Layer
11 4 (2.78%) SQL Server Integration Services
12 3 (2.08%) Amazon Aurora
12 3 (2.08%) HDInsight
12 3 (2.08%) PostgreSQL
Development Applications
1 55 (38.19%) Git
2 10 (6.94%) MLflow
3 7 (4.86%) Jenkins
4 3 (2.08%) Octopus Deploy
4 3 (2.08%) Team Foundation Server
4 3 (2.08%) TeamCity
5 2 (1.39%) CircleCI
5 2 (1.39%) GitLab
5 2 (1.39%) JUnit
5 2 (1.39%) MSTest
5 2 (1.39%) NUnit
General
1 66 (45.83%) Social Skills
2 12 (8.33%) Finance
3 8 (5.56%) Analytical Skills
3 8 (5.56%) Legal
4 6 (4.17%) Law
5 5 (3.47%) Banking
5 5 (3.47%) Inclusion and Diversity
5 5 (3.47%) Influencing Skills
5 5 (3.47%) Retail
6 2 (1.39%) Publishing
7 1 (0.69%) Organisational Skills
7 1 (0.69%) Pharmaceutical
7 1 (0.69%) Public Sector
Job Titles
1 85 (59.03%) Senior
2 69 (47.92%) Data Engineer
3 50 (34.72%) Data Scientist
4 47 (32.64%) Senior Data Scientist
5 31 (21.53%) Senior Data Engineer
6 23 (15.97%) Azure Engineer
7 22 (15.28%) Azure Data Engineer
8 16 (11.11%) Lead
9 7 (4.86%) Lead Data Engineer
9 7 (4.86%) Python Data Engineer
9 7 (4.86%) Python Engineer
9 7 (4.86%) Technical Leader
10 5 (3.47%) Customer Engineer
11 4 (2.78%) Data Manager
11 4 (2.78%) Engineering Manager
11 4 (2.78%) Senior Manager
12 3 (2.08%) Azure Data Architect
12 3 (2.08%) Data Consultant
12 3 (2.08%) Platform Engineer
12 3 (2.08%) Senior Platform Engineer
Libraries, Frameworks & Software Standards
1 52 (36.11%) Pandas
2 45 (31.25%) scikit-learn
2 45 (31.25%) TensorFlow
3 6 (4.17%) JSON
4 4 (2.78%) AWS CDK
5 2 (1.39%) MDX
5 2 (1.39%) SOAP
5 2 (1.39%) SQLAlchemy
6 1 (0.69%) Kafka
Miscellaneous
1 48 (33.33%) Algorithms
2 9 (6.25%) Product Ownership
3 4 (2.78%) Reinsurance
4 3 (2.08%) Data Structures
4 3 (2.08%) Greenfield Project
5 2 (1.39%) Hybrid Cloud
5 2 (1.39%) PropTech
5 2 (1.39%) Team-Oriented Environment
6 1 (0.69%) Blockchain
6 1 (0.69%) Distributed Computing
6 1 (0.69%) Low-Code
6 1 (0.69%) Management Information System
Operating Systems
1 1 (0.69%) Linux
1 1 (0.69%) Ubuntu
Processes & Methodologies
1 71 (49.31%) Machine Learning
2 65 (45.14%) Data Science
3 59 (40.97%) Data Engineering
4 54 (37.50%) Decision-Making
5 51 (35.42%) Visualisation
6 50 (34.72%) Artificial Intelligence
6 50 (34.72%) MLOps
7 48 (33.33%) Customer Experience
8 47 (32.64%) Mathematics
8 47 (32.64%) Natural Language Processing
8 47 (32.64%) Statistics
9 46 (31.94%) Predictive Modelling
10 45 (31.25%) Actionable Insight
10 45 (31.25%) Data Munging
10 45 (31.25%) Demand Forecasting
10 45 (31.25%) Physics
10 45 (31.25%) Propensity Modelling
10 45 (31.25%) Root Cause Analysis
10 45 (31.25%) Storytelling
10 45 (31.25%) WFM
Programming Languages
1 128 (88.89%) SQL
2 112 (77.78%) Python
3 13 (9.03%) T-SQL
4 6 (4.17%) Data Analysis Expressions
5 5 (3.47%) C#
5 5 (3.47%) R
6 4 (2.78%) Scala
Qualifications
1 54 (37.50%) Degree
2 47 (32.64%) Master's Degree
3 45 (31.25%) DBS Check
4 6 (4.17%) Computer Science Degree
5 4 (2.78%) CIMA
6 2 (1.39%) Microsoft Certification
6 2 (1.39%) PhD
6 2 (1.39%) Security Cleared
7 1 (0.69%) Azure Certification
7 1 (0.69%) Postgraduate
7 1 (0.69%) SC Cleared
Quality Assurance & Compliance
1 13 (9.03%) Data Quality
2 2 (1.39%) QA
System Software
1 2 (1.39%) Docker
Systems Management
1 19 (13.19%) Terraform
2 12 (8.33%) Kubernetes
Vendors
1 119 (82.64%) Databricks
2 17 (11.81%) Microsoft
3 5 (3.47%) Google
3 5 (3.47%) SAP
4 4 (2.78%) SnapLogic
5 2 (1.39%) Oracle
6 1 (0.69%) Marketo
6 1 (0.69%) OpenAI
6 1 (0.69%) Salesforce