PySpark Contracts

PySpark - Spark Python API
UK

The table below provides summary statistics for contract job vacancies requiring PySpark skills. It includes a benchmarking guide to the contractor rates offered in vacancies that cited PySpark over the 6 months leading up to 1 July 2025, comparing them to the same period in the previous two years.

6 months to
1 Jul 2025
Same period 2024 Same period 2023
Rank 319 319 431
Rank change year-on-year 0 +112 +42
Contract jobs citing PySpark 231 299 260
As % of all contract jobs advertised in the UK 0.75% 0.69% 0.47%
As % of the Libraries, Frameworks & Software Standards category 3.30% 3.23% 2.05%
Number of daily rates quoted 168 232 181
10th Percentile £363 £413 £450
25th Percentile £456 £468 £513
Median daily rate (50th Percentile) £550 £525 £600
Median % change year-on-year +4.76% -12.50% -
75th Percentile £828 £638 £713
90th Percentile £875 £800 £860
UK excluding London median daily rate £469 £490 £600
% change year-on-year -4.34% -18.33% +9.09%
Number of hourly rates quoted 0 7 0
Median hourly rate - £20.47 -
UK excluding London median hourly rate - £20.24 -

All Software Libraries and Frameworks
UK

PySpark falls under the Software Libraries and Frameworks category. For comparison with the information above, the following table provides summary statistics for all contract job vacancies requiring technical specification, industry standards, software libraries and framework skills.

Contract vacancies with a requirement for technical specification, industry standards, software libraries and framework skills 7,001 9,253 12,682
As % of all contract IT jobs advertised in the UK 22.72% 21.44% 22.73%
Number of daily rates quoted 4,746 6,337 9,221
10th Percentile £352 £350 £388
25th Percentile £413 £425 £463
Median daily rate (50th Percentile) £513 £538 £550
Median % change year-on-year -4.65% -2.27% -
75th Percentile £638 £650 £675
90th Percentile £763 £788 £788
UK excluding London median daily rate £475 £500 £500
% change year-on-year -5.00% - -0.99%
Number of hourly rates quoted 143 282 155
10th Percentile £27.81 £22.53 £28.50
25th Percentile £36.40 £43.00 £50.00
Median hourly rate £55.00 £55.00 £65.00
Median % change year-on-year - -15.38% +25.30%
75th Percentile £68.00 £70.00 £75.00
90th Percentile £80.81 £74.40 £80.70
UK excluding London median hourly rate £62.50 £58.30 £62.50
% change year-on-year +7.20% -6.72% +13.64%

PySpark
Job Vacancy Trend

Job postings citing PySpark as a proportion of all IT jobs advertised.

Job vacancy trend for PySpark in the UK

PySpark
Contractor Daily Rate Trend

3-month moving average daily rate quoted in jobs citing PySpark.

Daily rate trend for PySpark in the UK

PySpark
Daily Rate Histogram

Daily rate distribution for jobs citing PySpark over the 6 months to 1 July 2025.

Daily rate histogram for PySpark in the UK

PySpark
Contractor Hourly Rate Trend

3-month moving average hourly rates quoted in jobs citing PySpark.

Hourly rate trend for PySpark in the UK

PySpark
Top 15 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing PySpark within the UK over the 6 months to 1 July 2025. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Jobs
England +14 205 £575 +4.55% 454
London +18 152 £700 +27.27% 303
Work from Home +15 96 £573 +14.50% 223
UK excluding London +26 54 £469 -4.34% 190
North of England +11 22 £475 -2.06% 55
Yorkshire +2 12 £488 -0.51% 18
North West +9 10 £450 -3.74% 34
South East +53 9 £465 +24.00% 46
Midlands +19 7 £538 -10.42% 21
Scotland -11 7 £469 -6.25% 21
East of England - 7 £318 - 7
West Midlands +10 5 £600 - 12
East Midlands - 2 £475 - 9
South West +29 1 £425 -7.61% 37
Wales +2 1 £675 +104.55% 2

PySpark
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 70 (30.30%) Apache
2 61 (26.41%) Apache Spark
3 59 (25.54%) Apache Airflow
4 4 (1.73%) Jupyter
5 1 (0.43%) SharePoint
Applications
1 1 (0.43%) Spreadsheet
Cloud Services
1 94 (40.69%) AWS
2 93 (40.26%) Azure
3 51 (22.08%) dbt
4 36 (15.58%) Power Platform
5 32 (13.85%) Serverless
6 30 (12.99%) Azure Synapse Analytics
6 30 (12.99%) GCP
7 29 (12.55%) AWS Lambda
7 29 (12.55%) Azure Data Factory
8 26 (11.26%) Snowflake
9 23 (9.96%) Azure DevOps
10 20 (8.66%) Amazon ECS
11 19 (8.23%) Amazon S3
12 16 (6.93%) GitHub
13 15 (6.49%) OpenShift
14 13 (5.63%) Amazon EC2
15 9 (3.90%) Dagster
15 9 (3.90%) Microsoft Purview
16 6 (2.60%) GitHub Actions
17 5 (2.16%) Amazon EKS
Communications & Networking
1 4 (1.73%) SSH
2 1 (0.43%) Ethernet
Database & Business Intelligence
1 52 (22.51%) Amazon Athena
2 50 (21.65%) Data Lake
3 44 (19.05%) Apache Iceberg
4 40 (17.32%) Dremio
5 38 (16.45%) Data Warehouse
6 36 (15.58%) Apache Arrow
6 36 (15.58%) Power BI
7 26 (11.26%) Delta Lake
8 23 (9.96%) Big Data
9 21 (9.09%) Hadoop
10 20 (8.66%) Apache Ignite
11 19 (8.23%) Apache Hive
12 13 (5.63%) Spark SQL
13 12 (5.19%) Amazon Redshift
13 12 (5.19%) Metadata
14 11 (4.76%) Azure SQL Database
15 9 (3.90%) Relational Database
15 9 (3.90%) SQL Server
16 8 (3.46%) BigQuery
16 8 (3.46%) Tableau
Development Applications
1 30 (12.99%) Git
2 15 (6.49%) Artifactory
3 6 (2.60%) JIRA
4 5 (2.16%) Jenkins
5 4 (1.73%) RStudio
6 2 (0.87%) GitLab
7 1 (0.43%) Bitbucket
7 1 (0.43%) Postman
7 1 (0.43%) Selenium
7 1 (0.43%) SonarQube
7 1 (0.43%) SpecFlow
General
1 77 (33.33%) Finance
2 44 (19.05%) Banking
3 40 (17.32%) Front Office
4 38 (16.45%) Social Skills
5 35 (15.15%) Analytical Skills
6 27 (11.69%) Investment Banking
7 15 (6.49%) Public Sector
8 8 (3.46%) Retail
9 7 (3.03%) Financial Institution
10 4 (1.73%) Organisational Skills
11 3 (1.30%) Corporate Banking
11 3 (1.30%) Retail Banking
11 3 (1.30%) Telecoms
12 2 (0.87%) Automotive
12 2 (0.87%) Health Technology
12 2 (0.87%) Tech for Good
13 1 (0.43%) Law
13 1 (0.43%) Legal
13 1 (0.43%) Manufacturing
13 1 (0.43%) Presentation Skills
Job Titles
1 132 (57.14%) Data Engineer
2 48 (20.78%) AWS Engineer
3 44 (19.05%) Python Engineer
4 42 (18.18%) Developer
5 28 (12.12%) Senior
6 23 (9.96%) Software Engineer
7 21 (9.09%) AWS Software Engineer
7 21 (9.09%) Senior Data Engineer
8 20 (8.66%) Lead
9 19 (8.23%) Python Developer
10 18 (7.79%) Python Data Engineer
11 17 (7.36%) Azure Engineer
11 17 (7.36%) Java Engineer
11 17 (7.36%) Snowflake Engineer
11 17 (7.36%) Software Developer
12 16 (6.93%) AWS Developer
12 16 (6.93%) Azure Data Engineer
12 16 (6.93%) Developer/Software Engineer
12 16 (6.93%) DevOps Engineer
12 16 (6.93%) Snowflake Developer
Libraries, Frameworks & Software Standards
1 29 (12.55%) Kafka
2 28 (12.12%) Pandas
3 16 (6.93%) NumPy
4 15 (6.49%) J2EE
5 11 (4.76%) JSON
5 11 (4.76%) YAML
6 10 (4.33%) SciPy
7 9 (3.90%) TensorFlow
8 6 (2.60%) pydantic
8 6 (2.60%) pytest
8 6 (2.60%) React
9 5 (2.16%) Apache NiFi
9 5 (2.16%) gRPC
9 5 (2.16%) Protocol Buffers
9 5 (2.16%) scikit-learn
10 4 (1.73%) AngularJS
11 3 (1.30%) CSS
11 3 (1.30%) FastAPI
11 3 (1.30%) HTML
12 2 (0.87%) Plotly
Miscellaneous
1 22 (9.52%) Derivative
1 22 (9.52%) Hedge funds
2 21 (9.09%) Equities
2 21 (9.09%) Fixed Income
2 21 (9.09%) Foreign Exchange (FX)
3 18 (7.79%) Distributed Computing
4 12 (5.19%) Algorithms
5 10 (4.33%) Cloud Native
5 10 (4.33%) Data Structures
6 7 (3.03%) Credit Risk
6 7 (3.03%) Linux Command Line
7 6 (2.60%) Reference Data
8 5 (2.16%) Legacy Systems
8 5 (2.16%) Renewable Energy
9 4 (1.73%) Data Protection Act
9 4 (1.73%) Hybrid Cloud
9 4 (1.73%) Onboarding
10 3 (1.30%) Product Ownership
10 3 (1.30%) Virtual Team
11 2 (0.87%) Distributed Systems
Operating Systems
1 6 (2.60%) Linux
2 4 (1.73%) Unix
Processes & Methodologies
1 93 (40.26%) Data Pipeline
2 86 (37.23%) Data Engineering
3 79 (34.20%) ETL
4 76 (32.90%) Agile
5 68 (29.44%) CI/CD
6 58 (25.11%) Analytics
7 50 (21.65%) DevOps
8 46 (19.91%) Software Engineering
9 45 (19.48%) Data Modelling
10 43 (18.61%) Asset Management
11 38 (16.45%) Data Ingestion
11 38 (16.45%) Problem-Solving
12 36 (15.58%) Data Science
13 35 (15.15%) Machine Learning
14 31 (13.42%) Artificial Intelligence
15 28 (12.12%) Business Intelligence
15 28 (12.12%) Data Governance
16 25 (10.82%) Data Transformation
17 23 (9.96%) Data Management
17 23 (9.96%) Data Security
Programming Languages
1 195 (84.42%) Python
2 128 (55.41%) SQL
3 54 (23.38%) Java
4 23 (9.96%) Scala
5 15 (6.49%) Go
6 11 (4.76%) JavaScript
7 10 (4.33%) TypeScript
8 7 (3.03%) Bash
9 5 (2.16%) C++
10 2 (0.87%) C#
10 2 (0.87%) Data Analysis Expressions
10 2 (0.87%) PL/SQL
10 2 (0.87%) Shell Script
11 1 (0.43%) C
11 1 (0.43%) Rust
Qualifications
1 44 (19.05%) Security Cleared
2 39 (16.88%) SC Cleared
3 18 (7.79%) Degree
4 11 (4.76%) Master's Degree
5 10 (4.33%) PhD
6 9 (3.90%) Computer Science Degree
7 6 (2.60%) BPSS Clearance
8 3 (1.30%) Azure Certification
9 2 (0.87%) AWS Certification
9 2 (0.87%) AWS Certified Solutions Architect
10 1 (0.43%) DV Cleared
10 1 (0.43%) MCP
Quality Assurance & Compliance
1 31 (13.42%) Data Quality
2 14 (6.06%) GDPR
3 10 (4.33%) Accessibility
4 4 (1.73%) QA
System Software
1 29 (12.55%) Docker
2 1 (0.43%) Apache ZooKeeper
Systems Management
1 36 (15.58%) Terraform
2 28 (12.12%) Kubernetes
3 2 (0.87%) Ansible
3 2 (0.87%) Puppet
4 1 (0.43%) Progress Chef
Vendors
1 71 (30.74%) Databricks
2 15 (6.49%) Foundry
2 15 (6.49%) Microsoft
2 15 (6.49%) Palantir
3 10 (4.33%) Informatica
4 6 (2.60%) Google
5 5 (2.16%) Oracle
6 4 (1.73%) Talend
7 2 (0.87%) Brio
7 2 (0.87%) Confluent
7 2 (0.87%) MuleSoft
7 2 (0.87%) Salesforce
8 1 (0.43%) Bloomberg
8 1 (0.43%) Checkmarx
8 1 (0.43%) Dynatrace
8 1 (0.43%) IBM
8 1 (0.43%) Refinitiv
8 1 (0.43%) Reuters
8 1 (0.43%) StreamSets
8 1 (0.43%) Virgin Media