PySpark Contracts in the City of London

PySpark - Spark Python API
Central London > City of London

The following table provides summary statistics for contract job vacancies advertised in the City of London with a requirement for PySpark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited PySpark over the 6 months to 12 May 2024 with a comparison to the same period in the previous 2 years.

6 months to
12 May 2024
Same period 2023 Same period 2022
Rank 121 147 175
Rank change year-on-year +26 +28 -38
Contract jobs citing PySpark 9 26 19
As % of all contract jobs advertised in the City of London 0.41% 0.73% 0.45%
As % of the Libraries, Frameworks & Software Standards category 1.86% 2.59% 1.46%
Number of daily rates quoted 9 18 8
10th Percentile £520 £438 £494
25th Percentile £563 £470 £541
Median daily rate (50th Percentile) £700 £563 £595
Median % change year-on-year +24.44% -5.50% +7.25%
75th Percentile £763 £738 £659
90th Percentile £765 £784 £718
Central London median daily rate £675 £563 £590
% change year-on-year +20.00% -4.70% +6.35%

All Software Libraries and Frameworks
City of London

PySpark is in the Software Libraries and Frameworks category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in the City of London with a requirement for technical specification, industry standards, software libraries and framework skills.

Contract vacancies with a requirement for technical specification, industry standards, software libraries and framework skills 485 1,002 1,298
As % of all contract IT jobs advertised in the City of London 22.25% 28.06% 30.70%
Number of daily rates quoted 339 652 1,044
10th Percentile £400 £425 £425
25th Percentile £463 £506 £509
Median daily rate (50th Percentile) £560 £625 £600
Median % change year-on-year -10.40% +4.17% +9.09%
75th Percentile £688 £738 £713
90th Percentile £788 £820 £800
Central London median daily rate £550 £625 £600
% change year-on-year -12.00% +4.17% +9.09%
Number of hourly rates quoted 0 0 2
10th Percentile - - £49.33
25th Percentile - - £53.94
Median hourly rate - - £62.25
Median % change year-on-year - - +255.71%
75th Percentile - - £69.94
90th Percentile - - £74.18
Central London median hourly rate - - £62.25
% change year-on-year - - +255.71%

PySpark
Job Vacancy Trend in the City of London

Job postings citing PySpark as a proportion of all IT jobs advertised in the City of London.

Job vacancy trend for PySpark in the City of London

PySpark
Contractor Daily Rate Trend in the City of London

3-month moving average daily rate quoted in jobs citing PySpark in the City of London.

Daily rate trend for PySpark in the City of London

PySpark
Daily Rate Histogram in the City of London

Daily rate distribution for jobs citing PySpark in the City of London over the 6 months to 12 May 2024.

Daily rate histogram for PySpark in the City of London

PySpark
Co-occurring Skills and Capabilities in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 1 (11.11%) Apache
1 1 (11.11%) Apache Airflow
1 1 (11.11%) Apache Spark
Cloud Services
1 7 (77.78%) Azure
2 4 (44.44%) Power Platform
3 3 (33.33%) Azure Synapse Analytics
4 2 (22.22%) AWS
4 2 (22.22%) Azure Data Factory
5 1 (11.11%) Amazon EventBridge
5 1 (11.11%) Amazon Kinesis
5 1 (11.11%) Azure Monitor
5 1 (11.11%) GCP
Database & Business Intelligence
1 6 (66.67%) Data Warehouse
2 4 (44.44%) Delta Lake
2 4 (44.44%) Power BI
3 3 (33.33%) Big Data
3 3 (33.33%) Data Mart
4 2 (22.22%) Amazon Redshift
4 2 (22.22%) Hadoop
4 2 (22.22%) Presto
5 1 (11.11%) Azure SQL Database
5 1 (11.11%) Data Lake
5 1 (11.11%) NoSQL
5 1 (11.11%) Parquet
5 1 (11.11%) Star Schema
General
1 2 (22.22%) Finance
2 1 (11.11%) Social Skills
Job Titles
1 6 (66.67%) Data Engineer
2 4 (44.44%) Azure Engineer
3 2 (22.22%) Python Data Engineer
3 2 (22.22%) Python Engineer
4 1 (11.11%) AWS Data Engineer
4 1 (11.11%) AWS Engineer
4 1 (11.11%) Azure Data Engineer
4 1 (11.11%) Lead
4 1 (11.11%) Lead Data Engineer
Libraries, Frameworks & Software Standards
1 2 (22.22%) Kafka
2 1 (11.11%) Apache Avro
2 1 (11.11%) JSON
2 1 (11.11%) JSON Schema
2 1 (11.11%) Middleware
Miscellaneous
1 1 (11.11%) Greenfield Project
1 1 (11.11%) Market Risk
Processes & Methodologies
1 7 (77.78%) Analytics
2 6 (66.67%) Data Engineering
2 6 (66.67%) Data Modelling
3 4 (44.44%) Agile
3 4 (44.44%) ETL
4 3 (33.33%) Data Science
5 2 (22.22%) Business Analysis
5 2 (22.22%) Business Development
5 2 (22.22%) CI/CD
5 2 (22.22%) Commercialisation
5 2 (22.22%) Computer Science
5 2 (22.22%) Data Integration
5 2 (22.22%) Data Pipeline
5 2 (22.22%) ERP
5 2 (22.22%) Mathematics
5 2 (22.22%) Migration
5 2 (22.22%) Ontologies
5 2 (22.22%) Software Engineering
6 1 (11.11%) Log Analytics
6 1 (11.11%) Unit Testing
Programming Languages
1 9 (100.00%) SQL
2 5 (55.56%) Python
3 1 (11.11%) Scala
3 1 (11.11%) T-SQL
Quality Assurance & Compliance
1 1 (11.11%) Data Quality
System Software
1 1 (11.11%) Docker
Systems Management
1 2 (22.22%) Terraform
2 1 (11.11%) Kubernetes
Vendors
1 7 (77.78%) Databricks
2 5 (55.56%) Microsoft
3 2 (22.22%) Foundry
3 2 (22.22%) Oracle
3 2 (22.22%) Palantir
3 2 (22.22%) SAP