PySpark Contracts in the City of London

PySpark - Spark Python API
Central London > City of London

The following table provides summary statistics for contract job vacancies advertised in the City of London with a requirement for PySpark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited PySpark over the 6 months to 28 April 2024 with a comparison to the same period in the previous 2 years.

6 months to
28 Apr 2024
Same period 2023 Same period 2022
Rank 114 150 171
Rank change year-on-year +36 +21 -40
Contract jobs citing PySpark 12 27 21
As % of all contract jobs advertised in the City of London 0.54% 0.74% 0.50%
As % of the Libraries, Frameworks & Software Standards category 2.49% 2.59% 1.62%
Number of daily rates quoted 12 20 10
10th Percentile £503 £438 £495
25th Percentile £553 £498 £522
Median daily rate (50th Percentile) £675 £563 £595
Median % change year-on-year +20.00% -5.50% +7.25%
75th Percentile £763 £738 £650
90th Percentile £774 £761 £698
Central London median daily rate £675 £563 £595
% change year-on-year +20.00% -5.50% +7.25%

All Software Libraries and Frameworks
City of London

PySpark is in the Software Libraries and Frameworks category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in the City of London with a requirement for technical specification, industry standards, software libraries and framework skills.

Contract vacancies with a requirement for technical specification, industry standards, software libraries and framework skills 481 1,041 1,297
As % of all contract IT jobs advertised in the City of London 21.84% 28.56% 30.60%
Number of daily rates quoted 343 676 1,040
10th Percentile £405 £425 £425
25th Percentile £463 £506 £506
Median daily rate (50th Percentile) £565 £625 £600
Median % change year-on-year -9.60% +4.17% +9.09%
75th Percentile £688 £738 £713
90th Percentile £775 £825 £800
Central London median daily rate £551 £625 £600
% change year-on-year -11.92% +4.17% +9.09%
Number of hourly rates quoted 0 0 1
10th Percentile - - -
25th Percentile - - £46.25
Median hourly rate - - £47.50
Median % change year-on-year - - +171.43%
75th Percentile - - £48.75
90th Percentile - - -
Central London median hourly rate - - £47.50
% change year-on-year - - +171.43%

PySpark
Job Vacancy Trend in the City of London

Job postings citing PySpark as a proportion of all IT jobs advertised in the City of London.

Job vacancy trend for PySpark in the City of London

PySpark
Contractor Daily Rate Trend in the City of London

3-month moving average daily rate quoted in jobs citing PySpark in the City of London.

Daily rate trend for PySpark in the City of London

PySpark
Daily Rate Histogram in the City of London

Daily rate distribution for jobs citing PySpark in the City of London over the 6 months to 28 April 2024.

Daily rate histogram for PySpark in the City of London

PySpark
Co-occurring Skills and Capabilities in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 1 (8.33%) Apache
1 1 (8.33%) Apache Airflow
1 1 (8.33%) Apache Spark
Cloud Services
1 9 (75.00%) Azure
2 7 (58.33%) Power Platform
3 3 (25.00%) AWS
3 3 (25.00%) Azure Synapse Analytics
4 2 (16.67%) Azure Data Factory
5 1 (8.33%) Amazon EventBridge
5 1 (8.33%) Amazon Kinesis
5 1 (8.33%) Amazon S3
5 1 (8.33%) Azure Monitor
5 1 (8.33%) GCP
Database & Business Intelligence
1 8 (66.67%) Data Warehouse
2 7 (58.33%) Power BI
3 6 (50.00%) Delta Lake
4 5 (41.67%) Data Mart
5 3 (25.00%) Amazon Redshift
5 3 (25.00%) Big Data
5 3 (25.00%) Star Schema
6 2 (16.67%) Hadoop
6 2 (16.67%) Presto
7 1 (8.33%) Amazon Athena
7 1 (8.33%) Azure SQL Database
7 1 (8.33%) Data Lake
7 1 (8.33%) NoSQL
7 1 (8.33%) Parquet
7 1 (8.33%) Tableau
General
1 2 (16.67%) Finance
2 1 (8.33%) Social Skills
Job Titles
1 8 (66.67%) Data Engineer
2 6 (50.00%) Azure Engineer
3 2 (16.67%) Azure Data Engineer
3 2 (16.67%) Python Data Engineer
3 2 (16.67%) Python Engineer
4 1 (8.33%) AWS Data Engineer
4 1 (8.33%) AWS Engineer
4 1 (8.33%) Cloud Engineer
4 1 (8.33%) Lead
4 1 (8.33%) Lead Data Engineer
Libraries, Frameworks & Software Standards
1 3 (25.00%) Kafka
2 1 (8.33%) Apache Avro
2 1 (8.33%) JSON
2 1 (8.33%) JSON Schema
2 1 (8.33%) Middleware
Miscellaneous
1 3 (25.00%) Market Risk
2 1 (8.33%) Greenfield Project
Processes & Methodologies
1 9 (75.00%) Analytics
2 8 (66.67%) Data Engineering
2 8 (66.67%) Data Modelling
3 5 (41.67%) Agile
3 5 (41.67%) ETL
4 4 (33.33%) Business Intelligence
5 3 (25.00%) Data Science
6 2 (16.67%) Business Analysis
6 2 (16.67%) Business Development
6 2 (16.67%) CI/CD
6 2 (16.67%) Commercialisation
6 2 (16.67%) Computer Science
6 2 (16.67%) Data Integration
6 2 (16.67%) Data Pipeline
6 2 (16.67%) ERP
6 2 (16.67%) Mathematics
6 2 (16.67%) Migration
6 2 (16.67%) Ontologies
6 2 (16.67%) Software Engineering
7 1 (8.33%) Unit Testing
Programming Languages
1 12 (100.00%) SQL
2 6 (50.00%) Python
3 1 (8.33%) Scala
3 1 (8.33%) T-SQL
Quality Assurance & Compliance
1 1 (8.33%) Data Quality
System Software
1 1 (8.33%) Docker
Systems Management
1 2 (16.67%) Terraform
2 1 (8.33%) Kubernetes
Vendors
1 9 (75.00%) Databricks
2 7 (58.33%) Microsoft
3 2 (16.67%) Foundry
3 2 (16.67%) Oracle
3 2 (16.67%) Palantir
3 2 (16.67%) SAP
4 1 (8.33%) Informatica