PySpark Contracts with Work from Home Options

PySpark - Spark Python API
UK > Work from Home

The table below provides summary statistics for contract job vacancies with an option to work from home (WFH) and a requirement for PySpark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited PySpark over the 6 months to 16 May 2024 with a comparison to the same period in the previous 2 years.

6 months to
16 May 2024
Same period 2023 Same period 2022
Rank 265 348 374
Rank change year-on-year +83 +26 -10
Contract jobs citing PySpark 82 120 261
As % of all contract jobs with a WFH option 0.55% 0.49% 0.59%
As % of the Libraries, Frameworks & Software Standards category 2.26% 1.93% 2.01%
Number of daily rates quoted 71 88 166
10th Percentile £406 £446 £463
25th Percentile £475 £513 £507
Median daily rate (50th Percentile) £500 £606 £570
Median % change year-on-year -17.53% +6.36% +8.57%
75th Percentile £581 £741 £693
90th Percentile £625 £891 £794
UK median daily rate £525 £600 £600
% change year-on-year -12.50% - +14.29%
Number of hourly rates quoted 1 0 1
10th Percentile - - -
25th Percentile £20.50 - -
Median hourly rate £21.00 - £78.75
75th Percentile £21.50 - -
90th Percentile - - -
UK median hourly rate £20.47 - £78.75
% change year-on-year - - +215.00%

All Software Libraries and Frameworks
Work from Home

PySpark is in the Software Libraries and Frameworks category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a WFH option and a requirement for technical specification, industry standards, software libraries and framework skills.

Contract vacancies with a requirement for technical specification, industry standards, software libraries and framework skills 3,629 6,225 12,985
As % of all contract jobs with a WFH option 24.52% 25.26% 29.55%
Number of daily rates quoted 2,545 4,638 9,580
10th Percentile £339 £400 £400
25th Percentile £431 £450 £450
Median daily rate (50th Percentile) £525 £540 £550
Median % change year-on-year -2.78% -1.82% +12.24%
75th Percentile £650 £656 £645
90th Percentile £800 £750 £738
UK median daily rate £535 £550 £550
% change year-on-year -2.73% - +10.00%
Number of hourly rates quoted 93 77 124
10th Percentile £24.45 £22.50 £21.30
25th Percentile - £45.00 £35.00
Median hourly rate £43.00 £62.50 £50.00
Median % change year-on-year -31.20% +25.00% +11.11%
75th Percentile £62.00 £75.00 £65.25
90th Percentile £72.50 £80.25 £75.00
UK median hourly rate £47.50 £65.00 £50.00
% change year-on-year -26.92% +30.00% +11.11%

PySpark
Trend for Jobs with a WFH Option

Job vacancies with a work from home option citing PySpark as a proportion of all IT jobs advertised.

PySpark trend for jobs with a WFH option

PySpark
Daily Rate Trend for Jobs with a WFH Option

3-month moving average daily rate quoted in job vacancies with a work from home option citing PySpark.

PySpark daily rate for jobs with a WFH option

PySpark
Daily Rate Histogram for Jobs with a WFH Option

Daily rate distribution for jobs with a work from home option citing PySpark over the 6 months to 16 May 2024.

PySpark daily rate histogram for jobs with a WFH option

PySpark
Hourly Rate Trend for Jobs with a WFH Option

3-month moving average hourly rate quoted in job vacancies with a work from home option citing PySpark.

PySpark hourly rate for jobs with a WFH option

PySpark
Co-occurring Skills and Capabilities with WFH Options by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 17 (20.73%) Apache Spark
2 14 (17.07%) Apache Airflow
3 8 (9.76%) Apache
Business Applications
1 1 (1.22%) Dynamics CRM
Cloud Services
1 44 (53.66%) Azure
2 25 (30.49%) AWS
3 19 (23.17%) Azure Data Factory
4 15 (18.29%) Serverless
5 12 (14.63%) AWS Lambda
6 11 (13.41%) Azure Synapse Analytics
6 11 (13.41%) dbt
7 9 (10.98%) Amazon EMR
8 8 (9.76%) Amazon S3
8 8 (9.76%) Snowflake
9 7 (8.54%) Amazon Kinesis
9 7 (8.54%) Azure Logic Apps
9 7 (8.54%) Power Platform
10 6 (7.32%) Azure DevOps
10 6 (7.32%) Azure Monitor
11 5 (6.10%) Amazon EventBridge
11 5 (6.10%) Azure Cosmos DB
11 5 (6.10%) GCP
12 4 (4.88%) Amazon CloudWatch
12 4 (4.88%) Azure Key Vault
Database & Business Intelligence
1 28 (34.15%) Data Lake
2 27 (32.93%) Big Data
2 27 (32.93%) Data Warehouse
3 14 (17.07%) Amazon Redshift
4 10 (12.20%) Delta Lake
5 7 (8.54%) Amazon Athena
5 7 (8.54%) Power BI
6 6 (7.32%) Azure SQL Database
6 6 (7.32%) Parquet
7 5 (6.10%) Spark SQL
8 3 (3.66%) Snowflake Schema
8 3 (3.66%) SQL Server
9 2 (2.44%) Apache Hive
9 2 (2.44%) Hadoop
9 2 (2.44%) Metadata
9 2 (2.44%) NoSQL
9 2 (2.44%) Power Query
9 2 (2.44%) Relational Database
10 1 (1.22%) ClickHouse
10 1 (1.22%) Data Hub
Development Applications
1 11 (13.41%) Git
2 5 (6.10%) JIRA
3 1 (1.22%) Deequ
3 1 (1.22%) MLflow
3 1 (1.22%) RStudio
General
1 16 (19.51%) Social Skills
2 15 (18.29%) Finance
3 11 (13.41%) Analytical Skills
4 9 (10.98%) Public Sector
5 4 (4.88%) Financial Institution
5 4 (4.88%) Marketing
5 4 (4.88%) Presentation Skills
6 3 (3.66%) Retail
7 2 (2.44%) Czech Language
8 1 (1.22%) Banking
8 1 (1.22%) Health Technology
8 1 (1.22%) Law
8 1 (1.22%) Legal
8 1 (1.22%) Pharmaceutical
Job Titles
1 52 (63.41%) Data Engineer
2 17 (20.73%) Azure Data Engineer
2 17 (20.73%) Azure Engineer
3 11 (13.41%) Senior
4 9 (10.98%) AWS Engineer
4 9 (10.98%) Data Scientist
4 9 (10.98%) Lead
4 9 (10.98%) Python Engineer
5 8 (9.76%) Developer
5 8 (9.76%) Senior Data Engineer
6 7 (8.54%) AWS Data Engineer
6 7 (8.54%) Python Data Engineer
7 4 (4.88%) Analyst
7 4 (4.88%) Architect
7 4 (4.88%) Data Analyst
7 4 (4.88%) Data Architect
7 4 (4.88%) Data Modeller
8 3 (3.66%) Artificial Intelligence Engineer
8 3 (3.66%) Python Developer
8 3 (3.66%) Senior Data Scientist
Libraries, Frameworks & Software Standards
1 9 (10.98%) Kafka
2 6 (7.32%) Apache Avro
3 3 (3.66%) PyTorch
3 3 (3.66%) TensorFlow
4 2 (2.44%) JSON
4 2 (2.44%) JSON Schema
4 2 (2.44%) NumPy
4 2 (2.44%) Pandas
4 2 (2.44%) Spark Streaming
5 1 (1.22%) ARM Templates
5 1 (1.22%) Great Expectations
5 1 (1.22%) Middleware
5 1 (1.22%) pytest
5 1 (1.22%) Web Services
Miscellaneous
1 5 (6.10%) Distributed Computing
2 3 (3.66%) Team-Oriented Environment
3 2 (2.44%) Algorithms
3 2 (2.44%) Greenfield Project
3 2 (2.44%) Self-Motivation
4 1 (1.22%) Data Protection Act
4 1 (1.22%) Data Structures
4 1 (1.22%) Marketing Technology
4 1 (1.22%) NHS
4 1 (1.22%) Product Ownership
4 1 (1.22%) Smart Meter
Operating Systems
1 2 (2.44%) Linux
Processes & Methodologies
1 33 (40.24%) Data Pipeline
2 32 (39.02%) Analytics
2 32 (39.02%) Data Engineering
3 31 (37.80%) Agile
4 29 (35.37%) Data Modelling
4 29 (35.37%) ETL
5 20 (24.39%) CI/CD
6 18 (21.95%) DevOps
7 16 (19.51%) Business Intelligence
8 14 (17.07%) Data Science
9 12 (14.63%) Software Engineering
10 11 (13.41%) Problem-Solving
11 10 (12.20%) Data Transformation
12 9 (10.98%) Continuous Integration
12 9 (10.98%) SDLC
12 9 (10.98%) Stakeholder Management
12 9 (10.98%) Visualisation
13 8 (9.76%) Machine Learning
14 7 (8.54%) Unit Testing
14 7 (8.54%) Use Case
Programming Languages
1 75 (91.46%) Python
2 41 (50.00%) SQL
3 14 (17.07%) Scala
4 5 (6.10%) Java
5 3 (3.66%) Bash
5 3 (3.66%) R
6 1 (1.22%) C++
6 1 (1.22%) Data Analysis Expressions
6 1 (1.22%) T-SQL
Qualifications
1 7 (8.54%) SC Cleared
1 7 (8.54%) Security Cleared
2 2 (2.44%) Azure Certification
2 2 (2.44%) Microsoft Certification
3 1 (1.22%) AWS Certification
3 1 (1.22%) AWS Certified Solutions Architect
Quality Assurance & Compliance
1 14 (17.07%) Data Quality
2 1 (1.22%) QA
System Software
1 2 (2.44%) Docker
Systems Management
1 7 (8.54%) Terraform
2 3 (3.66%) Kubernetes
Vendors
1 40 (48.78%) Databricks
2 7 (8.54%) Microsoft
3 6 (7.32%) Ab Initio
4 3 (3.66%) Oracle
5 1 (1.22%) Bloomberg
5 1 (1.22%) Foundry
5 1 (1.22%) Palantir
5 1 (1.22%) SAP