PySpark Contracts with Work from Home Options

PySpark - Spark Python API
UK > Work from Home

The table below provides summary statistics for contract job vacancies with an option to work from home (WFH) and a requirement for PySpark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited PySpark over the 6 months to 9 May 2024 with a comparison to the same period in the previous 2 years.

6 months to
9 May 2024
Same period 2023 Same period 2022
Rank 259 342 371
Rank change year-on-year +83 +29 -13
Contract jobs citing PySpark 82 126 259
As % of all contract jobs with a WFH option 0.54% 0.50% 0.59%
As % of the Libraries, Frameworks & Software Standards category 2.26% 1.99% 2.00%
Number of daily rates quoted 70 91 159
10th Percentile £405 £450 £463
25th Percentile £475 £513 £512
Median daily rate (50th Percentile) £500 £613 £575
Median % change year-on-year -18.37% +6.52% +9.52%
75th Percentile £575 £750 £691
90th Percentile £625 £880 £790
UK median daily rate £525 £600 £600
% change year-on-year -12.50% - +14.29%
Number of hourly rates quoted 1 3 1
10th Percentile - £37.50 -
25th Percentile £20.50 £39.75 -
Median hourly rate £21.00 £43.50 £78.75
Median % change year-on-year -51.72% -44.76% +221.43%
75th Percentile £21.50 £43.57 -
90th Percentile - £43.60 -
UK median hourly rate £20.47 £43.50 £78.75
% change year-on-year -52.94% -44.76% +215.00%

All Software Libraries and Frameworks
Work from Home

PySpark is in the Software Libraries and Frameworks category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a WFH option and a requirement for technical specification, industry standards, software libraries and framework skills.

Contract vacancies with a requirement for technical specification, industry standards, software libraries and framework skills 3,636 6,344 12,982
As % of all contract jobs with a WFH option 24.12% 25.29% 29.59%
Number of daily rates quoted 2,551 4,723 9,580
10th Percentile £331 £399 £400
25th Percentile £430 £450 £450
Median daily rate (50th Percentile) £525 £540 £550
Median % change year-on-year -2.78% -1.82% +12.82%
75th Percentile £650 £656 £638
90th Percentile £788 £750 £730
UK median daily rate £534 £550 £550
% change year-on-year -2.91% - +10.00%
Number of hourly rates quoted 93 80 116
10th Percentile £24.45 £22.50 £21.00
25th Percentile - £43.60 £35.00
Median hourly rate £43.00 £62.50 £49.00
Median % change year-on-year -31.20% +27.55% +8.89%
75th Percentile £62.00 £75.00 £60.31
90th Percentile £72.50 £80.25 £67.65
UK median hourly rate £47.50 £65.00 £49.75
% change year-on-year -26.92% +30.65% +10.56%

PySpark
Trend for Jobs with a WFH Option

Job vacancies with a work from home option citing PySpark as a proportion of all IT jobs advertised.

PySpark trend for jobs with a WFH option

PySpark
Daily Rate Trend for Jobs with a WFH Option

3-month moving average daily rate quoted in job vacancies with a work from home option citing PySpark.

PySpark daily rate for jobs with a WFH option

PySpark
Daily Rate Histogram for Jobs with a WFH Option

Daily rate distribution for jobs with a work from home option citing PySpark over the 6 months to 9 May 2024.

PySpark daily rate histogram for jobs with a WFH option

PySpark
Hourly Rate Trend for Jobs with a WFH Option

3-month moving average hourly rate quoted in job vacancies with a work from home option citing PySpark.

PySpark hourly rate for jobs with a WFH option

PySpark
Co-occurring Skills and Capabilities with WFH Options by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 17 (20.73%) Apache Spark
2 15 (18.29%) Apache Airflow
3 8 (9.76%) Apache
Business Applications
1 1 (1.22%) Dynamics CRM
Cloud Services
1 41 (50.00%) Azure
2 29 (35.37%) AWS
3 19 (23.17%) Azure Data Factory
4 18 (21.95%) Serverless
5 15 (18.29%) AWS Lambda
6 13 (15.85%) dbt
7 11 (13.41%) Azure Synapse Analytics
8 10 (12.20%) Amazon EMR
8 10 (12.20%) Amazon Kinesis
8 10 (12.20%) Amazon S3
9 8 (9.76%) Power Platform
9 8 (9.76%) Snowflake
10 6 (7.32%) Amazon EventBridge
10 6 (7.32%) AWS Step Functions
10 6 (7.32%) Azure Logic Apps
10 6 (7.32%) Azure Monitor
10 6 (7.32%) GCP
11 5 (6.10%) Amazon CloudWatch
11 5 (6.10%) Azure Cosmos DB
12 4 (4.88%) Azure Key Vault
Database & Business Intelligence
1 29 (35.37%) Data Lake
2 28 (34.15%) Data Warehouse
3 25 (30.49%) Big Data
4 16 (19.51%) Amazon Redshift
5 10 (12.20%) Delta Lake
6 8 (9.76%) Amazon Athena
6 8 (9.76%) Power BI
7 6 (7.32%) Azure SQL Database
7 6 (7.32%) Parquet
7 6 (7.32%) Spark SQL
8 3 (3.66%) Snowflake Schema
8 3 (3.66%) SQL Server
9 2 (2.44%) Apache Hive
9 2 (2.44%) Data Hub
9 2 (2.44%) Hadoop
9 2 (2.44%) Metadata
9 2 (2.44%) NoSQL
9 2 (2.44%) Power Query
9 2 (2.44%) Relational Database
9 2 (2.44%) Tableau
Development Applications
1 12 (14.63%) Git
2 6 (7.32%) JIRA
3 1 (1.22%) Deequ
3 1 (1.22%) MLflow
3 1 (1.22%) RStudio
General
1 17 (20.73%) Social Skills
2 15 (18.29%) Finance
3 11 (13.41%) Analytical Skills
4 9 (10.98%) Public Sector
5 4 (4.88%) Financial Institution
5 4 (4.88%) Marketing
5 4 (4.88%) Presentation Skills
6 3 (3.66%) Retail
7 2 (2.44%) Czech Language
8 1 (1.22%) Banking
8 1 (1.22%) Health Technology
8 1 (1.22%) Law
8 1 (1.22%) Legal
8 1 (1.22%) Pharmaceutical
Job Titles
1 52 (63.41%) Data Engineer
2 16 (19.51%) Azure Data Engineer
2 16 (19.51%) Azure Engineer
3 12 (14.63%) AWS Engineer
4 10 (12.20%) AWS Data Engineer
5 9 (10.98%) Data Scientist
5 9 (10.98%) Lead
5 9 (10.98%) Senior
6 8 (9.76%) Developer
6 8 (9.76%) Python Engineer
7 6 (7.32%) Python Data Engineer
7 6 (7.32%) Senior Data Engineer
8 4 (4.88%) Analyst
8 4 (4.88%) Architect
8 4 (4.88%) Data Analyst
8 4 (4.88%) Data Architect
8 4 (4.88%) Data Modeller
9 3 (3.66%) Artificial Intelligence Engineer
9 3 (3.66%) Lead Data Engineer
9 3 (3.66%) Python Developer
Libraries, Frameworks & Software Standards
1 10 (12.20%) Kafka
2 6 (7.32%) Apache Avro
3 3 (3.66%) PyTorch
3 3 (3.66%) TensorFlow
4 2 (2.44%) JSON
4 2 (2.44%) JSON Schema
4 2 (2.44%) NumPy
4 2 (2.44%) Pandas
4 2 (2.44%) Spark Streaming
5 1 (1.22%) ARM Templates
5 1 (1.22%) Great Expectations
5 1 (1.22%) Middleware
5 1 (1.22%) Web Services
Miscellaneous
1 5 (6.10%) Distributed Computing
2 3 (3.66%) Team-Oriented Environment
3 2 (2.44%) Algorithms
3 2 (2.44%) Greenfield Project
3 2 (2.44%) Self-Motivation
4 1 (1.22%) Data Protection Act
4 1 (1.22%) Data Structures
4 1 (1.22%) IoT
4 1 (1.22%) Marketing Technology
4 1 (1.22%) NHS
4 1 (1.22%) Product Ownership
Operating Systems
1 2 (2.44%) Linux
Processes & Methodologies
1 35 (42.68%) Data Pipeline
2 33 (40.24%) Data Engineering
3 32 (39.02%) Agile
3 32 (39.02%) Analytics
4 30 (36.59%) Data Modelling
5 29 (35.37%) ETL
6 18 (21.95%) CI/CD
7 17 (20.73%) Business Intelligence
7 17 (20.73%) DevOps
8 15 (18.29%) Data Science
9 13 (15.85%) Software Engineering
10 12 (14.63%) Problem-Solving
11 10 (12.20%) Visualisation
12 9 (10.98%) SDLC
12 9 (10.98%) Stakeholder Management
13 8 (9.76%) Continuous Integration
13 8 (9.76%) Data Transformation
13 8 (9.76%) Machine Learning
13 8 (9.76%) Use Case
14 7 (8.54%) Unit Testing
Programming Languages
1 75 (91.46%) Python
2 42 (51.22%) SQL
3 15 (18.29%) Scala
4 5 (6.10%) Java
5 4 (4.88%) R
6 3 (3.66%) Bash
7 1 (1.22%) C++
7 1 (1.22%) Data Analysis Expressions
7 1 (1.22%) T-SQL
Qualifications
1 7 (8.54%) SC Cleared
1 7 (8.54%) Security Cleared
2 2 (2.44%) AWS Certification
2 2 (2.44%) AWS Certified Solutions Architect
2 2 (2.44%) Azure Certification
2 2 (2.44%) Microsoft Certification
Quality Assurance & Compliance
1 15 (18.29%) Data Quality
2 1 (1.22%) QA
System Software
1 3 (3.66%) Docker
Systems Management
1 8 (9.76%) Terraform
2 4 (4.88%) Kubernetes
Vendors
1 37 (45.12%) Databricks
2 7 (8.54%) Microsoft
3 6 (7.32%) Ab Initio
4 3 (3.66%) Oracle
5 2 (2.44%) Bloomberg
6 1 (1.22%) Foundry
6 1 (1.22%) Palantir
6 1 (1.22%) SAP