Period
to

The following table provides summary statistics for contract job vacancies with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 28 June 2022 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
UK
6 months to
28 Jun 2022
Same period 2021 Same period 2020
Rank 230 188 196
Rank change year-on-year -42 +8 -36
Contract jobs citing Apache Spark 1,069 936 567
As % of all contract jobs advertised in the UK 1.21% 1.46% 1.39%
As % of the Application Platforms category 10.29% 11.46% 9.99%
Number of daily rates quoted 731 582 401
10th Percentile £450 £425 £413
25th Percentile £520 £494 £463
Median daily rate (50th Percentile) £600 £575 £525
Median % change year-on-year +4.35% +9.52% -4.55%
75th Percentile £738 £645 £650
90th Percentile £825 £713 £688
UK excluding London median daily rate £550 £500 £500
% change year-on-year +10.00% - -2.91%
Number of hourly rates quoted 3 5 1
10th Percentile £62.35 £15.95 -
25th Percentile £68.50 £18.50 £17.50
Median hourly rate £78.75 £24.50 £20.00
Median % change year-on-year +221.43% +22.50% -
75th Percentile £84.83 £27.00 £22.50
90th Percentile £88.47 £34.05 -
UK excluding London median hourly rate - £19.50 £20.00
% change year-on-year - -2.50% -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for application platform skills.

All Application Platform Skills
UK
Contract vacancies with a requirement for application platform skills 10,388 8,170 5,678
As % of all contract IT jobs advertised in the UK 11.80% 12.71% 13.89%
Number of daily rates quoted 7,137 5,507 3,781
10th Percentile £313 £300 £288
25th Percentile £419 £400 £375
Median daily rate (50th Percentile) £525 £490 £475
Median % change year-on-year +7.25% +3.05% -
75th Percentile £626 £575 £550
90th Percentile £734 £663 £650
UK excluding London median daily rate £475 £448 £450
% change year-on-year +5.97% -0.39% +5.88%
Number of hourly rates quoted 159 161 121
10th Percentile £14.25 £12.25 £12.63
25th Percentile £16.75 £16.18 £16.00
Median hourly rate £35.00 £24.00 £20.00
Median % change year-on-year +45.83% +20.00% +0.98%
75th Percentile £65.48 £51.00 £33.75
90th Percentile £77.50 £65.54 £54.00
UK excluding London median hourly rate £22.25 £22.21 £19.67
% change year-on-year +0.20% +12.89% +14.66%

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a proportion of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Contractor Daily Rate Trend

3-month moving average daily rate quoted in jobs citing Apache Spark.

Daily rate trend for Apache Spark in the UK

Apache Spark
Daily Rate Histogram

Daily rate distribution for jobs citing Apache Spark over the 6 months to 28 June 2022.

Daily rate histogram for Apache Spark in the UK

Apache Spark
Contractor Hourly Rate Trend

3-month moving average hourly rates quoted in jobs citing Apache Spark.

Hourly rate trend for Apache Spark in the UK

Apache Spark
Hourly Rate Histogram

Hourly rate distribution of jobs citing Apache Spark over the 6 months to 28 June 2022.

Hourly rate histogram for Apache Spark in the UK

Apache Spark
Top 16 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the UK over the 6 months to 28 June 2022. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Jobs
England -65 803 £600 +4.35% 174
Work from Home +18 626 £600 +8.11% 86
London -45 616 £625 +7.30% 113
UK excluding London -82 168 £550 +10.00% 65
South East -59 39 £625 +19.05% 15
East of England +4 38 £563 +40.63% 8
North of England -71 27 £550 -4.35% 25
Scotland -6 26 £590 +27.57% 4
Midlands -39 17 £500 +11.11% 4
Yorkshire -38 16 £550 +4.41% 12
North West -39 11 £563 -11.42% 13
South West -39 11 £575 +15.00% 5
West Midlands -36 10 £519 +18.57% 3
East Midlands -10 7 £438 -18.60% 1
Wales -13 7 £546 +15.00% 1
Northern Ireland - 5 £475 - 3

For the 6 months to 28 June 2022, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for Apache Spark.

1 677 (63.33%) Python
2 544 (50.89%) Big Data
3 489 (45.74%) Kafka
4 439 (41.07%) Azure
5 429 (40.13%) SQL
6 424 (39.66%) Java
7 411 (38.45%) Agile
8 402 (37.61%) AWS
9 397 (37.14%) Scala
10 383 (35.83%) Hadoop
11 364 (34.05%) Analytics
12 329 (30.78%) Data Engineering
13 321 (30.03%) CI/CD
14 292 (27.32%) DevOps
15 291 (27.22%) Apache
16 286 (26.75%) ETL
17 265 (24.79%) GCP
18 259 (24.23%) Databricks
19 253 (23.67%) Finance
20 246 (23.01%) Data Pipeline
21 234 (21.89%) Data Science
22 215 (20.11%) Data Modelling
23 212 (19.83%) Kubernetes
24 210 (19.64%) Data Warehouse
25 207 (19.36%) Machine Learning
26 206 (19.27%) NoSQL
27 202 (18.90%) Data Lake
28 176 (16.46%) Apache Hive
28 176 (16.46%) Git (software)
29 170 (15.90%) Software Engineering

Apache Spark
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 291 (27.22%) Apache
2 117 (10.94%) Apache Airflow
3 31 (2.90%) nginx
4 25 (2.34%) Jupyter
5 24 (2.25%) Apache Solr
6 21 (1.96%) Apache Pig
7 19 (1.78%) Confluence
8 14 (1.31%) SAS
9 12 (1.12%) Luigi
10 11 (1.03%) SharePoint
11 10 (0.94%) Oracle Coherence
11 10 (0.94%) uWSGI
12 6 (0.56%) WebLogic
13 5 (0.47%) Apache Pulsar
13 5 (0.47%) JBoss
13 5 (0.47%) Tomcat
13 5 (0.47%) WebSphere
14 4 (0.37%) OpenStack
14 4 (0.37%) Oracle SOA Suite
15 3 (0.28%) Oracle Service Bus
Applications
1 9 (0.84%) SPSS
2 5 (0.47%) Microsoft Excel
3 4 (0.37%) Microsoft Office
3 4 (0.37%) Microsoft PowerPoint
4 3 (0.28%) MS Visio
Business Applications
1 5 (0.47%) SAP Oil and Gas
2 3 (0.28%) Oracle BPM
3 1 (0.094%) Dynamics AX
3 1 (0.094%) Dynamics CRM
3 1 (0.094%) SAP S/4HANA
Cloud Services
1 439 (41.07%) Azure
2 402 (37.61%) AWS
3 265 (24.79%) GCP
4 114 (10.66%) Serverless
5 110 (10.29%) Azure Data Factory
6 91 (8.51%) Amazon S3
7 79 (7.39%) Azure DevOps
8 78 (7.30%) Power Platform
9 63 (5.89%) Amazon EMR
10 59 (5.52%) AWS Lambda
10 59 (5.52%) Azure Synapse Analytics
11 54 (5.05%) Azure Functions
12 53 (4.96%) GitHub
13 46 (4.30%) Azure AKS
14 44 (4.12%) dbt
15 42 (3.93%) Azure Cosmos DB
16 37 (3.46%) Amazon EC2
16 37 (3.46%) Snowflake
17 33 (3.09%) Azure Monitor
18 25 (2.34%) Amazon Kinesis
Communications & Networking
1 10 (0.94%) Kerberos
2 8 (0.75%) SSH
3 7 (0.65%) Firewall
3 7 (0.65%) FTP
4 4 (0.37%) HTTP
5 3 (0.28%) Internet
6 2 (0.19%) Radio Access Network
7 1 (0.094%) DNS
7 1 (0.094%) IPFS
7 1 (0.094%) SMTP
7 1 (0.094%) TCP/IP
7 1 (0.094%) VPN
Database & Business Intelligence
1 544 (50.89%) Big Data
2 383 (35.83%) Hadoop
3 210 (19.64%) Data Warehouse
4 206 (19.27%) NoSQL
5 202 (18.90%) Data Lake
6 176 (16.46%) Apache Hive
7 93 (8.70%) BigQuery
8 85 (7.95%) Flink
9 84 (7.86%) HBase
10 77 (7.20%) Power BI
10 77 (7.20%) Relational Database
11 75 (7.02%) SQL Server
12 74 (6.92%) Apache Cassandra
13 71 (6.64%) Metadata
13 71 (6.64%) MongoDB
14 62 (5.80%) Amazon Redshift
15 54 (5.05%) Tableau
16 53 (4.96%) Elasticsearch
16 53 (4.96%) Pentaho
17 52 (4.86%) Delta Lake
Development Applications
1 176 (16.46%) Git (software)
2 91 (8.51%) Jenkins
3 71 (6.64%) GitLab
4 68 (6.36%) JIRA
5 64 (5.99%) Maven
6 37 (3.46%) Gradle
7 34 (3.18%) MLflow
8 27 (2.53%) TeamCity
9 25 (2.34%) Sonatype Nexus
10 17 (1.59%) CircleCI
11 16 (1.50%) ERwin
12 9 (0.84%) Artifactory
12 9 (0.84%) Bitbucket
12 9 (0.84%) ER Studio
12 9 (0.84%) Gerrit
13 8 (0.75%) IntelliJ
13 8 (0.75%) kappa
13 8 (0.75%) Subversion
14 6 (0.56%) JUnit
14 6 (0.56%) Mockito
General
1 253 (23.67%) Finance
2 148 (13.84%) Social Skills
3 139 (13.00%) Banking
4 131 (12.25%) Analytical Skills
5 44 (4.12%) Public Sector
6 41 (3.84%) Investment Banking
7 27 (2.53%) Publishing
8 23 (2.15%) Financial Institution
8 23 (2.15%) Legal
8 23 (2.15%) Marketing
9 18 (1.68%) Pharmaceutical
10 17 (1.59%) Retail
11 16 (1.50%) Presentation Skills
12 15 (1.40%) Law
13 10 (0.94%) Front Office
13 10 (0.94%) Telecoms
14 9 (0.84%) Documentation Skills
15 5 (0.47%) Games
16 3 (0.28%) Aerospace
16 3 (0.28%) Manufacturing
Job Titles
1 505 (47.24%) Data Engineer
2 157 (14.69%) Developer
3 119 (11.13%) Architect
4 101 (9.45%) Big Data Engineer
5 69 (6.45%) Azure Engineer
6 67 (6.27%) Data Architect
7 65 (6.08%) Python Engineer
8 64 (5.99%) Machine Learning Engineer
9 59 (5.52%) Senior Data Engineer
10 49 (4.58%) Big Data Developer
10 49 (4.58%) Java Engineer
11 48 (4.49%) Data Scientist
11 48 (4.49%) GCP Engineer
12 47 (4.40%) Azure Data Engineer
13 46 (4.30%) GCP Data Engineer
14 45 (4.21%) AWS Engineer
15 44 (4.12%) Cloud Engineer
16 39 (3.65%) DevOps Engineer
17 37 (3.46%) Solutions Architect
18 34 (3.18%) Python Developer
Libraries, Frameworks & Software Standards
1 489 (45.74%) Kafka
2 156 (14.59%) PySpark
3 80 (7.48%) Spark Streaming
4 58 (5.43%) TensorFlow
5 57 (5.33%) Pandas
6 54 (5.05%) Apache Beam
7 46 (4.30%) Apache NiFi
8 38 (3.55%) React
9 36 (3.37%) PyTorch
10 34 (3.18%) Spring
11 31 (2.90%) REST
11 31 (2.90%) scikit-learn
12 30 (2.81%) NumPy
13 29 (2.71%) Flask
14 27 (2.53%) BentoML
14 27 (2.53%) CSS
14 27 (2.53%) Django
14 27 (2.53%) RESTful
15 21 (1.96%) HTML
16 20 (1.87%) Node.js
Miscellaneous
1 81 (7.58%) Algorithms
2 59 (5.52%) Data Structures
3 55 (5.14%) Cloud Native
4 45 (4.21%) Public Cloud
5 36 (3.37%) Linux Command Line
6 32 (2.99%) iPad
7 26 (2.43%) Self-Motivation
8 24 (2.25%) Greenfield Project
9 19 (1.78%) Hedge funds
10 18 (1.68%) Legacy Systems
11 17 (1.59%) Equities
12 16 (1.50%) Virtual Team
13 15 (1.40%) Management Information System
13 15 (1.40%) Market Risk
14 14 (1.31%) Enterprise Software
15 12 (1.12%) Data Protection Act
16 9 (0.84%) Derivative
17 8 (0.75%) Data Centre
17 8 (0.75%) IoT
17 8 (0.75%) Replication
Operating Systems
1 157 (14.69%) Linux
2 48 (4.49%) Unix
3 18 (1.68%) Windows
4 15 (1.40%) Ubuntu
5 8 (0.75%) Red Hat Enterprise Linux
6 6 (0.56%) Debian
7 3 (0.28%) Android
7 3 (0.28%) CentOS
8 2 (0.19%) Apple iOS
8 2 (0.19%) VMS
Processes & Methodologies
1 411 (38.45%) Agile
2 364 (34.05%) Analytics
3 329 (30.78%) Data Engineering
4 321 (30.03%) CI/CD
5 292 (27.32%) DevOps
6 286 (26.75%) ETL
7 246 (23.01%) Data Pipeline
8 234 (21.89%) Data Science
9 215 (20.11%) Data Modelling
10 207 (19.36%) Machine Learning
11 170 (15.90%) Software Engineering
12 137 (12.82%) Business Intelligence
13 126 (11.79%) Microservices
14 123 (11.51%) TDD
15 114 (10.66%) Scrum
16 99 (9.26%) Migration
17 97 (9.07%) Visualisation
18 96 (8.98%) Computer Science
19 91 (8.51%) Problem-Solving
20 83 (7.76%) Artificial Intelligence
Programming Languages
1 677 (63.33%) Python
2 429 (40.13%) SQL
3 424 (39.66%) Java
4 397 (37.14%) Scala
5 93 (8.70%) R
6 54 (5.05%) C#
7 49 (4.58%) C++
8 48 (4.49%) Shell Script
9 42 (3.93%) Bash
10 38 (3.55%) Go
11 37 (3.46%) JavaScript
11 37 (3.46%) PowerShell
12 26 (2.43%) PHP
13 22 (2.06%) TypeScript
14 18 (1.68%) Ruby
15 17 (1.59%) C
16 15 (1.40%) T-SQL
17 12 (1.12%) Perl
18 8 (0.75%) Data Analysis Expressions (DAX)
18 8 (0.75%) PL/SQL
Qualifications
1 114 (10.66%) Security Cleared
2 99 (9.26%) SC Cleared
3 56 (5.24%) Degree
4 47 (4.40%) PhD
5 24 (2.25%) Computer Science Degree
6 15 (1.40%) FFIEC
7 14 (1.31%) PMP
8 11 (1.03%) Master's Degree
9 9 (0.84%) PMI Certification
10 8 (0.75%) DV Cleared
11 6 (0.56%) BPSS Clearance
12 4 (0.37%) AWS Certification
12 4 (0.37%) CCA
13 1 (0.094%) DBS Check
13 1 (0.094%) Microsoft Certification
13 1 (0.094%) Sun Certification
Quality Assurance & Compliance
1 61 (5.71%) Data Quality
2 24 (2.25%) QA
3 21 (1.96%) GDPR
4 15 (1.40%) GLBA
4 15 (1.40%) NIST
4 15 (1.40%) PCI DSS
5 12 (1.12%) SLA
6 3 (0.28%) IFRS
6 3 (0.28%) IFRS 17
7 1 (0.094%) PMO
System Software
1 153 (14.31%) Docker
2 94 (8.79%) HDFS
3 15 (1.40%) Apache Flume
4 13 (1.22%) Apache ZooKeeper
5 11 (1.03%) Active Directory
6 4 (0.37%) Virtual Machines
7 3 (0.28%) VMware Infrastructure
8 1 (0.094%) Tivoli Workload Scheduler
Systems Management
1 212 (19.83%) Kubernetes
2 114 (10.66%) Terraform
3 34 (3.18%) Argo
4 27 (2.53%) Kubeflow
5 26 (2.43%) Ansible
5 26 (2.43%) Hadoop YARN
6 19 (1.78%) Oozie
7 15 (1.40%) Grafana
8 12 (1.12%) Kibana
9 10 (0.94%) Istio
10 9 (0.84%) logstash
11 7 (0.65%) Puppet
12 6 (0.56%) Prometheus
13 5 (0.47%) Opscode Chef
14 4 (0.37%) Active Directory Federation Services
15 2 (0.19%) Ambari
15 2 (0.19%) Control-M
15 2 (0.19%) HP Fortify
15 2 (0.19%) Salt
16 1 (0.094%) Tivoli
Vendors
1 259 (24.23%) Databricks
2 95 (8.89%) Talend
3 90 (8.42%) Google
4 88 (8.23%) Cloudera
5 87 (8.14%) Microsoft
6 75 (7.02%) Oracle
7 31 (2.90%) SAP
8 30 (2.81%) Informatica
9 28 (2.62%) Salesforce.com
10 27 (2.53%) Hortonworks
11 20 (1.87%) Splunk
12 18 (1.68%) Alteryx
13 17 (1.59%) Netezza
14 16 (1.50%) AppDynamics
15 13 (1.22%) Teradata
16 12 (1.12%) Salesforce Marketing Cloud
17 11 (1.03%) Red Hat
18 10 (0.94%) IBM
19 9 (0.84%) MapR
19 9 (0.84%) StreamSets