Period
to 18 January 2022

The following table provides summary statistics for contract job vacancies with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 18 January 2022 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
UK
6 months to
18 Jan 2022
Same period 2021 Same period 2020
Rank 190 194 216
Rank change year-on-year +4 +22 -12
Contract jobs citing Apache Spark 1,279 566 916
As % of all contract jobs advertised in the UK 1.46% 1.39% 1.27%
As % of the Application Platforms category 11.82% 9.64% 8.81%
Number of daily rates quoted 802 396 655
10th Percentile £439 £394 £419
25th Percentile £513 £461 £475
Median daily rate (50th Percentile) £600 £550 £550
Median % change year-on-year +9.09% - -
75th Percentile £695 £625 £639
90th Percentile £800 £700 £725
UK excluding London median daily rate £500 £450 £525
% change year-on-year +11.11% -14.29% -
Number of hourly rates quoted 7 7 1
10th Percentile £22.59 £9.75 -
25th Percentile £26.49 £12.75 -
Median hourly rate £55.00 £22.38 £24.65
Median % change year-on-year +145.76% -9.21% -45.22%
75th Percentile £65.00 £24.41 -
90th Percentile £75.36 £46.04 -
UK excluding London median hourly rate £27.73 £25.05 £24.65
% change year-on-year +10.72% +1.60% -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for application platform skills.

All Application Platform Skills
UK
Contract vacancies with a requirement for application platform skills 10,818 5,869 10,392
As % of all contract IT jobs advertised in the UK 12.37% 14.45% 14.38%
Number of daily rates quoted 7,453 3,984 6,908
10th Percentile £313 £275 £250
25th Percentile £413 £363 £363
Median daily rate (50th Percentile) £520 £475 £450
Median % change year-on-year +9.47% +5.56% -4.26%
75th Percentile £625 £565 £550
90th Percentile £700 £650 £638
UK excluding London median daily rate £450 £430 £425
% change year-on-year +4.65% +1.18% -
Number of hourly rates quoted 171 174 229
10th Percentile £14.25 £13.68 £10.92
25th Percentile £16.68 £18.75 £14.25
Median hourly rate £24.36 £26.73 £20.00
Median % change year-on-year -8.87% +33.65% +7.38%
75th Percentile £51.00 £56.78 £36.70
90th Percentile £65.54 £64.50 £51.75
UK excluding London median hourly rate £19.00 £24.00 £18.25
% change year-on-year -20.83% +31.51% +4.29%

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a proportion of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Contractor Daily Rate Trend

3-month moving average daily rate quoted in jobs citing Apache Spark.

Daily rate trend for Apache Spark in the UK

Apache Spark
Daily Rate Histogram

Daily rate distribution for jobs citing Apache Spark over the 6 months to 18 January 2022.

Daily rate histogram for Apache Spark in the UK

Apache Spark
Contractor Hourly Rate Trend

3-month moving average hourly rates quoted in jobs citing Apache Spark.

Hourly rate trend for Apache Spark in the UK

Apache Spark
Hourly Rate Histogram

Hourly rate distribution of jobs citing Apache Spark over the 6 months to 18 January 2022.

Hourly rate histogram for Apache Spark in the UK

Apache Spark
Top 17 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the UK over the 6 months to 18 January 2022. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -8 999 £603 +8.56% 171
London -7 744 £625 +8.74% 103
Work from Home +19 418 £600 +9.09% 34
UK excluding London -17 245 £500 +11.11% 72
South East -41 74 £500 +2.56% 20
North of England -29 65 £530 +22.97% 22
Yorkshire -15 40 £520 -6.31% 11
Scotland -25 34 £500 +21.95% 8
Midlands -46 25 £500 +32.80% 10
Wales -12 23 £600 +4.35%
South West -45 17 £525 -4.55% 4
North West -57 14 £553 +30.00% 9
West Midlands -51 13 £450 +20.00% 6
North East - 11 £475 - 2
East Midlands -27 10 £500 +32.28% 4
East of England -62 8 £750 +87.50% 8
Northern Ireland -4 3 £475 +18.75%

For the 6 months to 18 January 2022, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for Apache Spark.

1 703 (54.96%) Python
2 682 (53.32%) Big Data
3 561 (43.86%) Kafka
4 560 (43.78%) Hadoop
5 552 (43.16%) SQL
6 501 (39.17%) AWS
7 412 (32.21%) Java
8 411 (32.13%) Azure
9 408 (31.90%) Scala
10 404 (31.59%) Agile Software Development
11 370 (28.93%) Analytics
12 331 (25.88%) Data Modelling
13 328 (25.65%) Data Engineering
14 316 (24.71%) DevOps
15 313 (24.47%) Apache Hive
16 305 (23.85%) Data Pipeline
17 288 (22.52%) ETL
18 264 (20.64%) Apache
19 260 (20.33%) Data Warehouse
20 255 (19.94%) Finance
21 247 (19.31%) NoSQL
22 238 (18.61%) Data Science
23 235 (18.37%) Kubernetes
24 220 (17.20%) Git (software)
24 220 (17.20%) Social Skills
25 217 (16.97%) Docker
26 215 (16.81%) PySpark
27 214 (16.73%) Databricks
27 214 (16.73%) Jenkins
28 209 (16.34%) Data Lake

Apache Spark
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 264 (20.64%) Apache
2 124 (9.70%) Apache Airflow
3 73 (5.71%) Confluence
4 64 (5.00%) Elasticsearch
5 32 (2.50%) Jupyter
6 27 (2.11%) Luigi
7 22 (1.72%) SharePoint
8 14 (1.09%) Apache Pig
8 14 (1.09%) SAS
8 14 (1.09%) Tomcat
9 6 (0.47%) Apache Pulsar
9 6 (0.47%) Camunda
10 3 (0.23%) Apache Solr
10 3 (0.23%) Apache Synapse
10 3 (0.23%) Mule
10 3 (0.23%) OpenStack
11 2 (0.16%) Oracle Coherence
12 1 (0.078%) Adobe Experience Manager
12 1 (0.078%) ArcGIS
12 1 (0.078%) nginx
Applications
1 9 (0.70%) Microsoft Excel
2 5 (0.39%) Microsoft PowerPoint
3 4 (0.31%) MS Visio
3 4 (0.31%) SPSS
4 1 (0.078%) Chatbot
4 1 (0.078%) Microsoft Office
4 1 (0.078%) Microsoft Project
4 1 (0.078%) SPSS Modeler
Business Applications
1 2 (0.16%) SAP Oil and Gas
1 2 (0.16%) SAP S/4HANA
2 1 (0.078%) Adobe Campaign
2 1 (0.078%) SAP R/3
Cloud Services
1 501 (39.17%) AWS
2 411 (32.13%) Azure
3 172 (13.45%) GCP
4 145 (11.34%) Amazon EMR
5 126 (9.85%) Amazon S3
6 117 (9.15%) Serverless
7 113 (8.84%) Azure Data Factory
8 90 (7.04%) Power Platform
9 83 (6.49%) GitHub
10 71 (5.55%) Azure Synapse Analytics
11 64 (5.00%) Snowflake
12 58 (4.53%) Azure Cosmos DB
13 56 (4.38%) Amazon EC2
14 46 (3.60%) AWS Lambda
15 43 (3.36%) AWS CloudFormation
16 41 (3.21%) Azure Functions
17 39 (3.05%) Amazon Kinesis
18 31 (2.42%) Amazon ECS
19 29 (2.27%) dbt
19 29 (2.27%) PaaS
Communications & Networking
1 35 (2.74%) HTTP
2 24 (1.88%) Kerberos
3 16 (1.25%) WAN
4 15 (1.17%) FTP
5 14 (1.09%) DNS
6 13 (1.02%) Network Security
7 8 (0.63%) Firewall
8 5 (0.39%) IPFS
8 5 (0.39%) VPN
9 3 (0.23%) Intranet
10 2 (0.16%) Internet
10 2 (0.16%) LAN
10 2 (0.16%) Multicast
10 2 (0.16%) PuTTY
10 2 (0.16%) Radio Access Network
10 2 (0.16%) SSH
11 1 (0.078%) HTTPS
11 1 (0.078%) SAN
11 1 (0.078%) SSL
11 1 (0.078%) TCP/IP
Database & Business Intelligence
1 682 (53.32%) Big Data
2 560 (43.78%) Hadoop
3 313 (24.47%) Apache Hive
4 260 (20.33%) Data Warehouse
5 247 (19.31%) NoSQL
6 209 (16.34%) Data Lake
7 170 (13.29%) HBase
8 143 (11.18%) Amazon Redshift
9 128 (10.01%) Apache Cassandra
10 108 (8.44%) MongoDB
11 107 (8.37%) Metadata
12 102 (7.97%) Tableau
13 99 (7.74%) Relational Database
14 88 (6.88%) PostgreSQL
14 88 (6.88%) Power BI
15 82 (6.41%) Flink
16 77 (6.02%) Amazon RDS
17 74 (5.79%) SQL Server
18 72 (5.63%) Impala
19 70 (5.47%) BigQuery
Development Applications
1 220 (17.20%) Git (software)
2 214 (16.73%) Jenkins
3 110 (8.60%) JIRA
4 93 (7.27%) GitLab
5 82 (6.41%) Maven
6 58 (4.53%) TeamCity
7 52 (4.07%) Gradle
8 37 (2.89%) Hudson
9 31 (2.42%) ERwin
10 20 (1.56%) Subversion
11 19 (1.49%) MLflow
12 15 (1.17%) CircleCI
12 15 (1.17%) PowerDesigner
13 14 (1.09%) Bitbucket
13 14 (1.09%) IntelliJ
14 12 (0.94%) Artifactory
14 12 (0.94%) ER Studio
15 11 (0.86%) Atlassian Bamboo
15 11 (0.86%) Cucumber
15 11 (0.86%) JUnit
General
1 255 (19.94%) Finance
2 220 (17.20%) Social Skills
3 205 (16.03%) Banking
4 136 (10.63%) Analytical Skills
5 112 (8.76%) Investment Banking
6 62 (4.85%) Public Sector
7 37 (2.89%) Presentation Skills
8 35 (2.74%) Marketing
9 33 (2.58%) Retail
10 31 (2.42%) Financial Institution
11 25 (1.95%) Legal
12 20 (1.56%) Publishing
13 15 (1.17%) Telecoms
14 11 (0.86%) Games
14 11 (0.86%) Law
15 7 (0.55%) Manufacturing
16 5 (0.39%) Aerospace
16 5 (0.39%) Front Office
16 5 (0.39%) Influencing Skills
17 4 (0.31%) German Language
Job Titles
1 612 (47.85%) Data Engineer
2 251 (19.62%) Developer
3 124 (9.70%) Architect
4 87 (6.80%) Big Data Engineer
5 82 (6.41%) AWS Engineer
6 68 (5.32%) Python Engineer
7 67 (5.24%) Azure Engineer
8 65 (5.08%) Cloud Engineer
9 63 (4.93%) Data Architect
10 60 (4.69%) Java Developer
11 59 (4.61%) Scala Developer
12 54 (4.22%) Senior Data Engineer
13 49 (3.83%) DevOps Engineer
14 46 (3.60%) Senior Developer
15 43 (3.36%) Data Scientist
16 38 (2.97%) Apache Spark Developer
16 38 (2.97%) Data Modeller
17 31 (2.42%) Senior Scala Developer
18 29 (2.27%) Big Data Developer
18 29 (2.27%) Scala Engineer
Libraries, Frameworks & Software Standards
1 561 (43.86%) Kafka
2 215 (16.81%) PySpark
3 122 (9.54%) Spark Streaming
4 69 (5.39%) Apache Avro
5 67 (5.24%) Apache NiFi
6 58 (4.53%) Spring
7 46 (3.60%) REST
8 42 (3.28%) Pandas
9 39 (3.05%) JSON
10 33 (2.58%) Spring Boot
11 30 (2.35%) RESTful
12 29 (2.27%) NumPy
13 27 (2.11%) XML
14 26 (2.03%) Akka
15 24 (1.88%) Play Framework
16 23 (1.80%) scikit-learn
16 23 (1.80%) TensorFlow
17 22 (1.72%) Elastic Stack
18 21 (1.64%) ARM Templates
19 20 (1.56%) pytest
Miscellaneous
1 77 (6.02%) Algorithms
2 55 (4.30%) Data Structures
3 35 (2.74%) Greenfield Project
4 30 (2.35%) Cloud Native
5 29 (2.27%) Data Protection Act
6 27 (2.11%) Virtual Team
7 25 (1.95%) iPad
7 25 (1.95%) Public Cloud
8 23 (1.80%) Enterprise Software
9 19 (1.49%) Hedge funds
10 15 (1.17%) Management Information System
10 15 (1.17%) PKI
10 15 (1.17%) Self-Motivation
11 13 (1.02%) Legacy Applications
11 13 (1.02%) Market Risk
12 12 (0.94%) Linux Command Line
13 9 (0.70%) Data Centre
13 9 (0.70%) Hybrid Cloud
13 9 (0.70%) Social Media
13 9 (0.70%) User Experience
Operating Systems
1 127 (9.93%) Linux
2 74 (5.79%) Unix
3 16 (1.25%) Red Hat Enterprise Linux
4 13 (1.02%) Windows
5 3 (0.23%) Debian
5 3 (0.23%) Ubuntu
6 2 (0.16%) Android
6 2 (0.16%) Apple iOS
6 2 (0.16%) CentOS
7 1 (0.078%) DC/OS
7 1 (0.078%) Mesosphere
7 1 (0.078%) Windows Server
7 1 (0.078%) Windows Server 2012
Processes & Methodologies
1 404 (31.59%) Agile Software Development
2 370 (28.93%) Analytics
3 331 (25.88%) Data Modelling
4 328 (25.65%) Data Engineering
5 316 (24.71%) DevOps
6 305 (23.85%) Data Pipeline
7 288 (22.52%) ETL
8 238 (18.61%) Data Science
9 206 (16.11%) Machine Learning
10 204 (15.95%) Business Intelligence
11 135 (10.56%) Scrum
11 135 (10.56%) Visualisation
12 132 (10.32%) Microservices
13 122 (9.54%) Data Integration
14 115 (8.99%) Data Analysis
15 110 (8.60%) Data Architecture
16 108 (8.44%) Problem-Solving
17 105 (8.21%) TDD
18 104 (8.13%) Data Analytics
19 103 (8.05%) Artificial Intelligence
Programming Languages
1 703 (54.96%) Python
2 552 (43.16%) SQL
3 412 (32.21%) Java
4 408 (31.90%) Scala
5 91 (7.11%) R
6 49 (3.83%) JavaScript
7 47 (3.67%) Bash
8 43 (3.36%) C#
8 43 (3.36%) PowerShell
9 38 (2.97%) Shell Script
10 37 (2.89%) C++
11 32 (2.50%) PL/SQL
12 27 (2.11%) C
13 22 (1.72%) Go
14 19 (1.49%) Java 8
15 16 (1.25%) T-SQL
16 11 (0.86%) Perl
17 9 (0.70%) Kotlin
18 8 (0.63%) Groovy
19 7 (0.55%) Java 11
Qualifications
1 134 (10.48%) Security Cleared
2 122 (9.54%) SC Cleared
3 80 (6.25%) Degree
4 39 (3.05%) Computer Science Degree
5 25 (1.95%) AWS Certification
6 16 (1.25%) BPSS Clearance
7 13 (1.02%) DV Cleared
8 6 (0.47%) Numerate Degree
9 5 (0.39%) Master's Degree
10 4 (0.31%) FFIEC
11 3 (0.23%) PhD
11 3 (0.23%) PMI Certification
11 3 (0.23%) PMP
11 3 (0.23%) TOGAF Certification
12 2 (0.16%) DBS Check
13 1 (0.078%) ISEB
13 1 (0.078%) MBA
13 1 (0.078%) Microsoft Certification
Quality Assurance & Compliance
1 110 (8.60%) Data Quality
2 44 (3.44%) GDPR
3 39 (3.05%) QA
4 12 (0.94%) SLA
5 9 (0.70%) IFRS
6 4 (0.31%) GLBA
6 4 (0.31%) NIST
6 4 (0.31%) PCI DSS
6 4 (0.31%) Sarbanes-Oxley
7 3 (0.23%) NCSC
8 2 (0.16%) MiFID
9 1 (0.078%) IFRS 9
9 1 (0.078%) PMO
System Software
1 217 (16.97%) Docker
2 126 (9.85%) HDFS
3 33 (2.58%) Apache Flume
4 17 (1.33%) Active Directory
5 12 (0.94%) Apache ZooKeeper
6 4 (0.31%) Active Directory Lightweight Directory Services
7 2 (0.16%) Virtual Machines
8 1 (0.078%) Hyper-V
8 1 (0.078%) VMware Infrastructure
Systems Management
1 235 (18.37%) Kubernetes
2 160 (12.51%) Terraform
3 71 (5.55%) Ansible
4 39 (3.05%) Oozie
5 28 (2.19%) Prometheus
6 26 (2.03%) Hadoop YARN
7 22 (1.72%) Kibana
8 14 (1.09%) logstash
9 11 (0.86%) Puppet
10 8 (0.63%) Kubeflow
11 7 (0.55%) Control-M
11 7 (0.55%) Grafana
11 7 (0.55%) Single Sign-On
12 5 (0.39%) Argo
12 5 (0.39%) Docker Swarm
13 4 (0.31%) Opscode Chef
14 3 (0.23%) Active Directory Federation Services
14 3 (0.23%) Ambari
14 3 (0.23%) Rancher
15 2 (0.16%) Consul
Vendors
1 214 (16.73%) Databricks
2 168 (13.14%) Cloudera
3 90 (7.04%) Talend
4 89 (6.96%) Hortonworks
5 73 (5.71%) Microsoft
6 67 (5.24%) Google
7 55 (4.30%) Oracle
8 45 (3.52%) Informatica
9 38 (2.97%) Splunk
10 27 (2.11%) Ab Initio
11 23 (1.80%) IBM
11 23 (1.80%) Red Hat
11 23 (1.80%) StreamSets
11 23 (1.80%) Teradata
12 22 (1.72%) MapR
13 20 (1.56%) SAP
14 19 (1.49%) Alteryx
15 18 (1.41%) Salesforce.com
16 16 (1.25%) MicroStrategy
17 14 (1.09%) BiZZdesign