Period
to

The following table provides summary statistics for contract job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 17 May 2022 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
UK > England
6 months to
17 May 2022
Same period 2021 Same period 2020
Rank 243 167 197
Rank change year-on-year -76 +30 -45
Contract jobs citing Apache Spark 794 717 609
As % of all contract jobs advertised in England 1.13% 1.58% 1.36%
As % of the Application Platforms category 9.25% 11.94% 9.46%
Number of daily rates quoted 501 462 454
10th Percentile £438 £425 £413
25th Percentile £525 £495 £463
Median daily rate (50th Percentile) £600 £563 £550
Median % change year-on-year +6.67% +2.27% -
75th Percentile £710 £638 £638
90th Percentile £825 £713 £688
UK median daily rate £600 £555 £539
% change year-on-year +8.11% +2.97% -2.00%
Number of hourly rates quoted 3 3 0
10th Percentile £62.35 £18.65 -
25th Percentile £68.50 £25.25 -
Median hourly rate £78.75 £37.50 -
Median % change year-on-year +110.00% - -
75th Percentile £84.83 £58.13 -
90th Percentile £88.47 £69.75 -
UK median hourly rate £78.75 £27.00 -
% change year-on-year +191.67% - -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Contract vacancies with a requirement for application platform skills 8,588 6,006 6,441
As % of all contract IT jobs advertised in England 12.22% 13.26% 14.40%
Number of daily rates quoted 5,926 4,155 4,353
10th Percentile £313 £300 £276
25th Percentile £413 £398 £375
Median daily rate (50th Percentile) £525 £483 £475
Median % change year-on-year +8.81% +1.58% -
75th Percentile £638 £580 £573
90th Percentile £738 £660 £650
UK median daily rate £525 £475 £473
% change year-on-year +10.53% +0.53% -0.53%
Number of hourly rates quoted 153 123 147
10th Percentile £14.25 £12.25 £12.63
25th Percentile £17.00 £15.74 £14.90
Median hourly rate £35.00 £24.50 £20.00
Median % change year-on-year +42.86% +22.50% +17.23%
75th Percentile £59.00 £56.50 £34.38
90th Percentile £77.02 £65.53 £54.25
UK median hourly rate £35.00 £25.50 £20.00
% change year-on-year +37.25% +27.50% +14.29%

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a proportion of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Contractor Daily Rate Trend in England

3-month moving average daily rate quoted in jobs citing Apache Spark in England.

Daily rate trend for Apache Spark in England

Apache Spark
Daily Rate Histogram in England

Daily rate distribution for jobs citing Apache Spark in England over the 6 months to 17 May 2022.

Daily rate histogram for Apache Spark in England

Apache Spark
Contractor Hourly Rate Trend in England

3-month moving average hourly rates quoted in jobs citing Apache Spark in England.

Hourly rate trend for Apache Spark in England

Apache Spark
Hourly Rate Histogram in England

Hourly rate distribution of jobs citing Apache Spark in England over the 6 months to 17 May 2022.

Hourly rate histogram for Apache Spark in England

Apache Spark
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the England region over the 6 months to 17 May 2022. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Jobs
London -49 612 £615 +6.96% 105
North of England -56 41 £550 -0.90% 13
South East -56 39 £584 +16.75% 10
Yorkshire -22 32 £550 -0.45% 5
East of England -23 21 £525 +49.86% 13
Midlands -41 16 £538 +26.47% 12
North West -59 9 £550 -7.37% 7
West Midlands -44 8 £550 +29.41% 8
East Midlands -9 7 £438 -18.60% 4
South West -58 6 £575 +15.00% 4
Apache Spark
UK

For the 6 months to 17 May 2022, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region with a requirement for Apache Spark.

1 470 (59.19%) Python
2 417 (52.52%) Big Data
3 368 (46.35%) Kafka
4 332 (41.81%) AWS
5 324 (40.81%) Azure
6 302 (38.04%) Analytics
7 295 (37.15%) SQL
7 295 (37.15%) Agile
8 292 (36.78%) Java
9 279 (35.14%) Scala
10 260 (32.75%) CI/CD
11 258 (32.49%) Hadoop
12 250 (31.49%) Data Engineering
13 219 (27.58%) ETL
14 198 (24.94%) Databricks
15 197 (24.81%) Data Pipeline
16 195 (24.56%) Apache
16 195 (24.56%) DevOps
17 185 (23.30%) Data Modelling
18 182 (22.92%) Finance
19 179 (22.54%) Machine Learning
20 178 (22.42%) Data Science
21 161 (20.28%) NoSQL
22 156 (19.65%) Data Warehouse
23 149 (18.77%) Git (software)
23 149 (18.77%) Data Lake
24 141 (17.76%) GCP
25 135 (17.00%) Business Intelligence
26 133 (16.75%) Apache Hive
27 122 (15.37%) Kubernetes

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 195 (24.56%) Apache
2 84 (10.58%) Apache Airflow
3 32 (4.03%) Elasticsearch
4 24 (3.02%) Confluence
5 17 (2.14%) Jupyter
6 16 (2.02%) nginx
6 16 (2.02%) SharePoint
7 14 (1.76%) Apache Pig
7 14 (1.76%) Luigi
8 8 (1.01%) Apache Pulsar
8 8 (1.01%) Apache Solr
8 8 (1.01%) SAS
8 8 (1.01%) uWSGI
9 7 (0.88%) Oracle Coherence
10 5 (0.63%) Tomcat
11 3 (0.38%) WebLogic
12 2 (0.25%) JBoss
12 2 (0.25%) OpenStack
12 2 (0.25%) WebSphere
13 1 (0.13%) Tornado
Applications
1 7 (0.88%) SPSS
2 6 (0.76%) Microsoft Excel
3 4 (0.50%) Microsoft PowerPoint
4 3 (0.38%) MS Visio
5 1 (0.13%) Microsoft Office
Business Applications
1 5 (0.63%) SAP Oil and Gas
2 1 (0.13%) SAP S/4HANA
Cloud Services
1 332 (41.81%) AWS
2 324 (40.81%) Azure
3 141 (17.76%) GCP
4 89 (11.21%) Amazon S3
5 77 (9.70%) Serverless
6 76 (9.57%) Amazon EMR
7 74 (9.32%) Azure Data Factory
8 69 (8.69%) Power Platform
9 57 (7.18%) Azure Synapse Analytics
9 57 (7.18%) GitHub
10 35 (4.41%) AWS Lambda
11 34 (4.28%) Azure Functions
12 32 (4.03%) Azure Cosmos DB
13 31 (3.90%) Snowflake
14 30 (3.78%) Amazon EC2
15 29 (3.65%) Amazon SageMaker
16 28 (3.53%) Amazon Kinesis
17 22 (2.77%) AWS CloudFormation
17 22 (2.77%) dbt
18 19 (2.39%) Azure Monitor
Communications & Networking
1 7 (0.88%) HTTP
2 6 (0.76%) FTP
3 5 (0.63%) WAN
4 4 (0.50%) Firewall
4 4 (0.50%) Kerberos
4 4 (0.50%) Network Security
4 4 (0.50%) SSH
5 3 (0.38%) Internet
5 3 (0.38%) IPFS
6 2 (0.25%) Radio Access Network
7 1 (0.13%) DNS
7 1 (0.13%) DWDM
7 1 (0.13%) MPLS
7 1 (0.13%) SD-WAN
7 1 (0.13%) SMTP
7 1 (0.13%) TCP/IP
7 1 (0.13%) VPN
Database & Business Intelligence
1 417 (52.52%) Big Data
2 258 (32.49%) Hadoop
3 161 (20.28%) NoSQL
4 156 (19.65%) Data Warehouse
5 149 (18.77%) Data Lake
6 133 (16.75%) Apache Hive
7 83 (10.45%) Amazon Redshift
8 69 (8.69%) MongoDB
8 69 (8.69%) Power BI
9 66 (8.31%) SQL Server
10 65 (8.19%) HBase
11 64 (8.06%) Metadata
12 61 (7.68%) Apache Cassandra
13 60 (7.56%) Relational Database
14 56 (7.05%) Tableau
15 51 (6.42%) Flink
16 47 (5.92%) Pentaho
17 46 (5.79%) Amazon Athena
18 44 (5.54%) Amazon RDS
19 43 (5.42%) BigQuery
Development Applications
1 149 (18.77%) Git (software)
2 89 (11.21%) Jenkins
3 60 (7.56%) JIRA
4 53 (6.68%) GitLab
4 53 (6.68%) Maven
5 34 (4.28%) Gradle
6 33 (4.16%) TeamCity
7 20 (2.52%) MLflow
8 12 (1.51%) CircleCI
9 11 (1.39%) Sonatype Nexus
10 9 (1.13%) ER Studio
10 9 (1.13%) ERwin
10 9 (1.13%) Gerrit
11 8 (1.01%) JUnit
11 8 (1.01%) Subversion
12 6 (0.76%) IBM UrbanCode
13 5 (0.63%) Eclipse
13 5 (0.63%) IntelliJ
13 5 (0.63%) kappa
14 4 (0.50%) Visual Studio
General
1 182 (22.92%) Finance
2 106 (13.35%) Social Skills
3 102 (12.85%) Banking
4 96 (12.09%) Analytical Skills
5 37 (4.66%) Public Sector
6 35 (4.41%) Financial Institution
7 33 (4.16%) Investment Banking
7 33 (4.16%) Marketing
8 28 (3.53%) Legal
9 24 (3.02%) Presentation Skills
9 24 (3.02%) Retail
10 18 (2.27%) Publishing
11 11 (1.39%) Law
11 11 (1.39%) Telecoms
12 7 (0.88%) Front Office
13 6 (0.76%) Games
13 6 (0.76%) Pharmaceutical
14 4 (0.50%) Documentation Skills
15 3 (0.38%) Manufacturing
16 2 (0.25%) Electronics
Job Titles
1 388 (48.87%) Data Engineer
2 119 (14.99%) Developer
3 73 (9.19%) Architect
4 69 (8.69%) Big Data Engineer
5 58 (7.30%) Python Engineer
6 57 (7.18%) Azure Engineer
7 49 (6.17%) Data Architect
7 49 (6.17%) Machine Learning Engineer
8 47 (5.92%) Cloud Engineer
8 47 (5.92%) Senior Data Engineer
9 38 (4.79%) Java Engineer
10 33 (4.16%) Scala Developer
11 32 (4.03%) AWS Engineer
12 29 (3.65%) Data Scientist
13 27 (3.40%) Big Data Developer
13 27 (3.40%) Hadoop Engineer
14 25 (3.15%) Software Engineer
15 23 (2.90%) DevOps Engineer
15 23 (2.90%) Java Developer
16 22 (2.77%) Scala Engineer
Libraries, Frameworks & Software Standards
1 368 (46.35%) Kafka
2 112 (14.11%) PySpark
3 70 (8.82%) Spark Streaming
4 47 (5.92%) Pandas
5 36 (4.53%) TensorFlow
6 33 (4.16%) Apache NiFi
7 28 (3.53%) NumPy
8 27 (3.40%) Apache Beam
9 24 (3.02%) Apache Avro
9 24 (3.02%) REST
9 24 (3.02%) scikit-learn
10 23 (2.90%) PyTorch
10 23 (2.90%) React
11 21 (2.64%) Flask
12 19 (2.39%) CSS
13 16 (2.02%) JSON
14 14 (1.76%) Elastic Stack
14 14 (1.76%) HTML
14 14 (1.76%) SciPy
15 13 (1.64%) pytest
Miscellaneous
1 68 (8.56%) Algorithms
2 50 (6.30%) Data Structures
3 31 (3.90%) Public Cloud
4 25 (3.15%) Hedge funds
5 24 (3.02%) iPad
5 24 (3.02%) Self-Motivation
6 22 (2.77%) Linux Command Line
7 18 (2.27%) Greenfield Project
7 18 (2.27%) Management Information System
8 17 (2.14%) Cloud Native
8 17 (2.14%) IoT
9 14 (1.76%) Legacy Systems
9 14 (1.76%) Virtual Team
10 11 (1.39%) Equities
10 11 (1.39%) Market Risk
11 9 (1.13%) Enterprise Software
12 8 (1.01%) Data Centre
12 8 (1.01%) Replication
13 6 (0.76%) Data Protection Act
13 6 (0.76%) Reference Data
Operating Systems
1 98 (12.34%) Linux
2 29 (3.65%) Unix
3 6 (0.76%) Windows
4 5 (0.63%) Ubuntu
5 2 (0.25%) Android
5 2 (0.25%) Apple iOS
6 1 (0.13%) Debian
6 1 (0.13%) Red Hat Enterprise Linux
Processes & Methodologies
1 302 (38.04%) Analytics
2 295 (37.15%) Agile
3 260 (32.75%) CI/CD
4 250 (31.49%) Data Engineering
5 219 (27.58%) ETL
6 197 (24.81%) Data Pipeline
7 195 (24.56%) DevOps
8 185 (23.30%) Data Modelling
9 179 (22.54%) Machine Learning
10 178 (22.42%) Data Science
11 135 (17.00%) Business Intelligence
12 112 (14.11%) TDD
13 111 (13.98%) Software Engineering
14 93 (11.71%) Visualisation
15 88 (11.08%) Scrum
16 85 (10.71%) Migration
17 79 (9.95%) Artificial Intelligence
17 79 (9.95%) Data Analysis
18 77 (9.70%) Microservices
19 76 (9.57%) BDD
Programming Languages
1 470 (59.19%) Python
2 295 (37.15%) SQL
3 292 (36.78%) Java
4 279 (35.14%) Scala
5 58 (7.30%) R
6 38 (4.79%) C#
7 30 (3.78%) Shell Script
8 29 (3.65%) Bash
9 28 (3.53%) C++
10 22 (2.77%) C
10 22 (2.77%) JavaScript
10 22 (2.77%) PowerShell
11 10 (1.26%) Go
11 10 (1.26%) TypeScript
12 9 (1.13%) PHP
12 9 (1.13%) Ruby
12 9 (1.13%) T-SQL
13 5 (0.63%) Perl
13 5 (0.63%) PL/SQL
14 4 (0.50%) MATLAB
Qualifications
1 84 (10.58%) Security Cleared
2 74 (9.32%) SC Cleared
3 48 (6.05%) Degree
4 18 (2.27%) Computer Science Degree
5 13 (1.64%) PhD
6 10 (1.26%) FFIEC
7 8 (1.01%) PMI Certification
7 8 (1.01%) PMP
8 7 (0.88%) Master's Degree
9 6 (0.76%) BPSS Clearance
9 6 (0.76%) DV Cleared
10 4 (0.50%) AWS Certification
10 4 (0.50%) CCA
11 3 (0.38%) Numerate Degree
12 1 (0.13%) DBS Check
12 1 (0.13%) Sun Certification
Quality Assurance & Compliance
1 63 (7.93%) Data Quality
2 26 (3.27%) GDPR
3 22 (2.77%) QA
4 10 (1.26%) GLBA
4 10 (1.26%) NIST
4 10 (1.26%) PCI DSS
4 10 (1.26%) SLA
5 3 (0.38%) IFRS
5 3 (0.38%) IFRS 17
6 1 (0.13%) PMO
System Software
1 116 (14.61%) Docker
2 61 (7.68%) HDFS
3 10 (1.26%) Apache Flume
4 5 (0.63%) Active Directory
5 4 (0.50%) Apache ZooKeeper
6 3 (0.38%) Virtual Machines
Systems Management
1 122 (15.37%) Kubernetes
2 87 (10.96%) Terraform
3 19 (2.39%) Oozie
4 14 (1.76%) Ansible
4 14 (1.76%) Kibana
5 13 (1.64%) Grafana
6 12 (1.51%) Hadoop YARN
7 11 (1.39%) logstash
8 10 (1.26%) Kubeflow
9 9 (1.13%) Argo
10 6 (0.76%) Prometheus
11 4 (0.50%) Opscode Chef
12 3 (0.38%) Istio
12 3 (0.38%) Puppet
13 2 (0.25%) Ambari
13 2 (0.25%) Control-M
Vendors
1 198 (24.94%) Databricks
2 82 (10.33%) Talend
3 72 (9.07%) Microsoft
4 67 (8.44%) Cloudera
5 60 (7.56%) Google
6 51 (6.42%) Oracle
7 32 (4.03%) Hortonworks
8 26 (3.27%) Informatica
9 24 (3.02%) Salesforce.com
10 21 (2.64%) SAP
11 19 (2.39%) Alteryx
11 19 (2.39%) Splunk
12 15 (1.89%) Salesforce Marketing Cloud
12 15 (1.89%) Teradata
13 12 (1.51%) Netezza
14 7 (0.88%) Foundry
14 7 (0.88%) IBM
14 7 (0.88%) MapR
14 7 (0.88%) Palantir
15 6 (0.76%) StreamSets