Period
to 21 October 2021

The following table provides summary statistics for contract job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 21 October 2021 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
UK > England
6 months to
21 Oct 2021
Same period 2020 Same period 2019
Rank 177 197 173
Rank change year-on-year +20 -24 +23
Contract jobs citing Apache Spark 1,058 385 1,259
As % of all contract jobs advertised in England 1.56% 1.41% 1.62%
As % of the Application Platforms category 12.16% 9.62% 10.82%
Number of daily rates quoted 673 253 952
10th Percentile £425 £400 £430
25th Percentile £500 £450 £513
Median daily rate (50th Percentile) £575 £550 £563
Median % change year-on-year +4.55% -2.22% +2.27%
75th Percentile £675 £638 £650
90th Percentile £738 £700 £725
UK median daily rate £575 £525 £550
% change year-on-year +9.52% -4.55% -
Number of hourly rates quoted 9 6 2
10th Percentile £18.58 £9.75 £21.59
25th Percentile £25.25 £11.25 £22.10
Median hourly rate £37.50 £19.44 £23.33
Median % change year-on-year +92.90% -16.66% -30.89%
75th Percentile £57.50 £23.41 £24.18
90th Percentile £65.00 £24.41 £24.46
UK median hourly rate £32.62 £20.00 £23.33
% change year-on-year +63.08% -14.26% -48.17%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Contract vacancies with a requirement for application platform skills 8,703 4,003 11,635
As % of all contract IT jobs advertised in England 12.87% 14.70% 15.00%
Number of daily rates quoted 6,049 2,736 7,757
10th Percentile £303 £300 £263
25th Percentile £406 £375 £363
Median daily rate (50th Percentile) £500 £475 £463
Median % change year-on-year +5.26% +2.70% -
75th Percentile £600 £563 £556
90th Percentile £688 £664 £638
UK median daily rate £500 £475 £450
% change year-on-year +5.26% +5.56% -
Number of hourly rates quoted 149 112 241
10th Percentile £14.25 £12.63 £10.60
25th Percentile £16.23 £16.14 £13.50
Median hourly rate £22.50 £22.50 £19.50
Median % change year-on-year - +15.38% -2.50%
75th Percentile £51.00 £37.69 £35.00
90th Percentile £65.50 £57.02 £50.25
UK median hourly rate £24.00 £22.50 £20.89
% change year-on-year +6.67% +7.71% +4.45%

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a proportion of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Contractor Daily Rate Trend in England

3-month moving average daily rate quoted in jobs citing Apache Spark in England.

Daily rate trend for Apache Spark in England

Apache Spark
Daily Rate Histogram in England

Daily rate distribution for jobs citing Apache Spark in England over the 6 months to 21 October 2021.

Daily rate histogram for Apache Spark in England

Apache Spark
Contractor Hourly Rate Trend in England

3-month moving average hourly rates quoted in jobs citing Apache Spark in England.

Hourly rate trend for Apache Spark in England

Apache Spark
Hourly Rate Histogram in England

Hourly rate distribution of jobs citing Apache Spark in England over the 6 months to 21 October 2021.

Hourly rate histogram for Apache Spark in England

Apache Spark
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the England region over the 6 months to 21 October 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London +7 787 £600 +9.09% 128
South East -32 103 £525 +10.53% 19
North of England -30 61 £564 +32.65% 6
North East - 34 £620 - 2
Midlands -53 21 £450 +15.68% 12
South West -45 19 £500 -23.81% 4
Yorkshire -32 17 £450 +5.88% 2
West Midlands -58 13 £450 +12.50% 3
North West -67 11 £490 +18.79% 2
East of England -67 10 £575 -6.12% 5
East Midlands -30 7 £525 +38.89% 9
Apache Spark
UK

For the 6 months to 21 October 2021, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region with a requirement for Apache Spark.

1 547 (51.70%) Python
2 541 (51.13%) Big Data
3 520 (49.15%) Hadoop
4 509 (48.11%) SQL
5 422 (39.89%) Kafka
6 383 (36.20%) Azure
7 381 (36.01%) Java
8 372 (35.16%) Scala
9 358 (33.84%) AWS
10 356 (33.65%) Analytics
11 312 (29.49%) Agile Software Development
12 302 (28.54%) Apache Hive
13 295 (27.88%) DevOps
14 251 (23.72%) Data Pipeline
15 243 (22.97%) Data Modelling
16 231 (21.83%) Finance
17 222 (20.98%) Data Engineering
18 221 (20.89%) Social Skills
19 212 (20.04%) Kubernetes
20 199 (18.81%) Docker
21 198 (18.71%) NoSQL
22 196 (18.53%) Apache
23 194 (18.34%) Data Science
24 189 (17.86%) Data Warehouse
25 187 (17.67%) Banking
26 185 (17.49%) ETL
26 185 (17.49%) Databricks
27 174 (16.45%) Jenkins
28 170 (16.07%) Machine Learning
29 161 (15.22%) HBase

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 196 (18.53%) Apache
2 113 (10.68%) Apache Airflow
3 76 (7.18%) Elasticsearch
4 48 (4.54%) Confluence
5 33 (3.12%) Luigi
6 20 (1.89%) Apache Pig
7 19 (1.80%) Tomcat
8 14 (1.32%) SAS
9 9 (0.85%) Jupyter
10 8 (0.76%) Oracle Coherence
11 6 (0.57%) SharePoint
12 5 (0.47%) Mule
12 5 (0.47%) nginx
13 4 (0.38%) Camunda
13 4 (0.38%) GemFire
14 3 (0.28%) JBoss
15 2 (0.19%) Apache Pulsar
15 2 (0.19%) Cloud Foundry
15 2 (0.19%) Oracle Workflow
16 1 (0.095%) R Markdown
Applications
1 5 (0.47%) SPSS
2 4 (0.38%) Microsoft Excel
3 2 (0.19%) Microsoft PowerPoint
4 1 (0.095%) Chatbot
4 1 (0.095%) SPSS Modeler
Business Applications
1 6 (0.57%) SAP Oil and Gas
2 4 (0.38%) SAP S/4HANA
3 2 (0.19%) SAP R/3
4 1 (0.095%) Distributed Ledger
4 1 (0.095%) Oracle Cash Management
4 1 (0.095%) Oracle Procure-to-Pay
4 1 (0.095%) Towers Watson Igloo
Cloud Services
1 383 (36.20%) Azure
2 358 (33.84%) AWS
3 156 (14.74%) GCP
4 107 (10.11%) Azure Data Factory
5 99 (9.36%) Serverless
6 90 (8.51%) Amazon EMR
7 78 (7.37%) Azure Cosmos DB
8 74 (6.99%) Amazon S3
9 67 (6.33%) Azure Functions
10 63 (5.95%) Power Platform
11 60 (5.67%) Amazon EC2
12 56 (5.29%) GitHub
13 48 (4.54%) Azure Synapse Analytics
14 38 (3.59%) Snowflake
15 37 (3.50%) Amazon ECS
16 36 (3.40%) Route 53
17 32 (3.02%) Collibra
18 30 (2.84%) Amazon EKS
18 30 (2.84%) AWS Lambda
18 30 (2.84%) PaaS
Communications & Networking
1 24 (2.27%) DNS
2 17 (1.61%) HTTP
3 12 (1.13%) Firewall
3 12 (1.13%) Kerberos
4 10 (0.95%) FTP
4 10 (0.95%) WAN
5 8 (0.76%) HTTPS
5 8 (0.76%) Network Security
6 7 (0.66%) SSL
7 5 (0.47%) Cisco ASA
7 5 (0.47%) SIGTRAN
7 5 (0.47%) SMPP
7 5 (0.47%) SMS
7 5 (0.47%) SS7
7 5 (0.47%) Wireshark
8 4 (0.38%) Internet
8 4 (0.38%) VPN
9 3 (0.28%) Intranet
10 2 (0.19%) LAN
10 2 (0.19%) TCP/IP
Database & Business Intelligence
1 541 (51.13%) Big Data
2 520 (49.15%) Hadoop
3 302 (28.54%) Apache Hive
4 198 (18.71%) NoSQL
5 189 (17.86%) Data Warehouse
6 161 (15.22%) HBase
7 153 (14.46%) Data Lake
8 126 (11.91%) Relational Database
9 102 (9.64%) PostgreSQL
10 100 (9.45%) Apache Cassandra
11 96 (9.07%) MongoDB
12 90 (8.51%) Tableau
13 84 (7.94%) SQL Server
14 79 (7.47%) BigQuery
15 71 (6.71%) Metadata
16 64 (6.05%) Impala
17 63 (5.95%) Power BI
18 61 (5.77%) Amazon Redshift
19 54 (5.10%) RDBMS
20 53 (5.01%) Flink
Development Applications
1 174 (16.45%) Jenkins
2 160 (15.12%) Git (software)
3 78 (7.37%) JIRA
4 58 (5.48%) GitLab
5 47 (4.44%) Maven
6 38 (3.59%) Hudson
7 37 (3.50%) ERwin
8 36 (3.40%) TeamCity
9 26 (2.46%) PowerDesigner
9 26 (2.46%) Subversion
10 18 (1.70%) Gradle
10 18 (1.70%) IntelliJ
11 14 (1.32%) Sonatype Nexus
12 13 (1.23%) IBM UrbanCode
13 12 (1.13%) Artifactory
14 11 (1.04%) Bitbucket
14 11 (1.04%) ER Studio
15 10 (0.95%) MLflow
15 10 (0.95%) Visual Studio Team System
16 9 (0.85%) Visual Studio
General
1 231 (21.83%) Finance
2 221 (20.89%) Social Skills
3 187 (17.67%) Banking
4 138 (13.04%) Analytical Skills
5 83 (7.84%) Investment Banking
6 46 (4.35%) Presentation Skills
7 37 (3.50%) Public Sector
8 22 (2.08%) Front Office
9 18 (1.70%) Marketing
10 17 (1.61%) Financial Institution
10 17 (1.61%) Retail
11 14 (1.32%) Legal
12 13 (1.23%) Publishing
13 12 (1.13%) Games
14 9 (0.85%) Manufacturing
14 9 (0.85%) Telecoms
15 7 (0.66%) Aerospace
16 5 (0.47%) Documentation Skills
16 5 (0.47%) Law
17 4 (0.38%) Back Office
Job Titles
1 427 (40.36%) Data Engineer
2 220 (20.79%) Developer
3 100 (9.45%) Architect
4 83 (7.84%) Big Data Engineer
5 73 (6.90%) Azure Engineer
6 72 (6.81%) Java Developer
7 71 (6.71%) DevOps Engineer
8 61 (5.77%) AWS Engineer
9 60 (5.67%) Data Architect
10 52 (4.91%) Scala Developer
11 47 (4.44%) Data Scientist
11 47 (4.44%) Senior Data Engineer
12 34 (3.21%) Data Modeller
13 32 (3.02%) Apache Spark Developer
14 31 (2.93%) Hadoop Engineer
14 31 (2.93%) Senior Developer
15 27 (2.55%) AWS DevOps
15 27 (2.55%) Cloud Engineer
16 26 (2.46%) Big Data Developer
16 26 (2.46%) Data Manager
Libraries, Frameworks & Software Standards
1 422 (39.89%) Kafka
2 146 (13.80%) PySpark
3 94 (8.88%) Spark Streaming
4 74 (6.99%) Spring
5 42 (3.97%) Spring Boot
6 41 (3.88%) RESTful
7 39 (3.69%) Apache NiFi
7 39 (3.69%) JSON
7 39 (3.69%) Web Services
8 32 (3.02%) REST
9 31 (2.93%) ARM Templates
10 29 (2.74%) Apache Avro
11 27 (2.55%) Node.js
12 22 (2.08%) TensorFlow
13 21 (1.98%) Pandas
14 19 (1.80%) .NET
14 19 (1.80%) Akka
15 16 (1.51%) NumPy
15 16 (1.51%) Velocity
15 16 (1.51%) XML
Miscellaneous
1 68 (6.43%) Algorithms
2 55 (5.20%) Data Structures
3 43 (4.06%) Virtual Team
4 34 (3.21%) Cloud Native
5 26 (2.46%) Market Risk
6 24 (2.27%) Management Information System
6 24 (2.27%) PKI
7 19 (1.80%) Data Protection Act
7 19 (1.80%) Legacy Applications
8 17 (1.61%) iPad
9 16 (1.51%) Greenfield Project
9 16 (1.51%) Self-Motivation
10 15 (1.42%) Data Centre
11 14 (1.32%) Public Cloud
12 12 (1.13%) Hedge funds
13 10 (0.95%) Foreign Exchange (FX)
13 10 (0.95%) Social Media
14 9 (0.85%) Hybrid Cloud
14 9 (0.85%) Linux Command Line
15 8 (0.76%) Grid Computing
Operating Systems
1 99 (9.36%) Linux
2 81 (7.66%) Unix
3 16 (1.51%) Red Hat Enterprise Linux
4 14 (1.32%) Windows
5 7 (0.66%) Android
6 3 (0.28%) Debian
6 3 (0.28%) Ubuntu
6 3 (0.28%) VMS
7 2 (0.19%) Apple iOS
8 1 (0.095%) Windows Server
8 1 (0.095%) Windows Server 2012
Processes & Methodologies
1 356 (33.65%) Analytics
2 312 (29.49%) Agile Software Development
3 295 (27.88%) DevOps
4 251 (23.72%) Data Pipeline
5 243 (22.97%) Data Modelling
6 222 (20.98%) Data Engineering
7 194 (18.34%) Data Science
8 185 (17.49%) ETL
9 170 (16.07%) Machine Learning
10 155 (14.65%) Business Intelligence
11 135 (12.76%) Scrum
12 113 (10.68%) Visualisation
13 110 (10.40%) Microservices
14 109 (10.30%) Computer Science
15 105 (9.92%) Data Management
16 102 (9.64%) Data Architecture
17 97 (9.17%) Open Source
18 89 (8.41%) Artificial Intelligence
19 86 (8.13%) Use Case
20 83 (7.84%) Data Ingestion
Programming Languages
1 547 (51.70%) Python
2 509 (48.11%) SQL
3 381 (36.01%) Java
4 372 (35.16%) Scala
5 81 (7.66%) R
6 41 (3.88%) JavaScript
7 37 (3.50%) C++
8 35 (3.31%) Bash Shell
8 35 (3.31%) C#
9 29 (2.74%) PL/SQL
9 29 (2.74%) PowerShell
10 26 (2.46%) Shell Script
11 25 (2.36%) Go
12 22 (2.08%) Groovy
12 22 (2.08%) Java 8
12 22 (2.08%) T-SQL
13 20 (1.89%) C
14 19 (1.80%) Kotlin
15 17 (1.61%) Perl
16 11 (1.04%) CQL
Qualifications
1 101 (9.55%) Degree
2 75 (7.09%) Security Cleared
3 56 (5.29%) SC Cleared
4 55 (5.20%) Computer Science Degree
5 18 (1.70%) BPSS Clearance
6 16 (1.51%) AWS Certification
7 13 (1.23%) Numerate Degree
8 7 (0.66%) DV Cleared
9 6 (0.57%) PhD
10 4 (0.38%) DBS Check
10 4 (0.38%) ISEB
11 3 (0.28%) Master's Degree
12 2 (0.19%) TOGAF Certification
13 1 (0.095%) CBAP
13 1 (0.095%) CCA
13 1 (0.095%) MBA
13 1 (0.095%) PMI Certification
Quality Assurance & Compliance
1 80 (7.56%) Data Quality
2 44 (4.16%) GDPR
3 41 (3.88%) QA
4 9 (0.85%) IFRS
5 6 (0.57%) Disclosure Scotland
6 5 (0.47%) MiFID
6 5 (0.47%) Sarbanes-Oxley
7 2 (0.19%) COBIT
7 2 (0.19%) ISO/IEC 27001
7 2 (0.19%) NIST
8 1 (0.095%) Dodd-Frank
8 1 (0.095%) EMIR
8 1 (0.095%) IFRS 9
8 1 (0.095%) ISO 20022
8 1 (0.095%) NCSC
8 1 (0.095%) PSD2
8 1 (0.095%) SLA
8 1 (0.095%) TARGET2
System Software
1 199 (18.81%) Docker
2 126 (11.91%) HDFS
3 28 (2.65%) Apache ZooKeeper
4 14 (1.32%) Active Directory
5 13 (1.23%) Apache Flume
6 3 (0.28%) Active Directory Lightweight Directory Services
7 2 (0.19%) VMware Infrastructure
8 1 (0.095%) Hyper-V
Systems Management
1 212 (20.04%) Kubernetes
2 156 (14.74%) Terraform
3 75 (7.09%) Ansible
4 58 (5.48%) Oozie
5 45 (4.25%) Prometheus
6 37 (3.50%) Hadoop YARN
7 21 (1.98%) Grafana
7 21 (1.98%) Puppet
8 14 (1.32%) Kibana
9 12 (1.13%) Control-M
9 12 (1.13%) Opscode Chef
9 12 (1.13%) Single Sign-On
10 11 (1.04%) Ambari
10 11 (1.04%) logstash
11 8 (0.76%) Docker Swarm
11 8 (0.76%) Kubeflow
12 6 (0.57%) Active Directory Federation Services
12 6 (0.57%) Rancher
13 5 (0.47%) FortiGate
14 3 (0.28%) Istio
Vendors
1 185 (17.49%) Databricks
2 134 (12.67%) Cloudera
3 90 (8.51%) Google
4 85 (8.03%) Oracle
5 75 (7.09%) Hortonworks
6 71 (6.71%) Microsoft
7 40 (3.78%) Informatica
8 39 (3.69%) Splunk
9 35 (3.31%) IBM
10 30 (2.84%) BiZZdesign
11 28 (2.65%) SAP
12 25 (2.36%) Ab Initio
13 22 (2.08%) Red Hat
13 22 (2.08%) Teradata
14 18 (1.70%) MapR
15 17 (1.61%) Alteryx
16 16 (1.51%) Salesforce.com
17 15 (1.42%) MicroStrategy
18 13 (1.23%) Confluent
19 11 (1.04%) Talend