Period
to 11 May 2021

The following table provides summary statistics for contract job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 11 May 2021 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
UK > England
6 months to
11 May 2021
Same period 2020 Same period 2019
Rank 167 199 152
Rank change year-on-year +32 -47 +20
Contract jobs citing Apache Spark 701 617 1,567
As % of all contract jobs advertised in England 1.58% 1.34% 1.84%
As % of the Application Platforms category 11.83% 9.32% 11.67%
Number of daily rates quoted 453 457 1,135
10th Percentile £425 £413 £450
25th Percentile £500 £463 £506
Median daily rate (50th Percentile) £575 £550 £550
Median % change year-on-year +4.55% - -
75th Percentile £638 £638 £625
90th Percentile £713 £688 £713
UK median daily rate £563 £550 £550
% change year-on-year +2.27% - -
Number of hourly rates quoted 2 0 1
10th Percentile £19.08 " "
25th Percentile £26.31 " £52.50
Median hourly rate £42.25 - £55.00
75th Percentile £61.81 " £57.50
90th Percentile £71.23 " "
UK median hourly rate £25.75 - £55.00

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Contract vacancies with a requirement for application platform skills 5,924 6,617 13,431
As % of all contract IT jobs advertised in England 13.33% 14.33% 15.81%
Number of daily rates quoted 4,098 4,454 8,793
10th Percentile £300 £275 £288
25th Percentile £400 £369 £375
Median daily rate (50th Percentile) £480 £475 £475
Median % change year-on-year +1.05% - +4.40%
75th Percentile £577 £571 £575
90th Percentile £660 £650 £645
UK median daily rate £475 £473 £475
% change year-on-year +0.53% -0.53% +5.56%
Number of hourly rates quoted 120 153 257
10th Percentile £12.25 £12.53 £11.75
25th Percentile £16.25 £14.80 £12.50
Median hourly rate £24.50 £19.90 £18.00
Median % change year-on-year +23.12% +10.56% -3.41%
75th Percentile £56.75 £33.75 £34.51
90th Percentile £65.50 £53.75 £48.75
UK median hourly rate £25.50 £20.00 £18.00
% change year-on-year +27.50% +11.11% +2.86%

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a proportion of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Contractor Daily Rate Trend in England

3-month moving average daily rate quoted in jobs citing Apache Spark in England.

Daily rate trend for Apache Spark in England

Apache Spark
Daily Rate Histogram in England

Daily rate distribution for jobs citing Apache Spark in England over the 6 months to 11 May 2021.

Daily rate histogram for Apache Spark in England

Apache Spark
Contractor Hourly Rate Trend in England

3-month moving average hourly rates quoted in jobs citing Apache Spark in England.

Hourly rate trend for Apache Spark in England

Apache Spark
Hourly Rate Histogram in England

Hourly rate distribution of jobs citing Apache Spark in England over the 6 months to 11 May 2021.

Hourly rate histogram for Apache Spark in England

Apache Spark
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the England region over the 6 months to 11 May 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London +21 496 £580 +5.45% 135
South East +16 59 £500 +2.56% 12
North of England +35 45 £555 +0.91% 10
South West +47 33 £500 -6.98% 1
North West +24 24 £594 +7.95% 8
Yorkshire +26 19 £553 +16.32% 2
Midlands +18 17 £425 -22.73% 4
East of England +23 14 £350 -29.11% 10
West Midlands -5 12 £425 -22.73% 4
North East +26 2 £605 -6.92%
East Midlands +18 2 £538 -
Apache Spark
UK

For the 6 months to 11 May 2021, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region with a requirement for Apache Spark.

1 372 (53.07%) Python
2 350 (49.93%) Big Data
3 343 (48.93%) Hadoop
4 298 (42.51%) SQL
5 287 (40.94%) Scala
6 273 (38.94%) Java
7 267 (38.09%) Kafka
8 228 (32.52%) Agile Software Development
9 212 (30.24%) Azure
9 212 (30.24%) Apache Hive
10 207 (29.53%) Analytics
11 202 (28.82%) Apache
12 194 (27.67%) Finance
13 189 (26.96%) AWS
14 166 (23.68%) Data Pipeline
15 154 (21.97%) DevOps
16 137 (19.54%) Kubernetes
17 135 (19.26%) Data Science
18 132 (18.83%) Social Skills
19 130 (18.54%) Machine Learning
20 129 (18.40%) Data Engineering
21 122 (17.40%) Banking
22 120 (17.12%) NoSQL
23 118 (16.83%) Data Warehouse
24 117 (16.69%) ETL
25 108 (15.41%) Docker
25 108 (15.41%) Databricks
26 106 (15.12%) Apache Airflow
26 106 (15.12%) Analytical Skills
27 104 (14.84%) Data Lake

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 202 (28.82%) Apache
2 106 (15.12%) Apache Airflow
3 40 (5.71%) Confluence
4 35 (4.99%) Elasticsearch
5 24 (3.42%) Oracle Coherence
6 12 (1.71%) SharePoint
7 11 (1.57%) Apache Pig
8 8 (1.14%) Luigi
8 8 (1.14%) SAS
9 7 (1.00%) GemFire
10 6 (0.86%) Apache Solr
11 4 (0.57%) Apache Pulsar
11 4 (0.57%) Jupyter
11 4 (0.57%) Oracle Workflow
12 3 (0.43%) ArcGIS
12 3 (0.43%) JBoss
13 1 (0.14%) DataSynapse
Applications
1 2 (0.29%) Microsoft PowerPoint
2 1 (0.14%) Microsoft Excel
2 1 (0.14%) Microsoft Office
2 1 (0.14%) MS Visio
2 1 (0.14%) SPSS
2 1 (0.14%) Stata
Business Applications
1 5 (0.71%) SAP Oil and Gas
2 1 (0.14%) SAP CRM
2 1 (0.14%) SAP ERP
2 1 (0.14%) SAP IS-U
Cloud Services
1 212 (30.24%) Azure
2 189 (26.96%) AWS
3 99 (14.12%) GCP
4 59 (8.42%) Amazon S3
5 58 (8.27%) Serverless
6 38 (5.42%) Amazon EMR
6 38 (5.42%) Azure Data Factory
7 37 (5.28%) GitHub
8 36 (5.14%) Azure Functions
9 35 (4.99%) Power Platform
10 33 (4.71%) Azure Cosmos DB
11 29 (4.14%) Amazon EC2
12 28 (3.99%) Azure Synapse Analytics
13 23 (3.28%) AWS Lambda
14 22 (3.14%) Amazon Kinesis
15 16 (2.28%) Google Analytics
16 15 (2.14%) Cloud Computing
17 12 (1.71%) AWS CloudFormation
17 12 (1.71%) Azure Storage
18 11 (1.57%) Amazon SageMaker
Communications & Networking
1 23 (3.28%) HTTP
2 10 (1.43%) Kerberos
3 9 (1.28%) Firewall
4 6 (0.86%) Network Security
5 5 (0.71%) DNS
5 5 (0.71%) SIGTRAN
5 5 (0.71%) SMPP
5 5 (0.71%) SMS
5 5 (0.71%) SS7
5 5 (0.71%) Wireshark
6 4 (0.57%) SSL
7 2 (0.29%) Internet
8 1 (0.14%) FTP
8 1 (0.14%) HTTPS
8 1 (0.14%) Software-Defined Networking
Database & Business Intelligence
1 350 (49.93%) Big Data
2 343 (48.93%) Hadoop
3 212 (30.24%) Apache Hive
4 120 (17.12%) NoSQL
5 118 (16.83%) Data Warehouse
6 104 (14.84%) Data Lake
7 87 (12.41%) HBase
8 69 (9.84%) Relational Database
9 67 (9.56%) MongoDB
10 58 (8.27%) Impala
11 49 (6.99%) Amazon Redshift
12 48 (6.85%) PostgreSQL
13 44 (6.28%) SQL Server
14 43 (6.13%) BigQuery
15 42 (5.99%) Snowflake Schema
16 41 (5.85%) Spark SQL
17 38 (5.42%) Flink
18 37 (5.28%) Tableau
19 35 (4.99%) Apache Cassandra
19 35 (4.99%) Power BI
Development Applications
1 100 (14.27%) Jenkins
2 95 (13.55%) Git (software)
3 45 (6.42%) JIRA
4 31 (4.42%) Maven
5 30 (4.28%) Sonatype Nexus
6 21 (3.00%) TeamCity
7 19 (2.71%) GitLab
8 14 (2.00%) Gradle
8 14 (2.00%) IBM UrbanCode
9 11 (1.57%) RStudio
10 10 (1.43%) Artifactory
10 10 (1.43%) JUnit
10 10 (1.43%) Visual Studio Team System
11 9 (1.28%) PowerDesigner
12 8 (1.14%) Bitbucket
12 8 (1.14%) Cucumber
12 8 (1.14%) MLflow
12 8 (1.14%) Oracle APEX
13 7 (1.00%) Hudson
14 5 (0.71%) ERwin
General
1 194 (27.67%) Finance
2 132 (18.83%) Social Skills
3 122 (17.40%) Banking
4 106 (15.12%) Analytical Skills
5 64 (9.13%) Front Office
6 54 (7.70%) Investment Banking
7 24 (3.42%) Marketing
8 21 (3.00%) Presentation Skills
9 20 (2.85%) Public Sector
10 12 (1.71%) Retail
11 11 (1.57%) Documentation Skills
11 11 (1.57%) Retail Banking
12 10 (1.43%) Financial Institution
13 9 (1.28%) Games
14 6 (0.86%) Back Office
15 5 (0.71%) French Language
15 5 (0.71%) Organisational Skills
16 4 (0.57%) Influencing Skills
16 4 (0.57%) Publishing
16 4 (0.57%) Telecoms
Job Titles
1 241 (34.38%) Data Engineer
2 225 (32.10%) Developer
3 73 (10.41%) Java Developer
3 73 (10.41%) Scala Developer
4 66 (9.42%) Architect
5 61 (8.70%) Big Data Engineer
6 52 (7.42%) Data Scientist
7 45 (6.42%) AWS Engineer
7 45 (6.42%) Senior Developer
8 41 (5.85%) Apache Spark Developer
8 41 (5.85%) Data Architect
9 38 (5.42%) Scala Engineer
10 34 (4.85%) Python Engineer
11 33 (4.71%) Hadoop Developer
12 29 (4.14%) Software Engineer
13 27 (3.85%) Big Data Developer
14 25 (3.57%) Azure Engineer
15 24 (3.42%) Python Developer
16 23 (3.28%) Senior Java Developer
17 20 (2.85%) Analyst
Libraries, Frameworks & Software Standards
1 267 (38.09%) Kafka
2 80 (11.41%) PySpark
3 54 (7.70%) Web Services
4 46 (6.56%) Spark Streaming
5 40 (5.71%) Spring
6 33 (4.71%) JSON
7 28 (3.99%) Node.js
7 28 (3.99%) RESTful
8 21 (3.00%) NumPy
9 20 (2.85%) TensorFlow
9 20 (2.85%) XML
10 19 (2.71%) Pandas
11 18 (2.57%) PyTorch
11 18 (2.57%) Velocity
12 17 (2.43%) React
12 17 (2.43%) Samza
13 16 (2.28%) Apache NiFi
13 16 (2.28%) Protocol Buffers
14 14 (2.00%) .NET
15 13 (1.85%) Apache Avro
Miscellaneous
1 60 (8.56%) Algorithms
2 45 (6.42%) Data Structures
3 31 (4.42%) Market Risk
4 29 (4.14%) Grid Computing
5 23 (3.28%) Linux Command Line
5 23 (3.28%) Virtual Team
6 22 (3.14%) Equities
6 22 (3.14%) Self-Motivation
7 17 (2.43%) Fixed Income
7 17 (2.43%) iPad
8 15 (2.14%) Derivative
8 15 (2.14%) Greenfield Project
9 12 (1.71%) Foreign Exchange (FX)
10 10 (1.43%) Management Information System
11 8 (1.14%) Hedge funds
11 8 (1.14%) Replication
12 7 (1.00%) Analytical Mindset
12 7 (1.00%) Cloud Native
12 7 (1.00%) User Experience
13 5 (0.71%) Social Media
Operating Systems
1 75 (10.70%) Linux
2 30 (4.28%) Unix
3 10 (1.43%) Windows
4 6 (0.86%) Android
5 3 (0.43%) Red Hat Enterprise Linux
6 1 (0.14%) Windows Server
Processes & Methodologies
1 228 (32.52%) Agile Software Development
2 207 (29.53%) Analytics
3 166 (23.68%) Data Pipeline
4 154 (21.97%) DevOps
5 135 (19.26%) Data Science
6 130 (18.54%) Machine Learning
7 129 (18.40%) Data Engineering
8 117 (16.69%) ETL
9 100 (14.27%) Data Modelling
10 88 (12.55%) TDD
11 82 (11.70%) Software Engineering
12 79 (11.27%) Business Intelligence
12 79 (11.27%) Scrum
13 76 (10.84%) Computer Science
14 74 (10.56%) Mathematics
15 66 (9.42%) BDD
16 62 (8.84%) Data Architecture
16 62 (8.84%) Open Source
17 60 (8.56%) Data Ingestion
18 58 (8.27%) Visualisation
Programming Languages
1 372 (53.07%) Python
2 298 (42.51%) SQL
3 287 (40.94%) Scala
4 273 (38.94%) Java
5 62 (8.84%) R
6 42 (5.99%) JavaScript
7 31 (4.42%) C#
8 25 (3.57%) C++
9 21 (3.00%) Shell Script
9 21 (3.00%) T-SQL
10 19 (2.71%) Haskell
11 17 (2.43%) PL/SQL
12 14 (2.00%) Go
12 14 (2.00%) Kotlin
13 13 (1.85%) Bash Shell
13 13 (1.85%) Groovy
14 12 (1.71%) Java 8
15 9 (1.28%) Elm
16 6 (0.86%) C
17 4 (0.57%) Ruby
Qualifications
1 70 (9.99%) Degree
2 54 (7.70%) Security Cleared
3 49 (6.99%) SC Cleared
4 27 (3.85%) Numerate Degree
5 14 (2.00%) Computer Science Degree
6 10 (1.43%) AWS Certification
6 10 (1.43%) ISEB
6 10 (1.43%) PhD
7 7 (1.00%) Master's Degree
8 3 (0.43%) BPSS Clearance
8 3 (0.43%) DBS Check
9 1 (0.14%) TOGAF Certification
Quality Assurance & Compliance
1 35 (4.99%) QA
2 33 (4.71%) Data Quality
3 17 (2.43%) GDPR
4 2 (0.29%) Disclosure Scotland
4 2 (0.29%) MiFID
4 2 (0.29%) SLA
5 1 (0.14%) ISO 20022
5 1 (0.14%) PCI DSS
5 1 (0.14%) SEPA
5 1 (0.14%) TARGET2
System Software
1 108 (15.41%) Docker
2 93 (13.27%) HDFS
3 17 (2.43%) Apache ZooKeeper
4 13 (1.85%) Apache Flume
5 1 (0.14%) Virtual Machines
Systems Management
1 137 (19.54%) Kubernetes
2 66 (9.42%) Terraform
3 53 (7.56%) Ansible
4 47 (6.70%) Oozie
5 38 (5.42%) Hadoop YARN
6 17 (2.43%) Puppet
7 9 (1.28%) Kibana
8 8 (1.14%) Control-M
9 6 (0.86%) Opscode Chef
10 5 (0.71%) Ambari
10 5 (0.71%) Grafana
10 5 (0.71%) Prometheus
11 2 (0.29%) Istio
11 2 (0.29%) Kubeflow
12 1 (0.14%) Rancher
12 1 (0.14%) Zipkin
Vendors
1 108 (15.41%) Databricks
2 81 (11.55%) Oracle
3 61 (8.70%) Google
4 58 (8.27%) Cloudera
5 53 (7.56%) Microsoft
6 17 (2.43%) Confluent
6 17 (2.43%) Hortonworks
7 14 (2.00%) BiZZdesign
7 14 (2.00%) SAP
7 14 (2.00%) Teradata
8 13 (1.85%) Informatica
8 13 (1.85%) Talend
9 12 (1.71%) Alteryx
9 12 (1.71%) Bloomberg
10 11 (1.57%) MapR
10 11 (1.57%) StreamSets
11 9 (1.28%) Adobe
12 7 (1.00%) Ab Initio
13 5 (0.71%) IBM
13 5 (0.71%) Red Hat