Period
to 14 August 2020

The following table provides summary statistics for contract job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 14 August 2020 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
UK > England
6 months to
14 Aug 2020
Same period 2019 Same period 2018
Rank 185 161 182
Rank change year-on-year -24 +21 +15
Contract jobs citing Apache Spark 432 1,465 1,460
As % of all contract jobs advertised in England 1.43% 1.77% 1.56%
As % of the Application Platforms category 9.66% 11.41% 9.32%
Number of daily rates quoted 291 1,085 1,061
Median daily rate £533 £550 £550
Median daily rate % change year-on-year -3.18% - -
10th Percentile £406 £450 £425
90th Percentile £700 £700 £688
UK median daily rate £525 £550 £550
% change year-on-year -4.55% - -
Number of hourly rates quoted 11 3 7
Median hourly rate £35.00 £24.65 £22.50
Median hourly rate % change year-on-year +41.99% +9.56% +16.61%
UK median hourly rate £35.00 £24.65 £35.00
% change year-on-year +41.99% -29.57% +81.39%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Contract vacancies with a requirement for application platform skills 4,471 12,842 15,664
As % of all contract IT jobs advertised in England 14.81% 15.52% 16.76%
Number of daily rates quoted 2,832 8,230 10,347
Median daily rate £475 £475 £455
Median daily rate % change year-on-year - +4.40% +1.11%
10th Percentile £300 £268 £275
90th Percentile £663 £638 £625
UK median daily rate £475 £468 £450
% change year-on-year +1.60% +3.89% +2.86%
Number of hourly rates quoted 129 273 374
Median hourly rate £26.96 £20.52 £21.00
Median hourly rate % change year-on-year +31.38% -2.29% +5.00%
10th Percentile £12.63 £12.00 £12.50
90th Percentile £47.08 £53.75 £53.28
Median hourly rate £28.00 £21.00 £20.50
% change year-on-year +33.33% +2.44% +5.68%

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a proportion of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Contractor Daily Rate Trend in England

3-month moving average daily rate quoted in jobs citing Apache Spark in England.

Daily rate trend for Apache Spark in England

Apache Spark
Daily Rate Histogram in England

Daily rate distribution for jobs citing Apache Spark in England over the 6 months to 14 August 2020.

Daily rate histogram for Apache Spark in England

Apache Spark
Contractor Hourly Rate Trend in England

3-month moving average hourly rates quoted in jobs citing Apache Spark in England.

Hourly rate trend for Apache Spark in England

Apache Spark
Hourly Rate Histogram in England

Hourly rate distribution of jobs citing Apache Spark in England over the 6 months to 14 August 2020.

Hourly rate histogram for Apache Spark in England

Apache Spark
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the England region over the 6 months to 14 August 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London -30 321 £550 -4.35% 85
South East +85 50 £475 -9.52% 12
North of England +14 26 £475 -9.31% 7
North West +43 18 £445 -15.24%
East of England +40 16 £675 +28.57% 12
South West +59 7 £545 -1.80% 3
Yorkshire +20 7 £475 -5.00% 7
Midlands +76 4 £400 -22.93% 6
East Midlands +55 3 - - 4
West Midlands +56 1 £400 -20.00% 2
Apache Spark
UK

For the 6 months to 14 August 2020, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region with a requirement for Apache Spark.

1 280 (64.81%) Python
2 274 (63.43%) Big Data
3 221 (51.16%) Hadoop
4 187 (43.29%) Scala
5 183 (42.36%) SQL
6 168 (38.89%) Java
7 160 (37.04%) AWS
8 158 (36.57%) Agile Software Development
9 143 (33.10%) Kafka
10 122 (28.24%) Analytics
11 115 (26.62%) Azure
12 113 (26.16%) Apache Hive
13 95 (21.99%) GCP
14 93 (21.53%) Data Science
15 92 (21.30%) Machine Learning
16 88 (20.37%) Finance
16 88 (20.37%) ETL
17 86 (19.91%) NoSQL
18 79 (18.29%) PySpark
19 77 (17.82%) Kubernetes
20 69 (15.97%) Apache
21 68 (15.74%) DevOps
22 65 (15.05%) Docker
22 65 (15.05%) Data Modelling
22 65 (15.05%) Banking
23 63 (14.58%) Microservices
23 63 (14.58%) Git (software)
24 62 (14.35%) Cloudera
25 58 (13.43%) Analytical Skills
26 56 (12.96%) Computer Science

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 69 (15.97%) Apache
2 23 (5.32%) Elasticsearch
3 19 (4.40%) Apache Pig
4 18 (4.17%) Apache Airflow
5 9 (2.08%) SAS
6 8 (1.85%) Confluence
6 8 (1.85%) Jupyter
7 6 (1.39%) Tornado
8 4 (0.93%) nginx
9 3 (0.69%) Tomcat
10 2 (0.46%) Cloud Foundry
10 2 (0.46%) GemFire
10 2 (0.46%) Luigi
10 2 (0.46%) WebSphere
11 1 (0.23%) Apache Solr
11 1 (0.23%) R Markdown
11 1 (0.23%) SharePoint
11 1 (0.23%) WebLogic
Applications
1 4 (0.93%) MS Visio
2 1 (0.23%) Microsoft Excel
2 1 (0.23%) Microsoft PowerPoint
Business Applications
1 5 (1.16%) SAP Oil and Gas
2 1 (0.23%) FINCAD Analytics
2 1 (0.23%) SAP CRM
2 1 (0.23%) SAP ERP
2 1 (0.23%) SAP IS-U
2 1 (0.23%) Shopify
2 1 (0.23%) Sparx Enterprise Architect
Cloud Services
1 160 (37.04%) AWS
2 115 (26.62%) Azure
3 95 (21.99%) GCP
4 29 (6.71%) Amazon S3
5 22 (5.09%) Azure Data Factory
6 21 (4.86%) Amazon EMR
7 20 (4.63%) Serverless
8 18 (4.17%) Power Platform
9 15 (3.47%) Amazon EC2
9 15 (3.47%) Amazon SQS
10 14 (3.24%) Cloud Computing
11 13 (3.01%) SaaS
12 10 (2.31%) AWS Lambda
13 9 (2.08%) GitHub
14 8 (1.85%) Amazon Kinesis
14 8 (1.85%) Azure Active Directory
15 7 (1.62%) Azure Functions
16 6 (1.39%) AWS CloudFormation
17 4 (0.93%) OpenShift
17 4 (0.93%) PaaS
Communications & Networking
1 28 (6.48%) SSL
2 25 (5.79%) HTTPS
3 11 (2.55%) HTTP
4 7 (1.62%) RTMP
5 6 (1.39%) Kerberos
6 4 (0.93%) Internet
7 3 (0.69%) DNS
7 3 (0.69%) Skype
8 2 (0.46%) 5G
8 2 (0.46%) DSL
9 1 (0.23%) Firewall
9 1 (0.23%) MQTT
9 1 (0.23%) VPN
9 1 (0.23%) Wi-Fi
9 1 (0.23%) Wireless
Database & Business Intelligence
1 274 (63.43%) Big Data
2 221 (51.16%) Hadoop
3 113 (26.16%) Apache Hive
4 86 (19.91%) NoSQL
5 52 (12.04%) Impala
6 43 (9.95%) Data Lake
7 41 (9.49%) Data Warehouse
8 40 (9.26%) HBase
9 37 (8.56%) Relational Database
10 33 (7.64%) MongoDB
11 28 (6.48%) Amazon Redshift
12 27 (6.25%) Apache Cassandra
13 21 (4.86%) SQL Server
14 20 (4.63%) Data Mining
14 20 (4.63%) PostgreSQL
14 20 (4.63%) Spark SQL
15 18 (4.17%) BigQuery
15 18 (4.17%) Power BI
16 15 (3.47%) Redis
16 15 (3.47%) Sqoop
Development Applications
1 63 (14.58%) Git (software)
2 40 (9.26%) Jenkins
3 17 (3.94%) GitLab
4 15 (3.47%) Maven
5 13 (3.01%) JIRA
6 12 (2.78%) TeamCity
7 9 (2.08%) Artifactory
7 9 (2.08%) Gradle
7 9 (2.08%) JUnit
8 8 (1.85%) Visual Studio Team System
9 7 (1.62%) Hudson
9 7 (1.62%) Mockito
9 7 (1.62%) NUnit
9 7 (1.62%) ProC
10 5 (1.16%) Bitbucket
10 5 (1.16%) Sonatype Nexus
11 4 (0.93%) Cucumber
11 4 (0.93%) ERwin
11 4 (0.93%) Octopus Deploy
11 4 (0.93%) Team Foundation Server
General
1 88 (20.37%) Finance
2 65 (15.05%) Banking
3 28 (6.48%) Front Office
4 27 (6.25%) Investment Banking
5 25 (5.79%) Retail
6 19 (4.40%) Telecoms
7 14 (3.24%) Marketing
8 12 (2.78%) Legal
9 11 (2.55%) Pharmaceutical
10 10 (2.31%) Public Sector
11 7 (1.62%) Electronics
11 7 (1.62%) Manufacturing
12 5 (1.16%) Aerospace
12 5 (1.16%) Financial Institution
12 5 (1.16%) Games
13 3 (0.69%) Law
14 2 (0.46%) Automotive
15 1 (0.23%) Billing
15 1 (0.23%) Dutch Language
15 1 (0.23%) Publishing
Job Titles
1 156 (36.11%) Data Engineer
2 126 (29.17%) Developer
3 64 (14.81%) Big Data Engineer
4 53 (12.27%) Java Developer
5 50 (11.57%) Data Scientist
6 42 (9.72%) Python Engineer
7 40 (9.26%) Architect
8 26 (6.02%) Scala Developer
9 21 (4.86%) Azure Engineer
9 21 (4.86%) Big Data Developer
10 20 (4.63%) Data Architect
10 20 (4.63%) Hadoop Engineer
11 18 (4.17%) Python Developer
11 18 (4.17%) Senior Developer
12 17 (3.94%) AWS Engineer
12 17 (3.94%) DevOps Engineer
13 16 (3.70%) Senior Data Scientist
14 13 (3.01%) Full Stack Developer
14 13 (3.01%) Senior Data Engineer
15 12 (2.78%) Machine Learning Engineer
Libraries, Frameworks & Software Standards
1 143 (33.10%) Kafka
2 79 (18.29%) PySpark
3 48 (11.11%) Spring
4 39 (9.03%) TensorFlow
5 38 (8.80%) RESTful
6 24 (5.56%) JAAS
6 24 (5.56%) Spring Security
7 22 (5.09%) Pandas
8 20 (4.63%) Akka
8 20 (4.63%) Guice
9 18 (4.17%) HTML
10 17 (3.94%) H2O
10 17 (3.94%) HTML5
10 17 (3.94%) React
10 17 (3.94%) scikit-learn
10 17 (3.94%) Spring Boot
11 16 (3.70%) .NET
11 16 (3.70%) MLlib
12 15 (3.47%) AngularJS
12 15 (3.47%) Spark Streaming
Miscellaneous
1 58 (13.43%) Analytical Skills
2 39 (9.03%) Algorithms
3 18 (4.17%) Greenfield Project
4 16 (3.70%) Credit Risk
4 16 (3.70%) Foreign Exchange (FX)
5 12 (2.78%) FMCG
6 11 (2.55%) Analytical Mindset
6 11 (2.55%) Cloud Native
6 11 (2.55%) Management Information System
7 10 (2.31%) Enterprise Software
7 10 (2.31%) Market Risk
7 10 (2.31%) Virtual Team
8 8 (1.85%) PMI
8 8 (1.85%) Self-Motivation
9 6 (1.39%) Data Structures
9 6 (1.39%) Life Science
9 6 (1.39%) Reference Data
10 4 (0.93%) Equities
10 4 (0.93%) Fixed Income
10 4 (0.93%) Grid Computing
Operating Systems
1 39 (9.03%) Linux
2 8 (1.85%) Debian
2 8 (1.85%) Red Hat Enterprise Linux
2 8 (1.85%) Unix
3 7 (1.62%) Windows
4 6 (1.39%) Ubuntu
Processes & Methodologies
1 158 (36.57%) Agile Software Development
2 122 (28.24%) Analytics
3 93 (21.53%) Data Science
4 92 (21.30%) Machine Learning
5 88 (20.37%) ETL
6 68 (15.74%) DevOps
7 65 (15.05%) Data Modelling
8 63 (14.58%) Microservices
9 56 (12.96%) Computer Science
10 51 (11.81%) Business Intelligence
10 51 (11.81%) Data Engineering
11 48 (11.11%) Mathematics
12 47 (10.88%) Visualisation
13 45 (10.42%) Continuous Integration
14 42 (9.72%) Artificial Intelligence
15 40 (9.26%) Data Analytics
15 40 (9.26%) Scrum
15 40 (9.26%) Software Engineering
16 38 (8.80%) SDLC
17 37 (8.56%) TDD
Programming Languages
1 280 (64.81%) Python
2 187 (43.29%) Scala
3 183 (42.36%) SQL
4 168 (38.89%) Java
5 36 (8.33%) JavaScript
6 32 (7.41%) Java 8
7 29 (6.71%) C#
8 27 (6.25%) Kotlin
9 24 (5.56%) R
10 16 (3.70%) Ruby
11 14 (3.24%) Go
12 13 (3.01%) T-SQL
13 11 (2.55%) Shell Script
14 9 (2.08%) Haskell
15 8 (1.85%) Bash Shell
15 8 (1.85%) C++
15 8 (1.85%) Groovy
15 8 (1.85%) PHP
15 8 (1.85%) PowerShell
16 6 (1.39%) Perl
Qualifications
1 40 (9.26%) Security Cleared
2 39 (9.03%) Degree
3 38 (8.80%) SC Cleared
4 17 (3.94%) Numerate Degree
5 11 (2.55%) Computer Science Degree
6 8 (1.85%) PMI Certification
7 7 (1.62%) Master's Degree
8 6 (1.39%) PhD
9 4 (0.93%) Microsoft Certification
10 1 (0.23%) AWS Certification
10 1 (0.23%) BPSS Clearance
10 1 (0.23%) CSPO
10 1 (0.23%) DBS Check
10 1 (0.23%) DV Cleared
Quality Assurance & Compliance
1 7 (1.62%) MiFID
2 5 (1.16%) Data Quality
3 4 (0.93%) GxP
4 3 (0.69%) SLA
5 2 (0.46%) Sarbanes-Oxley
6 1 (0.23%) CDISC
6 1 (0.23%) QA
System Software
1 65 (15.05%) Docker
2 42 (9.72%) HDFS
3 8 (1.85%) Active Directory
3 8 (1.85%) Apache ZooKeeper
4 6 (1.39%) Apache Flume
Systems Management
1 77 (17.82%) Kubernetes
2 29 (6.71%) Terraform
3 24 (5.56%) Ansible
4 22 (5.09%) Prometheus
4 22 (5.09%) Zipkin
5 17 (3.94%) Oozie
6 13 (3.01%) Hadoop YARN
7 6 (1.39%) Opscode Chef
8 5 (1.16%) Puppet
9 3 (0.69%) Ambari
9 3 (0.69%) Grafana
10 1 (0.23%) Docker Swarm
10 1 (0.23%) Fluentd
10 1 (0.23%) FortiGate
10 1 (0.23%) Istio
10 1 (0.23%) Knative
10 1 (0.23%) logstash
10 1 (0.23%) Single Sign-On
Vendors
1 62 (14.35%) Cloudera
2 42 (9.72%) Tableau
3 40 (9.26%) Databricks
4 30 (6.94%) Google
5 28 (6.48%) Oracle
6 22 (5.09%) Hortonworks
7 20 (4.63%) Teradata
8 16 (3.70%) Microsoft
8 16 (3.70%) Talend
9 13 (3.01%) SAP
10 11 (2.55%) MapR
11 10 (2.31%) StreamSets
12 8 (1.85%) IBM
12 8 (1.85%) Informatica
12 8 (1.85%) MuleSoft
13 7 (1.62%) Red Hat
13 7 (1.62%) Salesforce.com
14 4 (0.93%) Ab Initio
15 3 (0.69%) Birst
15 3 (0.69%) Sybase