Period
to 21 January 2020

The following table provides summary statistics for contract job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 21 January 2020 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
UK > England
6 months to
21 Jan 2020
Same period 2019 Same period 2018
Rank 211 181 172
Rank change year-on-year -30 -9 +33
Contract jobs citing Apache Spark 844 1,385 1,455
As % of all contract IT jobs advertised in England 1.27% 1.53% 1.60%
As % of the Application Platforms category 8.86% 9.26% 9.63%
Number of daily rates quoted 619 982 1,078
England median daily rate £563 £550 £550
Median daily rate % change year-on-year +2.27% - -
10th Percentile £413 £438 £400
90th Percentile £725 £725 £675
UK median daily rate £550 £550 £550
Number of hourly rates quoted 2 4 10
England median hourly rate £38.58 £50.00 £23.42
Median hourly rate % change year-on-year -22.85% +113.49% -61.13%
10th Percentile £27.31 £42.50 £16.64
90th Percentile £50.84 £140.98 £51.88
UK median hourly rate £38.58 £50.00 £23.42
% change year-on-year -22.85% +113.49% +14.24%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Contract vacancies with a requirement for application platform skills 9,522 14,955 15,103
As % of all contract IT jobs advertised in England 14.35% 16.51% 16.61%
Number of daily rates quoted 6,262 9,759 10,146
England median daily rate £450 £475 £450
Median daily rate % change year-on-year -5.26% +5.56% +5.88%
10th Percentile £263 £281 £267
90th Percentile £638 £638 £625
UK median daily rate £450 £473 £450
% change year-on-year -4.86% +5.11% +5.88%
Number of hourly rates quoted 221 319 367
England median hourly rate £20.50 £20.00 £19.50
Median hourly rate % change year-on-year +2.50% +2.56% -2.50%
10th Percentile £11.16 £12.00 £11.25
90th Percentile £53.00 £47.50 £47.83
UK median hourly rate £21.00 £20.00 £19.50
% change year-on-year +5.00% +2.56% +0.33%

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a percentage of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Contractor Daily Rate Trend in England

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Apache Spark in England.

Contractor daily rate trend for Apache Spark in England

Apache Spark
Contractor Daily Rate Histogram in England

The daily rate distribution of IT jobs citing Apache Spark in England over the 6 months to 21 January 2020.

Contractor daily rate histogram for Apache Spark in England

Apache Spark
Contractor Hourly Rate Trend in England

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing Apache Spark in England.

Contractor hourly rate trend for Apache Spark in England

Apache Spark
Contractor Hourly Rate Histogram in England

The hourly rate distribution of IT jobs citing Apache Spark in England over the 6 months to 21 January 2020.

Contractor hourly rate histogram for Apache Spark in England

Apache Spark
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the England region over the 6 months to 21 January 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London -9 551 £575 - 211
South East -20 120 £500 -4.76% 44
North of England +24 72 £550 +4.76% 18
Yorkshire +31 41 £525 +5.00% 9
East of England +29 36 £500 -9.09% 14
Midlands +56 33 £525 +16.67% 18
South West -19 32 £550 - 10
West Midlands +63 26 £525 - 15
North West +5 22 £550 -2.65% 8
North East +21 9 £600 +20.00% 1
East Midlands +18 7 £500 +11.11% 3
Apache Spark
UK

For the 6 months to 21 January 2020, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region with a requirement for Apache Spark.

1 539 (63.86%) Python
2 533 (63.15%) Big Data
3 491 (58.18%) Hadoop
4 344 (40.76%) SQL
5 312 (36.97%) AWS
6 295 (34.95%) Java
7 292 (34.60%) Kafka
8 288 (34.12%) Agile Software Development
9 274 (32.46%) Scala
10 235 (27.84%) Data Science
11 232 (27.49%) Apache Hive
12 212 (25.12%) Azure
13 210 (24.88%) Analytics
14 187 (22.16%) ETL
15 185 (21.92%) Finance
15 185 (21.92%) Machine Learning
16 168 (19.91%) Apache
17 157 (18.60%) NoSQL
18 149 (17.65%) DevOps
19 147 (17.42%) Git (software)
20 144 (17.06%) Linux
21 143 (16.94%) Docker
22 138 (16.35%) Data Engineering
23 132 (15.64%) Kubernetes
24 130 (15.40%) PySpark
25 129 (15.28%) Cloudera
26 126 (14.93%) Jenkins
27 125 (14.81%) GCP
28 118 (13.98%) Data Modelling
29 116 (13.74%) Computer Science

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 168 (19.91%) Apache
2 55 (6.52%) Elasticsearch
3 47 (5.57%) Jupyter
4 40 (4.74%) Apache Airflow
5 25 (2.96%) SAS
6 11 (1.30%) Luigi
6 11 (1.30%) OpenStack
7 10 (1.18%) Apache Pig
7 10 (1.18%) Confluence
8 6 (0.71%) Tomcat
9 5 (0.59%) Apache Solr
9 5 (0.59%) Cloud Foundry
9 5 (0.59%) DNN
9 5 (0.59%) nginx
10 4 (0.47%) GemFire
10 4 (0.47%) WebSphere
11 2 (0.24%) CMS
11 2 (0.24%) Pachyderm
11 2 (0.24%) Tornado
11 2 (0.24%) WebLogic
Applications
1 12 (1.42%) Microsoft Excel
2 1 (0.12%) Chatbot
2 1 (0.12%) Microsoft Office
2 1 (0.12%) Microsoft Project
Business Applications
1 9 (1.07%) FINCAD Analytics
2 4 (0.47%) SAP CRM
2 4 (0.47%) SAP ERP
2 4 (0.47%) SAP IS-U
3 3 (0.36%) Distributed Ledger
4 2 (0.24%) Adobe Campaign
4 2 (0.24%) NetSuite
5 1 (0.12%) Cognos Business Intelligence
5 1 (0.12%) SAP Oil and Gas
5 1 (0.12%) Sparx Enterprise Architect
Cloud Services
1 312 (36.97%) AWS
2 212 (25.12%) Azure
3 125 (14.81%) GCP
4 69 (8.18%) Amazon EMR
5 65 (7.70%) Amazon Kinesis
5 65 (7.70%) Amazon S3
6 56 (6.64%) Serverless
7 48 (5.69%) AWS Lambda
8 31 (3.67%) OpenShift
9 29 (3.44%) Power Platform
10 27 (3.20%) Azure Machine Learning
11 25 (2.96%) Amazon SQS
12 24 (2.84%) Azure Data Factory
12 24 (2.84%) GitHub
13 23 (2.73%) PaaS
14 20 (2.37%) AWS CloudFormation
15 19 (2.25%) Amazon EC2
16 18 (2.13%) Cloud Computing
17 14 (1.66%) Amazon EKS
18 13 (1.54%) AWS Step Functions
Communications & Networking
1 35 (4.15%) Kerberos
2 21 (2.49%) SSL
3 14 (1.66%) DNS
4 11 (1.30%) Firewall
5 7 (0.83%) HTTP
5 7 (0.83%) SAN
6 6 (0.71%) VPN
7 5 (0.59%) HTTPS
8 4 (0.47%) DSL
8 4 (0.47%) F5 BIG-IP GTM
8 4 (0.47%) F5 BIG-IP LTM
8 4 (0.47%) Internet
9 2 (0.24%) RTMP
9 2 (0.24%) TCP/IP
10 1 (0.12%) Multicast
10 1 (0.12%) SSH
Database & Business Intelligence
1 533 (63.15%) Big Data
2 491 (58.18%) Hadoop
3 232 (27.49%) Apache Hive
4 157 (18.60%) NoSQL
5 99 (11.73%) Impala
6 96 (11.37%) Data Warehouse
7 95 (11.26%) HBase
8 84 (9.95%) Relational Database
9 77 (9.12%) Amazon Redshift
10 75 (8.89%) PostgreSQL
11 72 (8.53%) Data Lake
12 67 (7.94%) MongoDB
13 65 (7.70%) SQL Server
14 61 (7.23%) Apache Cassandra
15 57 (6.75%) Amazon Athena
16 56 (6.64%) Flink
17 47 (5.57%) Spark SQL
18 40 (4.74%) BigQuery
19 36 (4.27%) MySQL
20 29 (3.44%) Power BI
Development Applications
1 147 (17.42%) Git (software)
2 126 (14.93%) Jenkins
3 43 (5.09%) JIRA
4 33 (3.91%) Maven
5 28 (3.32%) Sonatype Nexus
6 21 (2.49%) Artifactory
7 15 (1.78%) Selenium
8 13 (1.54%) Visual Studio Team System
9 12 (1.42%) Atlassian Bamboo
9 12 (1.42%) IBM UrbanCode
10 11 (1.30%) IntelliJ
10 11 (1.30%) Vagrant
11 10 (1.18%) Cucumber
12 9 (1.07%) Gradle
13 6 (0.71%) Bitbucket
13 6 (0.71%) ERwin
13 6 (0.71%) Gerrit
13 6 (0.71%) JUnit
13 6 (0.71%) Mercurial
13 6 (0.71%) Mockito
General
1 185 (21.92%) Finance
2 74 (8.77%) Banking
3 41 (4.86%) Front Office
4 29 (3.44%) Retail
5 25 (2.96%) Investment Banking
6 21 (2.49%) Public Sector
7 18 (2.13%) Marketing
8 17 (2.01%) Telecoms
9 14 (1.66%) Legal
10 12 (1.42%) Pharmaceutical
11 7 (0.83%) Games
11 7 (0.83%) Manufacturing
11 7 (0.83%) Retail Banking
12 2 (0.24%) Aerospace
12 2 (0.24%) Automotive
12 2 (0.24%) Corporate Banking
12 2 (0.24%) Publishing
13 1 (0.12%) Electronics
13 1 (0.12%) Financial Institution
13 1 (0.12%) Law
Job Titles
1 306 (36.26%) Data Engineer
2 195 (23.10%) Developer
3 117 (13.86%) Big Data Engineer
4 102 (12.09%) Data Scientist
5 85 (10.07%) Architect
6 71 (8.41%) Python Developer
7 66 (7.82%) Data Architect
8 50 (5.92%) AWS Engineer
9 49 (5.81%) Java Developer
10 46 (5.45%) Python Engineer
11 43 (5.09%) DevOps Engineer
12 42 (4.98%) Big Data Developer
13 41 (4.86%) Senior Data Engineer
14 39 (4.62%) Senior Developer
15 34 (4.03%) Platform Engineer
15 34 (4.03%) Senior Data Scientist
16 31 (3.67%) Big Data Architect
16 31 (3.67%) Scala Developer
17 25 (2.96%) Machine Learning Scientist
18 24 (2.84%) Hadoop Developer
Libraries, Frameworks & Software Standards
1 292 (34.60%) Kafka
2 130 (15.40%) PySpark
3 77 (9.12%) Pandas
4 53 (6.28%) Spark Streaming
5 50 (5.92%) TensorFlow
6 45 (5.33%) NumPy
6 45 (5.33%) React
6 45 (5.33%) Spring
7 42 (4.98%) scikit-learn
8 39 (4.62%) Web Services
9 30 (3.55%) AngularJS
10 29 (3.44%) Natural Language Processing
10 29 (3.44%) RESTful
11 28 (3.32%) Apache NiFi
12 27 (3.20%) Apache Beam
13 26 (3.08%) Keras
14 24 (2.84%) Spring Boot
15 21 (2.49%) MLlib
15 21 (2.49%) RabbitMQ
16 19 (2.25%) SciPy
Miscellaneous
1 89 (10.55%) Analytical Skills
2 80 (9.48%) Algorithms
3 46 (5.45%) Data Structures
4 45 (5.33%) Greenfield Project
5 26 (3.08%) Public Cloud
6 18 (2.13%) User Experience
7 16 (1.90%) Self-Motivation
8 14 (1.66%) Fixed Income
8 14 (1.66%) Foreign Exchange (FX)
8 14 (1.66%) Hedge funds
9 12 (1.42%) Credit Risk
9 12 (1.42%) Derivative
10 10 (1.18%) Management Information System
11 9 (1.07%) Grid Computing
11 9 (1.07%) Lidar
12 8 (0.95%) Data Centre
12 8 (0.95%) Smart City
13 6 (0.71%) Market Risk
14 5 (0.59%) Replication
14 5 (0.59%) Social Media
Operating Systems
1 144 (17.06%) Linux
2 39 (4.62%) Unix
3 32 (3.79%) Windows
4 20 (2.37%) Red Hat Enterprise Linux
5 17 (2.01%) CentOS
6 8 (0.95%) Ubuntu
6 8 (0.95%) Windows Server
7 6 (0.71%) VMS
8 4 (0.47%) Android
8 4 (0.47%) Apple iOS
9 2 (0.24%) SUSE Linux Enterprise Server
10 1 (0.12%) Debian
10 1 (0.12%) Mac OS X
10 1 (0.12%) Oracle Linux
10 1 (0.12%) Windows Server 2012
Processes & Methodologies
1 288 (34.12%) Agile Software Development
2 235 (27.84%) Data Science
3 210 (24.88%) Analytics
4 187 (22.16%) ETL
5 185 (21.92%) Machine Learning
6 149 (17.65%) DevOps
7 138 (16.35%) Data Engineering
8 118 (13.98%) Data Modelling
9 116 (13.74%) Computer Science
10 94 (11.14%) Continuous Integration
11 93 (11.02%) Visualisation
12 84 (9.95%) Software Engineering
13 83 (9.83%) Scrum
14 81 (9.60%) Microservices
15 80 (9.48%) Migration
16 79 (9.36%) Artificial Intelligence
17 77 (9.12%) TDD
18 70 (8.29%) Problem-Solving
19 69 (8.18%) Data Analysis
20 68 (8.06%) Open Source
Programming Languages
1 539 (63.86%) Python
2 344 (40.76%) SQL
3 295 (34.95%) Java
4 274 (32.46%) Scala
5 85 (10.07%) R
6 36 (4.27%) JavaScript
7 34 (4.03%) Bash Shell
7 34 (4.03%) C#
7 34 (4.03%) Go
8 27 (3.20%) C
8 27 (3.20%) C++
9 22 (2.61%) Ruby
10 19 (2.25%) Shell Script
11 18 (2.13%) Perl
12 17 (2.01%) Kotlin
13 13 (1.54%) PowerShell
14 11 (1.30%) Groovy
15 9 (1.07%) Java 8
16 8 (0.95%) Clojure
16 8 (0.95%) PL/SQL
Qualifications
1 113 (13.39%) Degree
2 37 (4.38%) PhD
3 30 (3.55%) Computer Science Degree
3 30 (3.55%) Security Cleared
4 26 (3.08%) Master's Degree
5 18 (2.13%) Numerate Degree
5 18 (2.13%) SC Cleared
6 9 (1.07%) AWS Certification
7 7 (0.83%) BPSS Clearance
8 6 (0.71%) Cisco Certification
9 5 (0.59%) DBS Check
10 4 (0.47%) CCIE
10 4 (0.47%) CCNP
11 3 (0.36%) Microsoft Certification
11 3 (0.36%) PMI Certification
11 3 (0.36%) PMP
12 2 (0.24%) GIAC
12 2 (0.24%) Linux+ Certification
12 2 (0.24%) Network+ Certification
12 2 (0.24%) SSCP
Quality Assurance & Compliance
1 19 (2.25%) QA
2 17 (2.01%) Data Quality
3 5 (0.59%) MiFID
4 4 (0.47%) Disclosure Scotland
4 4 (0.47%) EMIR
4 4 (0.47%) GDPR
4 4 (0.47%) PCI DSS
5 1 (0.12%) ISO/IEC 27001
5 1 (0.12%) WCAG
System Software
1 143 (16.94%) Docker
2 101 (11.97%) HDFS
3 17 (2.01%) Active Directory
4 9 (1.07%) VirtualBox
5 8 (0.95%) Apache Flume
6 7 (0.83%) Virtual Desktop
6 7 (0.83%) Virtual Servers
7 5 (0.59%) Apache ZooKeeper
8 2 (0.24%) VMware Infrastructure
Systems Management
1 132 (15.64%) Kubernetes
2 79 (9.36%) Ansible
3 75 (8.89%) Hadoop YARN
3 75 (8.89%) Terraform
4 29 (3.44%) Oozie
5 21 (2.49%) Puppet
6 16 (1.90%) Opscode Chef
7 9 (1.07%) Ambari
7 9 (1.07%) Docker Swarm
8 7 (0.83%) Kibana
9 6 (0.71%) logstash
9 6 (0.71%) Single Sign-On
10 4 (0.47%) FortiGate
10 4 (0.47%) Prometheus
11 3 (0.36%) Cobbler
11 3 (0.36%) Graphite
11 3 (0.36%) Mesos
11 3 (0.36%) Red Hat Satellite
12 2 (0.24%) Grafana
12 2 (0.24%) Salt
Vendors
1 129 (15.28%) Cloudera
2 83 (9.83%) Google
3 82 (9.72%) Oracle
4 81 (9.60%) Hortonworks
5 67 (7.94%) Microsoft
6 66 (7.82%) Tableau
7 61 (7.23%) Databricks
8 39 (4.62%) Informatica
9 26 (3.08%) Talend
10 23 (2.73%) IBM
10 23 (2.73%) Red Hat
11 22 (2.61%) Palantir
11 22 (2.61%) SAP
12 18 (2.13%) Teradata
13 17 (2.01%) MapR
14 16 (1.90%) Splunk
15 15 (1.78%) StreamSets
16 9 (1.07%) Bloomberg
16 9 (1.07%) Datactics
16 9 (1.07%) IBM Sterling