Period
to 22 August 2019

The following table provides summary statistics for contract job vacancies advertised in the UK excluding London with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 22 August 2019 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
UK > UK excluding London
6 months to
22 Aug 2019
Same period 2018 Same period 2017
Rank 245 255 323
Rank change year-on-year +10 +68 +188
Contract jobs citing Apache Spark 492 540 343
As % of all contract IT jobs advertised in the UK excluding London 1.02% 0.99% 0.67%
As % of the Application Platforms category 6.69% 6.06% 4.10%
Number of daily rates quoted 297 363 178
UK excluding London median daily rate £525 £500 £450
Median daily rate % change year-on-year +5.00% +11.11% +5.88%
10th Percentile £413 £413 £310
90th Percentile £638 £611 £588
UK median daily rate £550 £550 £550
Number of hourly rates quoted 3 4 3
UK excluding London median hourly rate £24.65 £22.25 £19.30
Median hourly rate % change year-on-year +10.79% +15.31% -5.88%
10th Percentile £21.93 £18.20 £16.64
90th Percentile £32.93 £41.10 £32.39
UK median hourly rate £24.65 £35.00 £19.30
% change year-on-year -29.57% +81.39% -5.88%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in the UK excluding London with a requirement for application platform skills.

All Application Platform Skills
UK excluding London
Contract vacancies with a requirement for application platform skills 7,359 8,911 8,368
As % of all contract IT jobs advertised in the UK excluding London 15.24% 16.33% 16.45%
Number of daily rates quoted 4,377 5,455 5,045
UK excluding London median daily rate £425 £425 £400
Median daily rate % change year-on-year - +6.25% +2.56%
10th Percentile £250 £255 £239
90th Percentile £588 £569 £550
UK median daily rate £463 £450 £438
% change year-on-year +2.78% +2.86% +4.17%
Number of hourly rates quoted 251 295 310
UK excluding London median hourly rate £21.00 £20.00 £19.00
Median hourly rate % change year-on-year +5.00% +5.26% -8.35%
10th Percentile £11.31 £12.00 £10.50
90th Percentile £53.75 £48.75 £50.00
UK median hourly rate £21.16 £20.50 £19.30
% change year-on-year +3.22% +6.25% -4.46%

Apache Spark
Job Vacancy Trend in the UK excluding London

Job postings citing Apache Spark as a percentage of all IT jobs advertised in the UK excluding London.

Job vacancy trend for Apache Spark in the UK excluding London

Apache Spark
Contractor Daily Rate Trend in the UK excluding London

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Apache Spark in the UK excluding London.

Contractor daily rate trend for Apache Spark in the UK excluding London

Apache Spark
Contractor Daily Rate Histogram in the UK excluding London

The daily rate distribution of IT jobs citing Apache Spark in the UK excluding London over the 6 months to 22 August 2019.

Contractor daily rate histogram for Apache Spark in the UK excluding London

Apache Spark
Contractor Hourly Rate Trend in the UK excluding London

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing Apache Spark in the UK excluding London.

Contractor hourly rate trend for Apache Spark in the UK excluding London

Apache Spark
Contractor Hourly Rate Histogram in the UK excluding London

The hourly rate distribution of IT jobs citing Apache Spark in the UK excluding London over the 6 months to 22 August 2019.

Contractor hourly rate histogram for Apache Spark in the UK excluding London

Apache Spark
Contract Job Locations in the UK excluding London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the UK excluding London region over the 6 months to 22 August 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
South East -28 140 £518 +3.50% 16
North of England +36 135 £525 - 24
Scotland +79 97 £475 - 19
North West +49 66 £525 -4.55% 17
Yorkshire +17 64 £500 +5.26% 6
East of England +20 47 £520 +8.90% 14
South West -40 37 £560 +6.67% 8
Midlands +32 29 £525 +7.69% 6
West Midlands +42 16 £507 +35.07% 6
East Midlands +19 13 £525 +2.44%
North East +24 5 £563 -4.26% 1
Wales +9 4 £613 +11.36% 4
Northern Ireland -18 3 - - 1
Apache Spark
UK

For the 6 months to 22 August 2019, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the UK excluding London region with a requirement for Apache Spark.

1 345 (70.12%) Big Data
2 328 (66.67%) Hadoop
3 267 (54.27%) Python
4 201 (40.85%) Kafka
5 198 (40.24%) Apache Hive
6 195 (39.63%) SQL
7 182 (36.99%) Java
8 163 (33.13%) Scala
9 161 (32.72%) AWS
10 159 (32.32%) Agile Software Development
11 133 (27.03%) Linux
12 125 (25.41%) Jenkins
13 120 (24.39%) Git (software)
13 120 (24.39%) HBase
14 114 (23.17%) NoSQL
15 110 (22.36%) Impala
16 109 (22.15%) Analytics
17 100 (20.33%) Data Science
18 93 (18.90%) HDFS
19 92 (18.70%) DevOps
20 90 (18.29%) Continuous Integration
21 85 (17.28%) Azure
22 82 (16.67%) Data Warehouse
23 78 (15.85%) Data Modelling
24 73 (14.84%) Apache
24 73 (14.84%) Machine Learning
25 71 (14.43%) ETL
26 70 (14.23%) Finance
27 68 (13.82%) Software Engineering
28 67 (13.62%) Docker

Apache Spark
Co-occurring IT Skills in the UK excluding London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 73 (14.84%) Apache
2 28 (5.69%) Elasticsearch
3 20 (4.07%) Apache Airflow
4 17 (3.46%) Jupyter
5 16 (3.25%) Apache Pig
6 15 (3.05%) SAS
7 13 (2.64%) Luigi
8 9 (1.83%) Apache Solr
8 9 (1.83%) nginx
9 5 (1.02%) Tomcat
10 4 (0.81%) ColdFusion
10 4 (0.81%) Jetty
11 3 (0.61%) Cloud Foundry
11 3 (0.61%) Confluence
11 3 (0.61%) GemFire
11 3 (0.61%) WebSphere
12 1 (0.20%) Apache Qpid
12 1 (0.20%) Hyperledger
12 1 (0.20%) JBoss
12 1 (0.20%) Oracle Workflow
Applications
1 8 (1.63%) Stata
2 7 (1.42%) SPSS
3 6 (1.22%) Microsoft Excel
4 5 (1.02%) Microsoft Office
Business Applications
1 1 (0.20%) Cognos Business Intelligence
Cloud Services
1 161 (32.72%) AWS
2 85 (17.28%) Azure
3 65 (13.21%) Amazon EMR
4 56 (11.38%) Amazon S3
5 31 (6.30%) Amazon EC2
5 31 (6.30%) Amazon SQS
6 30 (6.10%) GCP
6 30 (6.10%) Serverless
7 28 (5.69%) AWS Lambda
8 26 (5.28%) Azure Data Factory
9 24 (4.88%) GitHub
9 24 (4.88%) Power Platform
10 19 (3.86%) Amazon Kinesis
10 19 (3.86%) AWS CloudFormation
10 19 (3.86%) PaaS
11 13 (2.64%) Amazon ECS
12 11 (2.24%) Cloud Computing
13 9 (1.83%) Amazon CloudWatch
14 5 (1.02%) Amazon Glacier
14 5 (1.02%) Virtual Private Cloud
Communications & Networking
1 25 (5.08%) Kerberos
2 14 (2.85%) DNS
2 14 (2.85%) SSL
3 10 (2.03%) MQTT
4 5 (1.02%) DSL
4 5 (1.02%) Firewall
5 4 (0.81%) FTP
5 4 (0.81%) HTTP
6 3 (0.61%) Internet
6 3 (0.61%) SAN
7 2 (0.41%) 2.5G
7 2 (0.41%) 2G
7 2 (0.41%) 3G
7 2 (0.41%) 4G
7 2 (0.41%) 5G
7 2 (0.41%) F5 BIG-IP GTM
7 2 (0.41%) F5 BIG-IP LTM
7 2 (0.41%) Intrusion Detection
7 2 (0.41%) Multicast
Database & Business Intelligence
1 345 (70.12%) Big Data
2 328 (66.67%) Hadoop
3 198 (40.24%) Apache Hive
4 120 (24.39%) HBase
5 114 (23.17%) NoSQL
6 110 (22.36%) Impala
7 82 (16.67%) Data Warehouse
8 50 (10.16%) MongoDB
9 49 (9.96%) Amazon Redshift
10 46 (9.35%) Data Lake
11 42 (8.54%) PostgreSQL
12 40 (8.13%) Apache Cassandra
12 40 (8.13%) Relational Database
13 38 (7.72%) MySQL
14 28 (5.69%) Parquet
14 28 (5.69%) SQL Server
15 24 (4.88%) Amazon Aurora
15 24 (4.88%) Power BI
16 22 (4.47%) Spark SQL
17 19 (3.86%) Amazon DynamoDB
Development Applications
1 125 (25.41%) Jenkins
2 120 (24.39%) Git (software)
3 25 (5.08%) Maven
3 25 (5.08%) Sonatype Nexus
4 21 (4.27%) Gradle
4 21 (4.27%) IntelliJ
5 20 (4.07%) Artifactory
5 20 (4.07%) Subversion
6 19 (3.86%) JIRA
7 17 (3.46%) Atlassian Bamboo
7 17 (3.46%) Eclipse
8 13 (2.64%) JUnit
9 10 (2.03%) Cucumber
9 10 (2.03%) GitLab
10 9 (1.83%) FindBugs
10 9 (1.83%) Selenium
11 7 (1.42%) TeamCity
12 6 (1.22%) Mercurial
13 5 (1.02%) Hudson
13 5 (1.02%) SQL Server Management Studio (SSMS)
General
1 70 (14.23%) Finance
2 46 (9.35%) Banking
3 16 (3.25%) Investment Banking
4 13 (2.64%) Public Sector
5 12 (2.44%) Retail
6 9 (1.83%) Marketing
7 8 (1.63%) Retail Banking
8 7 (1.42%) Legal
9 5 (1.02%) Telecoms
10 4 (0.81%) Automotive
10 4 (0.81%) Manufacturing
10 4 (0.81%) Pharmaceutical
11 2 (0.41%) Aerospace
12 1 (0.20%) International Banking
12 1 (0.20%) Police
12 1 (0.20%) Private Banking
Job Titles
1 152 (30.89%) Developer
2 144 (29.27%) Data Engineer
3 64 (13.01%) Big Data Developer
4 63 (12.80%) Big Data Engineer
5 59 (11.99%) Architect
6 45 (9.15%) Data Architect
7 43 (8.74%) Scala Developer
8 40 (8.13%) Java Developer
9 28 (5.69%) Hadoop Developer
10 27 (5.49%) Python Developer
11 25 (5.08%) Data Scientist
12 24 (4.88%) DevOps Engineer
13 19 (3.86%) Hadoop Engineer
14 18 (3.66%) Apache Spark Developer
15 17 (3.46%) Analyst
16 16 (3.25%) AWS Engineer
17 14 (2.85%) Administrator
17 14 (2.85%) Python Engineer
18 12 (2.44%) Platform Engineer
18 12 (2.44%) Solutions Architect
Libraries, Frameworks & Software Standards
1 201 (40.85%) Kafka
2 43 (8.74%) Spring
3 32 (6.50%) Apache Avro
4 29 (5.89%) Web Services
5 26 (5.28%) RabbitMQ
6 23 (4.67%) Spring Boot
7 21 (4.27%) PySpark
8 19 (3.86%) scikit-learn
9 16 (3.25%) Spark Streaming
10 14 (2.85%) NumPy
10 14 (2.85%) Pandas
11 13 (2.64%) AngularJS
11 13 (2.64%) RESTful
12 11 (2.24%) MLlib
13 9 (1.83%) Apache NiFi
13 9 (1.83%) Django
13 9 (1.83%) Flask
14 8 (1.63%) Middleware
14 8 (1.63%) Natural Language Processing
14 8 (1.63%) Pyramid
Miscellaneous
1 59 (11.99%) Analytical Skills
2 28 (5.69%) Data Structures
3 25 (5.08%) Greenfield Project
4 21 (4.27%) Algorithms
5 18 (3.66%) Self-Motivation
6 11 (2.24%) Cloud Native
7 10 (2.03%) User Experience
8 9 (1.83%) Replication
9 7 (1.42%) Life Science
9 7 (1.42%) Management Information System
9 7 (1.42%) N-Tier
10 5 (1.02%) Public Cloud
11 4 (0.81%) Linux Command Line
11 4 (0.81%) Smart Home
11 4 (0.81%) Virtual Team
12 3 (0.61%) Enterprise Software
12 3 (0.61%) Mobile App
12 3 (0.61%) NHS
13 2 (0.41%) Internet of Things
13 2 (0.41%) Reference Data
Operating Systems
1 133 (27.03%) Linux
2 55 (11.18%) Unix
3 15 (3.05%) Windows
4 14 (2.85%) Red Hat Enterprise Linux
5 11 (2.24%) CentOS
6 4 (0.81%) SUSE Linux Enterprise Server
6 4 (0.81%) Windows Server
7 1 (0.20%) Android
7 1 (0.20%) Apple iOS
7 1 (0.20%) Oracle Linux
7 1 (0.20%) Windows Server 2012
Processes & Methodologies
1 159 (32.32%) Agile Software Development
2 109 (22.15%) Analytics
3 100 (20.33%) Data Science
4 92 (18.70%) DevOps
5 90 (18.29%) Continuous Integration
6 78 (15.85%) Data Modelling
7 73 (14.84%) Machine Learning
8 71 (14.43%) ETL
9 68 (13.82%) Software Engineering
10 58 (11.79%) TDD
11 54 (10.98%) Data Analysis
12 52 (10.57%) Business Intelligence
13 51 (10.37%) Migration
13 51 (10.37%) Scrum
14 50 (10.16%) Data Engineering
14 50 (10.16%) Problem-Solving
15 47 (9.55%) MapReduce
15 47 (9.55%) Test Automation
15 47 (9.55%) Visualisation
16 37 (7.52%) Data Ingestion
Programming Languages
1 267 (54.27%) Python
2 195 (39.63%) SQL
3 182 (36.99%) Java
4 163 (33.13%) Scala
5 65 (13.21%) R
6 29 (5.89%) Shell Script
7 28 (5.69%) Ruby
8 19 (3.86%) JavaScript
9 18 (3.66%) Perl
10 17 (3.46%) Bash Shell
11 14 (2.85%) C#
12 13 (2.64%) C
13 9 (1.83%) Java 8
14 8 (1.63%) Go
15 7 (1.42%) C++
16 6 (1.22%) MATLAB
16 6 (1.22%) PL/SQL
17 5 (1.02%) Groovy
17 5 (1.02%) T-SQL
18 4 (0.81%) PHP
Qualifications
1 48 (9.76%) Degree
2 24 (4.88%) Security Cleared
3 18 (3.66%) SC Cleared
4 6 (1.22%) BPSS Clearance
4 6 (1.22%) Master's Degree
4 6 (1.22%) PhD
5 5 (1.02%) AWS Certification
6 4 (0.81%) Computer Science Degree
6 4 (0.81%) ISTQB
7 3 (0.61%) DV Cleared
8 2 (0.41%) CCIE
8 2 (0.41%) CCNP
8 2 (0.41%) Cisco Certification
8 2 (0.41%) HNC
8 2 (0.41%) HND
8 2 (0.41%) ITIL Certification
8 2 (0.41%) Numerate Degree
9 1 (0.20%) DBS Check
9 1 (0.20%) TOGAF Certification
Quality Assurance & Compliance
1 14 (2.85%) Data Quality
2 8 (1.63%) QA
3 7 (1.42%) GDPR
4 3 (0.61%) IFRS
5 2 (0.41%) Disclosure Scotland
5 2 (0.41%) EMIR
5 2 (0.41%) MiFID
6 1 (0.20%) HL7
6 1 (0.20%) ISO/IEC 27001
6 1 (0.20%) PCI DSS
System Software
1 93 (18.90%) HDFS
2 67 (13.62%) Docker
3 25 (5.08%) Apache Flume
4 14 (2.85%) Apache ZooKeeper
5 12 (2.44%) Active Directory
6 3 (0.61%) Virtual Desktop
6 3 (0.61%) Virtual Servers
7 1 (0.20%) Virtual Machines
Systems Management
1 52 (10.57%) Ansible
1 52 (10.57%) Terraform
2 43 (8.74%) Oozie
3 37 (7.52%) Kubernetes
4 31 (6.30%) Puppet
5 29 (5.89%) Hadoop YARN
6 28 (5.69%) Opscode Chef
7 6 (1.22%) logstash
8 5 (1.02%) Control-M
9 4 (0.81%) Graphite
9 4 (0.81%) Kibana
9 4 (0.81%) Nagios
9 4 (0.81%) StatsD
10 3 (0.61%) Ambari
11 2 (0.41%) Consul
11 2 (0.41%) Docker Swarm
11 2 (0.41%) FortiGate
11 2 (0.41%) Mesos
11 2 (0.41%) Prometheus
11 2 (0.41%) Salt
Vendors
1 61 (12.40%) Cloudera
2 41 (8.33%) Oracle
2 41 (8.33%) Tableau
3 36 (7.32%) Microsoft
4 30 (6.10%) Databricks
4 30 (6.10%) Hortonworks
5 24 (4.88%) Google
6 15 (3.05%) Informatica
7 13 (2.64%) Splunk
8 12 (2.44%) Red Hat
8 12 (2.44%) Talend
9 11 (2.24%) MapR
10 9 (1.83%) Teradata
11 8 (1.63%) MicroStrategy
11 8 (1.63%) Minitab
12 4 (0.81%) CA Agile Central
12 4 (0.81%) Dataiku
12 4 (0.81%) New Relic
12 4 (0.81%) StreamSets
13 3 (0.61%) Ab Initio