Period
to 6 June 2020

The following table provides summary statistics for contract job vacancies advertised in the UK excluding London with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 6 June 2020 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
UK > UK excluding London
6 months to
6 Jun 2020
Same period 2019 Same period 2018
Rank 238 229 242
Rank change year-on-year -9 +13 +139
Contract jobs citing Apache Spark 220 552 559
As % of all contract jobs advertised in the UK excluding London 0.90% 1.14% 1.05%
As % of the Application Platforms category 6.38% 7.27% 6.70%
Number of daily rates quoted 148 365 364
Median daily rate £500 £523 £500
Median daily rate % change year-on-year -4.31% +4.50% +5.26%
10th Percentile £411 £413 £413
90th Percentile £660 £650 £613
UK median daily rate £538 £550 £550
% change year-on-year -2.18% - -
Number of hourly rates quoted 9 0 4
Median hourly rate £35.00 - £22.25
UK median hourly rate £35.00 - £47.50

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in the UK excluding London with a requirement for application platform skills.

All Application Platform Skills
UK excluding London
Contract vacancies with a requirement for application platform skills 3,446 7,598 8,340
As % of all contract IT jobs advertised in the UK excluding London 14.12% 15.65% 15.70%
Number of daily rates quoted 2,125 4,530 5,169
Median daily rate £450 £430 £415
Median daily rate % change year-on-year +4.65% +3.61% +3.75%
10th Percentile £261 £254 £255
90th Percentile £588 £588 £575
UK median daily rate £475 £475 £450
% change year-on-year - +5.56% +5.88%
Number of hourly rates quoted 112 243 272
Median hourly rate £20.00 £17.00 £17.51
Median hourly rate % change year-on-year +17.65% -2.88% -2.75%
10th Percentile £12.63 £11.80 £11.00
90th Percentile £52.88 £48.60 £47.50
Median hourly rate £25.00 £18.73 £19.00
% change year-on-year +33.51% -1.45% -

Apache Spark
Job Vacancy Trend in the UK excluding London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in the UK excluding London.

Job vacancy trend for Apache Spark in the UK excluding London

Apache Spark
Contractor Daily Rate Trend in the UK excluding London

3-month moving average daily rate quoted in jobs citing Apache Spark in the UK excluding London.

Daily rate trend for Apache Spark in the UK excluding London

Apache Spark
Daily Rate Histogram in the UK excluding London

Daily rate distribution for jobs citing Apache Spark in the UK excluding London over the 6 months to 6 June 2020.

Daily rate histogram for Apache Spark in the UK excluding London

Apache Spark
Contractor Hourly Rate Trend in the UK excluding London

3-month moving average hourly rates quoted in jobs citing Apache Spark in the UK excluding London.

Hourly rate trend for Apache Spark in the UK excluding London

Apache Spark
Hourly Rate Histogram in the UK excluding London

Hourly rate distribution of jobs citing Apache Spark in the UK excluding London over the 6 months to 6 June 2020.

Hourly rate histogram for Apache Spark in the UK excluding London

Apache Spark
Contract Job Locations in the UK excluding London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the UK excluding London region over the 6 months to 6 June 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
South East +37 75 £490 -6.67% 15
Wales +55 36 £500 -4.76% 2
North of England -18 36 £550 +10.00% 16
East of England +37 25 £550 +4.76% 14
South West +43 24 £528 +0.48% 1
North West +11 19 £550 -2.22% 11
Scotland -9 18 £575 +21.05% 7
Yorkshire -7 14 £475 -5.00% 5
Midlands +56 8 £550 +10.00%
West Midlands +55 6 £550 +10.00%
East Midlands +40 2 - -
North East +29 2 £650 +13.04%
Apache Spark
UK

For the 6 months to 6 June 2020, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the UK excluding London region with a requirement for Apache Spark.

1 140 (63.64%) Python
2 134 (60.91%) Big Data
3 132 (60.00%) Hadoop
4 103 (46.82%) SQL
5 99 (45.00%) Apache Hive
6 88 (40.00%) AWS
6 88 (40.00%) Scala
7 67 (30.45%) Kafka
8 62 (28.18%) Agile Software Development
9 60 (27.27%) Data Modelling
9 60 (27.27%) ETL
10 57 (25.91%) Java
11 56 (25.45%) HDFS
12 52 (23.64%) Analytics
12 52 (23.64%) Git (software)
13 51 (23.18%) Impala
13 51 (23.18%) Data Engineering
14 49 (22.27%) DevOps
15 47 (21.36%) Data Science
16 46 (20.91%) Apache
17 44 (20.00%) Cloudera
17 44 (20.00%) Azure
18 38 (17.27%) NoSQL
18 38 (17.27%) HBase
19 36 (16.36%) Computer Science
19 36 (16.36%) Data Munging
19 36 (16.36%) Jenkins
20 35 (15.91%) Security Cleared
21 34 (15.45%) Linux
22 32 (14.55%) Visualisation

Apache Spark
Co-occurring IT Skills in the UK excluding London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 46 (20.91%) Apache
2 12 (5.45%) Jupyter
3 11 (5.00%) SAS
4 10 (4.55%) Elasticsearch
5 6 (2.73%) Tomcat
6 5 (2.27%) nginx
7 4 (1.82%) Cloud Foundry
7 4 (1.82%) GemFire
7 4 (1.82%) WebSphere
8 3 (1.36%) Confluence
9 2 (0.91%) IIS
9 2 (0.91%) Jetty
9 2 (0.91%) Luigi
9 2 (0.91%) Tornado
10 1 (0.45%) Apache Pig
Applications
1 3 (1.36%) MS Visio
2 1 (0.45%) Microsoft Excel
2 1 (0.45%) SPSS
Business Applications
1 4 (1.82%) SAP CRM
1 4 (1.82%) SAP ERP
1 4 (1.82%) SAP IS-U
2 1 (0.45%) Shopify
Cloud Services
1 88 (40.00%) AWS
2 44 (20.00%) Azure
3 31 (14.09%) Amazon S3
4 19 (8.64%) Serverless
5 18 (8.18%) Amazon Kinesis
5 18 (8.18%) AWS Lambda
6 17 (7.73%) Amazon EMR
7 13 (5.91%) GitHub
8 12 (5.45%) Power Platform
9 10 (4.55%) Cloud Computing
9 10 (4.55%) CloudFront
10 9 (4.09%) Amazon SQS
10 9 (4.09%) GCP
11 6 (2.73%) Amazon EC2
12 4 (1.82%) Amazon ECS
12 4 (1.82%) Amazon ELB
12 4 (1.82%) Azure Data Factory
12 4 (1.82%) OpenShift
13 3 (1.36%) Amazon SageMaker
13 3 (1.36%) Virtual Private Cloud
Communications & Networking
1 7 (3.18%) Kerberos
2 4 (1.82%) DSL
3 3 (1.36%) DNS
3 3 (1.36%) SAN
3 3 (1.36%) SSL
4 1 (0.45%) Firewall
4 1 (0.45%) MQTT
Database & Business Intelligence
1 134 (60.91%) Big Data
2 132 (60.00%) Hadoop
3 99 (45.00%) Apache Hive
4 51 (23.18%) Impala
5 38 (17.27%) HBase
5 38 (17.27%) NoSQL
6 26 (11.82%) Relational Database
7 25 (11.36%) SQL Server
8 22 (10.00%) Amazon Redshift
9 20 (9.09%) Data Lake
10 17 (7.73%) Apache Cassandra
11 16 (7.27%) Data Mining
12 13 (5.91%) MongoDB
13 12 (5.45%) Power BI
14 11 (5.00%) Amazon Athena
14 11 (5.00%) Spark SQL
14 11 (5.00%) Sqoop
15 10 (4.55%) Data Warehouse
16 9 (4.09%) PostgreSQL
17 8 (3.64%) SQL Server Integration Services
Development Applications
1 52 (23.64%) Git (software)
2 36 (16.36%) Jenkins
3 19 (8.64%) Sonatype Nexus
4 15 (6.82%) Maven
5 13 (5.91%) IBM UrbanCode
6 12 (5.45%) JIRA
7 11 (5.00%) Cucumber
7 11 (5.00%) Selenium
8 6 (2.73%) Gerrit
9 5 (2.27%) ERwin
10 4 (1.82%) Artifactory
10 4 (1.82%) Liquibase
10 4 (1.82%) Protractor
10 4 (1.82%) Redmine
10 4 (1.82%) TeamCity
11 2 (0.91%) Gatling
11 2 (0.91%) Subversion
11 2 (0.91%) Team Foundation Server
11 2 (0.91%) Travis CI
12 1 (0.45%) JBehave
General
1 31 (14.09%) Finance
2 22 (10.00%) Banking
3 9 (4.09%) Public Sector
4 7 (3.18%) Marketing
5 6 (2.73%) Investment Banking
6 5 (2.27%) Pharmaceutical
7 4 (1.82%) Legal
7 4 (1.82%) Retail
8 1 (0.45%) Aerospace
8 1 (0.45%) Automotive
8 1 (0.45%) Games
8 1 (0.45%) Manufacturing
8 1 (0.45%) Telecoms
Job Titles
1 81 (36.82%) Data Engineer
2 41 (18.64%) Big Data Engineer
3 27 (12.27%) Developer
4 26 (11.82%) Architect
5 22 (10.00%) DevOps Engineer
6 21 (9.55%) Data Scientist
7 20 (9.09%) AWS Engineer
8 18 (8.18%) Data Architect
9 12 (5.45%) Hadoop Engineer
9 12 (5.45%) Platform Engineer
9 12 (5.45%) Software Engineer
10 10 (4.55%) AWS DevOps
10 10 (4.55%) Python Engineer
11 9 (4.09%) AWS DevOps Engineer
11 9 (4.09%) Scala Developer
12 8 (3.64%) Apache Spark Developer
12 8 (3.64%) Big Data Software Engineer
12 8 (3.64%) Cloud Architect
13 7 (3.18%) Java Engineer
13 7 (3.18%) Scala Software Engineer
Libraries, Frameworks & Software Standards
1 67 (30.45%) Kafka
2 20 (9.09%) PySpark
3 12 (5.45%) Node.js
3 12 (5.45%) Pandas
4 11 (5.00%) AngularJS
5 9 (4.09%) NumPy
6 8 (3.64%) Apache NiFi
7 7 (3.18%) D3.js
7 7 (3.18%) React
7 7 (3.18%) Spring
8 6 (2.73%) pytest
8 6 (2.73%) Spring Boot
9 5 (2.27%) RESTful
10 4 (1.82%) .NET
10 4 (1.82%) ASP.NET
10 4 (1.82%) J2EE
10 4 (1.82%) JSON
10 4 (1.82%) LINQ
10 4 (1.82%) RabbitMQ
10 4 (1.82%) XML
Miscellaneous
1 32 (14.55%) Analytical Skills
2 13 (5.91%) Algorithms
3 7 (3.18%) Greenfield Project
3 7 (3.18%) Management Information System
4 5 (2.27%) Analytical Mindset
4 5 (2.27%) Reference Data
5 4 (1.82%) Public Cloud
6 3 (1.36%) Data Structures
6 3 (1.36%) Mainframe
7 2 (0.91%) Clustering
7 2 (0.91%) Data Centre
7 2 (0.91%) Self-Motivation
8 1 (0.45%) Linux Command Line
8 1 (0.45%) NHS
8 1 (0.45%) Video Conferencing
Operating Systems
1 34 (15.45%) Linux
2 12 (5.45%) Windows
3 7 (3.18%) Red Hat Enterprise Linux
4 6 (2.73%) CentOS
5 3 (1.36%) Unix
5 3 (1.36%) Windows Server
6 1 (0.45%) DC/OS
Processes & Methodologies
1 62 (28.18%) Agile Software Development
2 60 (27.27%) Data Modelling
2 60 (27.27%) ETL
3 52 (23.64%) Analytics
4 51 (23.18%) Data Engineering
5 49 (22.27%) DevOps
6 47 (21.36%) Data Science
7 36 (16.36%) Computer Science
7 36 (16.36%) Data Munging
8 32 (14.55%) Visualisation
9 31 (14.09%) Machine Learning
9 31 (14.09%) Software Engineering
10 30 (13.64%) Scrum
11 28 (12.73%) TDD
12 25 (11.36%) Business Intelligence
12 25 (11.36%) Continuous Integration
12 25 (11.36%) Data Ingestion
13 24 (10.91%) Data Visualisation
13 24 (10.91%) ETL Development
14 22 (10.00%) Statistics
Programming Languages
1 140 (63.64%) Python
2 103 (46.82%) SQL
3 88 (40.00%) Scala
4 57 (25.91%) Java
5 18 (8.18%) R
5 18 (8.18%) Ruby
6 14 (6.36%) JavaScript
7 9 (4.09%) C#
7 9 (4.09%) Go
8 8 (3.64%) Bash Shell
9 7 (3.18%) C
9 7 (3.18%) Shell Script
10 6 (2.73%) Groovy
11 4 (1.82%) Data Analysis Expressions (DAX)
11 4 (1.82%) T-SQL
12 3 (1.36%) TypeScript
13 2 (0.91%) Java 8
13 2 (0.91%) Korn
13 2 (0.91%) VBA
14 1 (0.45%) Haskell
Qualifications
1 35 (15.91%) Security Cleared
2 32 (14.55%) Degree
3 28 (12.73%) SC Cleared
4 19 (8.64%) Computer Science Degree
5 10 (4.55%) AWS Certification
6 6 (2.73%) Master's Degree
7 4 (1.82%) Numerate Degree
7 4 (1.82%) Postgraduate
8 2 (0.91%) DBS Check
9 1 (0.45%) BPSS Clearance
9 1 (0.45%) DV Cleared
9 1 (0.45%) PhD
Quality Assurance & Compliance
1 9 (4.09%) Data Quality
2 5 (2.27%) SLA
3 3 (1.36%) GxP
4 1 (0.45%) IFRS
4 1 (0.45%) PCI DSS
System Software
1 56 (25.45%) HDFS
2 13 (5.91%) Docker
3 5 (2.27%) Active Directory
3 5 (2.27%) Apache Flume
4 3 (1.36%) Virtual Desktop
4 3 (1.36%) Virtual Servers
5 2 (0.91%) Apache ZooKeeper
Systems Management
1 18 (8.18%) Terraform
2 16 (7.27%) Hadoop YARN
3 12 (5.45%) Ansible
4 11 (5.00%) Kubernetes
5 10 (4.55%) Oozie
6 7 (3.18%) Kibana
7 6 (2.73%) Opscode Chef
8 4 (1.82%) Puppet
9 2 (0.91%) Netcool
10 1 (0.45%) Apache Brooklyn
10 1 (0.45%) FortiGate
10 1 (0.45%) logstash
10 1 (0.45%) Marathon
10 1 (0.45%) Mesos
Vendors
1 44 (20.00%) Cloudera
2 23 (10.45%) Oracle
3 17 (7.73%) Talend
4 16 (7.27%) Informatica
4 16 (7.27%) SAP
5 14 (6.36%) Hortonworks
5 14 (6.36%) Microsoft
5 14 (6.36%) Tableau
6 10 (4.55%) IBM
7 9 (4.09%) Databricks
7 9 (4.09%) IBM Sterling
8 6 (2.73%) StreamSets
9 5 (2.27%) Google
10 3 (1.36%) Red Hat
11 2 (0.91%) Cognos
11 2 (0.91%) Kony
11 2 (0.91%) MapR
11 2 (0.91%) Micro Focus
11 2 (0.91%) Remedy
12 1 (0.45%) MuleSoft