Period
to 24 May 2020

The following table provides summary statistics for contract job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 24 May 2020 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
England > London
6 months to
24 May 2020
Same period 2019 Same period 2018
Rank 132 99 136
Rank change year-on-year -33 +37 -8
Contract jobs citing Apache Spark 410 1,088 996
As % of all contract jobs advertised in London 2.06% 2.68% 2.18%
As % of the Application Platforms category 13.50% 16.83% 12.64%
Number of daily rates quoted 310 838 758
Median daily rate £550 £556 £550
Median daily rate % change year-on-year -1.12% +1.14% -2.22%
10th Percentile £413 £481 £427
90th Percentile £701 £713 £700
England median daily rate £550 £550 £550
Number of hourly rates quoted 1 1 1
Median hourly rate £52.50 £55.00 £82.50
Median hourly rate % change year-on-year -4.55% -33.33% -
10th Percentile £51.25 £52.50 £73.75
90th Percentile £53.75 £57.50 £91.25
England median hourly rate £35.00 £55.00 £72.50
% change year-on-year -36.36% -24.14% -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Contract vacancies with a requirement for application platform skills 3,038 6,464 7,878
As % of all contract IT jobs advertised in London 15.28% 15.95% 17.27%
Number of daily rates quoted 1,991 4,478 5,623
Median daily rate £500 £525 £500
Median daily rate % change year-on-year -4.76% +5.00% +5.26%
10th Percentile £313 £313 £300
90th Percentile £700 £675 £650
England median daily rate £475 £475 £460
% change year-on-year - +3.26% +6.98%
Number of hourly rates quoted 40 43 77
Median hourly rate £35.00 £34.51 £20.50
Median hourly rate % change year-on-year +1.42% +68.34% -6.82%
10th Percentile £15.00 £18.53 £13.25
90th Percentile £54.13 £56.00 £55.25
Median hourly rate £22.50 £18.50 £19.00
% change year-on-year +21.62% -2.63% -2.56%

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Contractor Daily Rate Trend in London

3-month moving average daily rate quoted in jobs citing Apache Spark in London.

Daily rate trend for Apache Spark in London

Apache Spark
Daily Rate Histogram in London

Daily rate distribution for jobs citing Apache Spark in London over the 6 months to 24 May 2020.

Daily rate histogram for Apache Spark in London

Apache Spark
Contractor Hourly Rate Trend in London

3-month moving average hourly rates quoted in jobs citing Apache Spark in London.

Hourly rate trend for Apache Spark in London

Apache Spark
Contract Job Locations in London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the London region over the 6 months to 24 May 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London -47 73 £572 +3.91% 15
West London +10 11 £525 - 6
East London +27 9 £525 - 10
South London +33 8 £500 -2.87% 3
Apache Spark
England

For the 6 months to 24 May 2020, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the London region with a requirement for Apache Spark.

1 268 (65.37%) Big Data
2 266 (64.88%) Python
3 211 (51.46%) Hadoop
4 175 (42.68%) SQL
5 168 (40.98%) Agile Software Development
6 162 (39.51%) Scala
7 161 (39.27%) Java
8 156 (38.05%) AWS
9 148 (36.10%) Kafka
10 123 (30.00%) Analytics
11 117 (28.54%) GCP
12 116 (28.29%) Machine Learning
13 101 (24.63%) Data Science
14 98 (23.90%) Kubernetes
15 95 (23.17%) NoSQL
16 90 (21.95%) Finance
16 90 (21.95%) PySpark
17 87 (21.22%) Azure
18 81 (19.76%) Apache Hive
19 80 (19.51%) Docker
20 70 (17.07%) ETL
21 62 (15.12%) Microservices
22 61 (14.88%) TensorFlow
23 57 (13.90%) Banking
24 53 (12.93%) Spring
24 53 (12.93%) Apache
25 49 (11.95%) Git (software)
25 49 (11.95%) Algorithms
25 49 (11.95%) DevOps
25 49 (11.95%) HBase

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 53 (12.93%) Apache
2 23 (5.61%) Apache Airflow
3 17 (4.15%) Elasticsearch
4 16 (3.90%) Apache Pig
5 8 (1.95%) Confluence
6 7 (1.71%) Jupyter
7 5 (1.22%) SAS
8 3 (0.73%) Apache Pulsar
8 3 (0.73%) DNN
8 3 (0.73%) Oracle Workflow
9 2 (0.49%) Luigi
9 2 (0.49%) nginx
9 2 (0.49%) Tomcat
9 2 (0.49%) Tornado
9 2 (0.49%) WebLogic
10 1 (0.24%) Apache Solr
10 1 (0.24%) SharePoint
Applications
1 1 (0.24%) Chatbot
1 1 (0.24%) MS Visio
Business Applications
1 1 (0.24%) FINCAD Analytics
1 1 (0.24%) SAP Oil and Gas
Cloud Services
1 156 (38.05%) AWS
2 117 (28.54%) GCP
3 87 (21.22%) Azure
4 22 (5.37%) Amazon Kinesis
5 21 (5.12%) Serverless
6 18 (4.39%) AWS Lambda
7 16 (3.90%) Amazon EMR
8 15 (3.66%) Amazon S3
9 12 (2.93%) GitHub
10 11 (2.68%) Amazon EC2
10 11 (2.68%) Azure Data Factory
10 11 (2.68%) Cloud Computing
11 10 (2.44%) AWS CloudFormation
12 7 (1.71%) Amazon Glacier
12 7 (1.71%) Power Platform
12 7 (1.71%) SaaS
13 5 (1.22%) Azure Machine Learning
14 4 (0.98%) Amazon SQS
14 4 (0.98%) IaaS
14 4 (0.98%) PaaS
Communications & Networking
1 27 (6.59%) SSL
2 26 (6.34%) HTTPS
3 12 (2.93%) HTTP
4 9 (2.20%) Kerberos
4 9 (2.20%) RTMP
5 6 (1.46%) VPN
6 5 (1.22%) Internet
7 3 (0.73%) DNS
7 3 (0.73%) Firewall
8 2 (0.49%) 5G
8 2 (0.49%) Skype
8 2 (0.49%) TCP/IP
Database & Business Intelligence
1 268 (65.37%) Big Data
2 211 (51.46%) Hadoop
3 95 (23.17%) NoSQL
4 81 (19.76%) Apache Hive
5 49 (11.95%) HBase
6 44 (10.73%) MongoDB
7 43 (10.49%) Data Lake
8 42 (10.24%) Data Warehouse
9 40 (9.76%) Apache Cassandra
10 37 (9.02%) Relational Database
11 29 (7.07%) Impala
12 28 (6.83%) SQL Server
13 27 (6.59%) Amazon Redshift
13 27 (6.59%) BigQuery
14 26 (6.34%) Amazon Athena
15 24 (5.85%) PostgreSQL
15 24 (5.85%) Spark SQL
16 20 (4.88%) Flink
17 16 (3.90%) Redis
17 16 (3.90%) Sqoop
Development Applications
1 49 (11.95%) Git (software)
2 33 (8.05%) Jenkins
3 24 (5.85%) JIRA
4 16 (3.90%) Maven
5 14 (3.41%) Visual Studio Team System
6 13 (3.17%) TeamCity
7 11 (2.68%) GitLab
7 11 (2.68%) Gradle
7 11 (2.68%) JUnit
8 9 (2.20%) Mockito
8 9 (2.20%) NUnit
9 8 (1.95%) Artifactory
9 8 (1.95%) CircleCI
10 6 (1.46%) Team Foundation Server
11 5 (1.22%) Travis CI
12 4 (0.98%) Bitbucket
12 4 (0.98%) Octopus Deploy
13 3 (0.73%) GoCD
13 3 (0.73%) RStudio
13 3 (0.73%) Sonatype Nexus
General
1 90 (21.95%) Finance
2 57 (13.90%) Banking
3 29 (7.07%) Retail
4 24 (5.85%) Investment Banking
5 21 (5.12%) Telecoms
6 17 (4.15%) Front Office
7 10 (2.44%) Legal
7 10 (2.44%) Marketing
8 9 (2.20%) Public Sector
9 6 (1.46%) Financial Institution
9 6 (1.46%) Pharmaceutical
10 5 (1.22%) Retail Banking
11 4 (0.98%) Games
11 4 (0.98%) Manufacturing
12 2 (0.49%) Electronics
13 1 (0.24%) Aerospace
13 1 (0.24%) Automotive
13 1 (0.24%) Law
13 1 (0.24%) Publishing
Job Titles
1 156 (38.05%) Data Engineer
2 113 (27.56%) Developer
3 67 (16.34%) Data Scientist
4 52 (12.68%) Java Developer
5 49 (11.95%) Big Data Engineer
6 35 (8.54%) Architect
7 32 (7.80%) Python Engineer
8 30 (7.32%) AWS Engineer
9 29 (7.07%) Senior Data Scientist
10 24 (5.85%) Data Architect
11 23 (5.61%) Azure Engineer
12 21 (5.12%) Big Data Developer
13 20 (4.88%) Python Developer
14 17 (4.15%) Scala Developer
15 15 (3.66%) DevOps Engineer
16 14 (3.41%) Senior Developer
17 13 (3.17%) Hadoop Engineer
18 12 (2.93%) Cloud Engineer
18 12 (2.93%) Hadoop Developer
19 11 (2.68%) AWS Developer
Libraries, Frameworks & Software Standards
1 148 (36.10%) Kafka
2 90 (21.95%) PySpark
3 61 (14.88%) TensorFlow
4 53 (12.93%) Spring
5 40 (9.76%) RESTful
6 32 (7.80%) Pandas
7 31 (7.56%) Spark Streaming
8 29 (7.07%) scikit-learn
9 28 (6.83%) MLlib
10 26 (6.34%) Spring Boot
11 23 (5.61%) Spring Security
12 22 (5.37%) JAAS
13 20 (4.88%) .NET
13 20 (4.88%) H2O
13 20 (4.88%) Natural Language Processing
14 19 (4.63%) SciPy
14 19 (4.63%) Web Services
15 18 (4.39%) Guice
15 18 (4.39%) Keras
15 18 (4.39%) React
Miscellaneous
1 49 (11.95%) Algorithms
2 38 (9.27%) Analytical Skills
3 16 (3.90%) Data Structures
4 15 (3.66%) Self-Motivation
5 14 (3.41%) Foreign Exchange (FX)
5 14 (3.41%) Greenfield Project
6 11 (2.68%) Credit Risk
6 11 (2.68%) Enterprise Software
6 11 (2.68%) Market Risk
7 10 (2.44%) Social Media
8 9 (2.20%) Virtual Team
9 8 (1.95%) Cloud Native
9 8 (1.95%) PMI
10 6 (1.46%) Data Centre
10 6 (1.46%) Life Science
10 6 (1.46%) Linux Command Line
10 6 (1.46%) Management Information System
11 4 (0.98%) Fixed Income
12 3 (0.73%) FMCG
12 3 (0.73%) Grid Computing
Operating Systems
1 41 (10.00%) Linux
2 14 (3.41%) Unix
3 4 (0.98%) Debian
4 2 (0.49%) Red Hat Enterprise Linux
4 2 (0.49%) Ubuntu
4 2 (0.49%) Windows
Processes & Methodologies
1 168 (40.98%) Agile Software Development
2 123 (30.00%) Analytics
3 116 (28.29%) Machine Learning
4 101 (24.63%) Data Science
5 70 (17.07%) ETL
6 62 (15.12%) Microservices
7 49 (11.95%) DevOps
8 48 (11.71%) Artificial Intelligence
8 48 (11.71%) Business Intelligence
8 48 (11.71%) Data Modelling
9 47 (11.46%) Computer Science
10 45 (10.98%) Visualisation
11 41 (10.00%) Continuous Integration
12 40 (9.76%) Mathematics
12 40 (9.76%) Use Case
13 39 (9.51%) Data Engineering
14 36 (8.78%) Software Engineering
15 35 (8.54%) E-Commerce
15 35 (8.54%) Statistics
16 34 (8.29%) TDD
Programming Languages
1 266 (64.88%) Python
2 175 (42.68%) SQL
3 162 (39.51%) Scala
4 161 (39.27%) Java
5 36 (8.78%) R
6 31 (7.56%) Java 8
7 26 (6.34%) JavaScript
8 25 (6.10%) C#
9 22 (5.37%) Kotlin
10 16 (3.90%) Bash Shell
10 16 (3.90%) Go
11 12 (2.93%) Ruby
12 10 (2.44%) T-SQL
13 7 (1.71%) C++
13 7 (1.71%) PHP
13 7 (1.71%) PL/SQL
13 7 (1.71%) PowerShell
14 5 (1.22%) C
15 4 (0.98%) Korn
15 4 (0.98%) TypeScript
Qualifications
1 27 (6.59%) Degree
2 12 (2.93%) PhD
3 10 (2.44%) Master's Degree
4 9 (2.20%) SC Cleared
4 9 (2.20%) Security Cleared
5 8 (1.95%) PMI Certification
6 6 (1.46%) Computer Science Degree
7 4 (0.98%) Microsoft Certification
8 1 (0.24%) DBS Check
8 1 (0.24%) MBA
8 1 (0.24%) Numerate Degree
Quality Assurance & Compliance
1 8 (1.95%) Data Quality
2 4 (0.98%) QA
3 1 (0.24%) ISO/IEC 27001
3 1 (0.24%) MiFID
3 1 (0.24%) WCAG
System Software
1 80 (19.51%) Docker
2 42 (10.24%) HDFS
3 8 (1.95%) Apache Flume
4 2 (0.49%) Active Directory
4 2 (0.49%) Apache ZooKeeper
Systems Management
1 98 (23.90%) Kubernetes
2 33 (8.05%) Terraform
3 27 (6.59%) Hadoop YARN
4 22 (5.37%) Ansible
5 20 (4.88%) Prometheus
5 20 (4.88%) Zipkin
6 14 (3.41%) Oozie
7 7 (1.71%) Ambari
8 6 (1.46%) Single Sign-On
9 5 (1.22%) Puppet
10 3 (0.73%) Grafana
11 1 (0.24%) Control-M
11 1 (0.24%) Docker Swarm
11 1 (0.24%) Kibana
11 1 (0.24%) Opscode Chef
11 1 (0.24%) Tivoli
Vendors
1 44 (10.73%) Cloudera
2 42 (10.24%) Google
3 29 (7.07%) Databricks
3 29 (7.07%) Tableau
4 26 (6.34%) Hortonworks
4 26 (6.34%) Oracle
5 21 (5.12%) MapR
6 17 (4.15%) Microsoft
7 11 (2.68%) Teradata
8 7 (1.71%) Confluent
8 7 (1.71%) SAP
8 7 (1.71%) Splunk
8 7 (1.71%) Talend
9 6 (1.46%) Ab Initio
9 6 (1.46%) Informatica
9 6 (1.46%) Palantir
9 6 (1.46%) StreamSets
9 6 (1.46%) Sybase
10 3 (0.73%) Birst
10 3 (0.73%) IBM