Period
to 13 May 2021

The following table provides summary statistics for contract job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 13 May 2021 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
England > London
6 months to
13 May 2021
Same period 2020 Same period 2019
Rank 123 139 101
Rank change year-on-year +16 -38 +37
Contract jobs citing Apache Spark 500 412 1,055
As % of all contract jobs advertised in London 2.11% 1.97% 2.61%
As % of the Application Platforms category 14.92% 13.10% 16.41%
Number of daily rates quoted 319 311 793
10th Percentile £450 £413 £488
25th Percentile £513 £475 £513
Median daily rate (50th Percentile) £575 £550 £560
Median % change year-on-year +4.55% -1.79% +1.82%
75th Percentile £650 £650 £650
90th Percentile £725 £700 £713
England median daily rate £563 £550 £550
% change year-on-year +2.27% - -
Number of hourly rates quoted 1 0 1
10th Percentile " " "
25th Percentile £36.25 " £52.50
Median hourly rate £37.50 - £55.00
75th Percentile £38.75 " £57.50
90th Percentile " " "
England median hourly rate £37.50 - £55.00

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Contract vacancies with a requirement for application platform skills 3,351 3,144 6,428
As % of all contract IT jobs advertised in London 14.16% 15.07% 15.90%
Number of daily rates quoted 2,399 2,173 4,504
10th Percentile £325 £313 £313
25th Percentile £420 £413 £413
Median daily rate (50th Percentile) £513 £500 £525
Median % change year-on-year +2.50% -4.76% +5.00%
75th Percentile £613 £600 £600
90th Percentile £688 £690 £675
England median daily rate £481 £475 £475
% change year-on-year +1.32% - +4.40%
Number of hourly rates quoted 26 27 37
10th Percentile £17.00 £15.00 £17.24
25th Percentile £21.25 £19.56 £19.00
Median hourly rate £40.25 £26.48 £24.11
Median % change year-on-year +52.00% +9.83% +20.55%
75th Percentile £57.50 £44.69 £38.75
90th Percentile £68.95 £61.20 £51.70
England median hourly rate £24.50 £19.95 £17.50
% change year-on-year +22.81% +14.00% -4.38%

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Contractor Daily Rate Trend in London

3-month moving average daily rate quoted in jobs citing Apache Spark in London.

Daily rate trend for Apache Spark in London

Apache Spark
Daily Rate Histogram in London

Daily rate distribution for jobs citing Apache Spark in London over the 6 months to 13 May 2021.

Daily rate histogram for Apache Spark in London

Apache Spark
Contractor Hourly Rate Trend in London

3-month moving average hourly rates quoted in jobs citing Apache Spark in London.

Hourly rate trend for Apache Spark in London

Apache Spark
Contract Job Locations in London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the London region over the 6 months to 13 May 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +12 61 £600 +9.09% 41
West London +17 10 £622 +18.43% 12
South London +33 4 £625 +25.00% 2
East London +6 1 - - 5
Apache Spark
England

For the 6 months to 13 May 2021, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the London region with a requirement for Apache Spark.

1 264 (52.80%) Python
2 254 (50.80%) Big Data
3 224 (44.80%) SQL
4 219 (43.80%) Hadoop
5 191 (38.20%) Scala
6 189 (37.80%) Java
7 175 (35.00%) Agile Software Development
7 175 (35.00%) Kafka
8 165 (33.00%) Analytics
9 154 (30.80%) Finance
10 153 (30.60%) Azure
11 145 (29.00%) Apache
12 133 (26.60%) AWS
13 131 (26.20%) Data Pipeline
14 124 (24.80%) Apache Hive
15 115 (23.00%) Data Science
16 103 (20.60%) Banking
17 102 (20.40%) DevOps
18 99 (19.80%) Social Skills
19 96 (19.20%) Machine Learning
20 95 (19.00%) ETL
21 91 (18.20%) Data Warehouse
22 88 (17.60%) Apache Airflow
23 87 (17.40%) GCP
24 86 (17.20%) NoSQL
25 82 (16.40%) Kubernetes
25 82 (16.40%) Data Engineering
26 79 (15.80%) Analytical Skills
27 78 (15.60%) Databricks
28 74 (14.80%) Docker

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 145 (29.00%) Apache
2 88 (17.60%) Apache Airflow
3 22 (4.40%) Confluence
4 21 (4.20%) Oracle Coherence
5 12 (2.40%) SharePoint
6 9 (1.80%) Apache Pig
7 8 (1.60%) Elasticsearch
7 8 (1.60%) Luigi
8 7 (1.40%) GemFire
8 7 (1.40%) SAS
9 5 (1.00%) Apache Solr
9 5 (1.00%) Oracle Workflow
10 4 (0.80%) Jupyter
11 3 (0.60%) JBoss
12 1 (0.20%) DataSynapse
12 1 (0.20%) Mule
Applications
1 1 (0.20%) Microsoft PowerPoint
1 1 (0.20%) SPSS
Business Applications
1 5 (1.00%) SAP Oil and Gas
2 1 (0.20%) SAP CRM
2 1 (0.20%) SAP ERP
2 1 (0.20%) SAP IS-U
2 1 (0.20%) Towers Watson Igloo
Cloud Services
1 153 (30.60%) Azure
2 133 (26.60%) AWS
3 87 (17.40%) GCP
4 49 (9.80%) Serverless
5 45 (9.00%) Amazon S3
6 35 (7.00%) Azure Functions
7 28 (5.60%) Amazon EMR
8 26 (5.20%) Amazon EC2
9 24 (4.80%) Azure Cosmos DB
10 23 (4.60%) GitHub
11 21 (4.20%) Azure Data Factory
12 20 (4.00%) Azure Synapse Analytics
13 19 (3.80%) Amazon Kinesis
13 19 (3.80%) Power Platform
14 14 (2.80%) AWS Lambda
15 13 (2.60%) AWS CloudFormation
16 12 (2.40%) Google Analytics
17 10 (2.00%) Azure Storage
18 8 (1.60%) Google Data Studio
19 7 (1.40%) Cloud Computing
Communications & Networking
1 15 (3.00%) HTTP
2 9 (1.80%) Firewall
3 6 (1.20%) Network Security
4 4 (0.80%) SSL
5 3 (0.60%) DNS
5 3 (0.60%) Kerberos
6 1 (0.20%) FTP
6 1 (0.20%) HTTPS
6 1 (0.20%) Internet
6 1 (0.20%) SIGTRAN
6 1 (0.20%) SMPP
6 1 (0.20%) SMS
6 1 (0.20%) Software-Defined Networking
6 1 (0.20%) SS7
6 1 (0.20%) Wireshark
Database & Business Intelligence
1 254 (50.80%) Big Data
2 219 (43.80%) Hadoop
3 124 (24.80%) Apache Hive
4 91 (18.20%) Data Warehouse
5 86 (17.20%) NoSQL
6 61 (12.20%) Data Lake
7 47 (9.40%) Relational Database
8 42 (8.40%) Amazon Redshift
8 42 (8.40%) HBase
9 39 (7.80%) MongoDB
10 38 (7.60%) BigQuery
11 33 (6.60%) Snowflake Schema
12 31 (6.20%) Impala
12 31 (6.20%) Parquet
13 29 (5.80%) Amazon RDS
13 29 (5.80%) PostgreSQL
14 28 (5.60%) SQL Server
14 28 (5.60%) Tableau
15 27 (5.40%) Spark SQL
16 26 (5.20%) Apache Cassandra
Development Applications
1 67 (13.40%) Jenkins
2 66 (13.20%) Git (software)
3 28 (5.60%) JIRA
4 16 (3.20%) TeamCity
5 13 (2.60%) Maven
6 10 (2.00%) Artifactory
6 10 (2.00%) Visual Studio Team System
7 9 (1.80%) PowerDesigner
8 8 (1.60%) Bitbucket
8 8 (1.60%) Oracle APEX
8 8 (1.60%) Sonatype Nexus
9 7 (1.40%) Hudson
9 7 (1.40%) MLflow
10 5 (1.00%) ProC
11 4 (0.80%) Atlassian Bamboo
12 3 (0.60%) ERwin
12 3 (0.60%) Gradle
12 3 (0.60%) Subversion
12 3 (0.60%) Visual Studio
13 2 (0.40%) Mockito
General
1 154 (30.80%) Finance
2 103 (20.60%) Banking
3 99 (19.80%) Social Skills
4 79 (15.80%) Analytical Skills
5 61 (12.20%) Front Office
6 47 (9.40%) Investment Banking
7 23 (4.60%) Marketing
8 19 (3.80%) Presentation Skills
9 13 (2.60%) Public Sector
10 11 (2.20%) Retail
10 11 (2.20%) Retail Banking
11 10 (2.00%) Financial Institution
12 9 (1.80%) Games
13 6 (1.20%) Back Office
14 5 (1.00%) French Language
15 4 (0.80%) Influencing Skills
15 4 (0.80%) Organisational Skills
16 3 (0.60%) Automotive
16 3 (0.60%) Local Government
17 2 (0.40%) Publishing
Job Titles
1 176 (35.20%) Data Engineer
2 157 (31.40%) Developer
3 53 (10.60%) Java Developer
4 51 (10.20%) Scala Developer
5 43 (8.60%) Architect
5 43 (8.60%) Data Scientist
6 40 (8.00%) Big Data Engineer
7 37 (7.40%) Senior Developer
8 34 (6.80%) AWS Engineer
9 30 (6.00%) Python Engineer
10 27 (5.40%) Data Architect
11 24 (4.80%) Apache Spark Developer
12 22 (4.40%) Python Developer
13 20 (4.00%) Scala Engineer
14 19 (3.80%) Senior Scala Developer
14 19 (3.80%) Software Engineer
15 18 (3.60%) Azure Engineer
15 18 (3.60%) Big Data Developer
16 17 (3.40%) Analyst
17 16 (3.20%) Senior Java Developer
Libraries, Frameworks & Software Standards
1 175 (35.00%) Kafka
2 52 (10.40%) PySpark
3 28 (5.60%) Spark Streaming
4 27 (5.40%) JSON
4 27 (5.40%) Spring
5 26 (5.20%) Web Services
6 20 (4.00%) Node.js
7 17 (3.40%) NumPy
8 16 (3.20%) Pandas
9 15 (3.00%) RESTful
9 15 (3.00%) Velocity
9 15 (3.00%) XML
10 13 (2.60%) .NET
11 12 (2.40%) Akka
11 12 (2.40%) React
11 12 (2.40%) scikit-learn
11 12 (2.40%) TensorFlow
12 11 (2.20%) Apache Avro
12 11 (2.20%) HTML
13 10 (2.00%) Protocol Buffers
Miscellaneous
1 52 (10.40%) Algorithms
2 38 (7.60%) Data Structures
3 24 (4.80%) Market Risk
4 22 (4.40%) Virtual Team
5 21 (4.20%) Grid Computing
6 18 (3.60%) Self-Motivation
7 16 (3.20%) Equities
8 14 (2.80%) Fixed Income
9 12 (2.40%) Derivative
9 12 (2.40%) Foreign Exchange (FX)
10 11 (2.20%) Greenfield Project
11 10 (2.00%) Management Information System
12 9 (1.80%) Replication
13 8 (1.60%) Hedge funds
14 6 (1.20%) Linux Command Line
15 5 (1.00%) Cloud Native
15 5 (1.00%) User Experience
16 2 (0.40%) CUDA
16 2 (0.40%) Data Centre
16 2 (0.40%) Public Cloud
Operating Systems
1 45 (9.00%) Linux
2 15 (3.00%) Unix
3 8 (1.60%) Windows
4 3 (0.60%) Red Hat Enterprise Linux
5 1 (0.20%) Android
Processes & Methodologies
1 175 (35.00%) Agile Software Development
2 165 (33.00%) Analytics
3 131 (26.20%) Data Pipeline
4 115 (23.00%) Data Science
5 102 (20.40%) DevOps
6 96 (19.20%) Machine Learning
7 95 (19.00%) ETL
8 82 (16.40%) Data Engineering
9 66 (13.20%) Mathematics
10 65 (13.00%) Data Modelling
11 63 (12.60%) Computer Science
11 63 (12.60%) TDD
12 58 (11.60%) Business Intelligence
13 57 (11.40%) Scrum
14 56 (11.20%) Software Engineering
15 47 (9.40%) Data Architecture
16 46 (9.20%) BDD
17 42 (8.40%) Data Management
18 41 (8.20%) Visualisation
19 40 (8.00%) Data Integration
Programming Languages
1 264 (52.80%) Python
2 224 (44.80%) SQL
3 191 (38.20%) Scala
4 189 (37.80%) Java
5 44 (8.80%) R
6 22 (4.40%) C#
6 22 (4.40%) JavaScript
7 21 (4.20%) C++
8 16 (3.20%) T-SQL
9 13 (2.60%) Go
9 13 (2.60%) PL/SQL
10 11 (2.20%) Java 8
11 10 (2.00%) Haskell
11 10 (2.00%) Kotlin
12 6 (1.20%) Groovy
13 5 (1.00%) Bash Shell
14 4 (0.80%) Ruby
14 4 (0.80%) Shell Script
15 2 (0.40%) C
16 1 (0.20%) Perl
Qualifications
1 57 (11.40%) Degree
2 27 (5.40%) Numerate Degree
3 18 (3.60%) SC Cleared
3 18 (3.60%) Security Cleared
4 12 (2.40%) Computer Science Degree
5 10 (2.00%) AWS Certification
5 10 (2.00%) ISEB
6 9 (1.80%) PhD
7 5 (1.00%) Master's Degree
8 3 (0.60%) DBS Check
Quality Assurance & Compliance
1 23 (4.60%) QA
2 22 (4.40%) Data Quality
3 16 (3.20%) GDPR
4 2 (0.40%) MiFID
4 2 (0.40%) SLA
5 1 (0.20%) PCI DSS
System Software
1 74 (14.80%) Docker
2 52 (10.40%) HDFS
3 5 (1.00%) Apache Flume
4 3 (0.60%) Apache ZooKeeper
5 1 (0.20%) Virtual Machines
Systems Management
1 82 (16.40%) Kubernetes
2 55 (11.00%) Terraform
3 28 (5.60%) Ansible
4 20 (4.00%) Hadoop YARN
5 17 (3.40%) Oozie
6 11 (2.20%) Puppet
7 7 (1.40%) Control-M
8 6 (1.20%) Opscode Chef
9 5 (1.00%) Ambari
9 5 (1.00%) Grafana
10 3 (0.60%) Prometheus
11 2 (0.40%) Istio
11 2 (0.40%) Kibana
12 1 (0.20%) Kubeflow
12 1 (0.20%) Rancher
12 1 (0.20%) Zipkin
Vendors
1 78 (15.60%) Databricks
2 62 (12.40%) Oracle
3 52 (10.40%) Google
4 42 (8.40%) Microsoft
5 31 (6.20%) Cloudera
6 15 (3.00%) Hortonworks
7 14 (2.80%) BiZZdesign
8 13 (2.60%) Informatica
9 12 (2.40%) Alteryx
9 12 (2.40%) SAP
9 12 (2.40%) Teradata
10 11 (2.20%) MapR
11 10 (2.00%) StreamSets
11 10 (2.00%) Talend
12 9 (1.80%) Adobe
12 9 (1.80%) Bloomberg
13 8 (1.60%) Confluent
14 7 (1.40%) Ab Initio
15 5 (1.00%) Red Hat
16 4 (0.80%) Salesforce.com