Period
to 5 March 2021

The following table provides summary statistics for contract job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 5 March 2021 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
England > London
6 months to
5 Mar 2021
Same period 2020 Same period 2019
Rank 129 141 127
Rank change year-on-year +12 -14 +11
Contract jobs citing Apache Spark 424 540 936
As % of all contract jobs advertised in London 2.11% 1.92% 2.14%
As % of the Application Platforms category 14.46% 12.90% 12.90%
Number of daily rates quoted 353 427 703
Median daily rate £575 £588 £575
Median daily rate % change year-on-year -2.13% +2.17% +2.22%
10th Percentile £450 £420 £475
90th Percentile £713 £741 £725
England median daily rate £570 £557 £550
% change year-on-year +2.29% +1.32% -
Number of hourly rates quoted 0 0 3
Median hourly rate - - £45.00
England median hourly rate £23.77 - £45.00

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Contract vacancies with a requirement for application platform skills 2,933 4,187 7,257
As % of all contract IT jobs advertised in London 14.59% 14.92% 16.56%
Number of daily rates quoted 2,069 2,960 5,186
Median daily rate £500 £500 £525
Median daily rate % change year-on-year - -4.76% +5.00%
10th Percentile £313 £288 £306
90th Percentile £688 £675 £675
England median daily rate £475 £463 £475
% change year-on-year +2.70% -2.63% +5.56%
Number of hourly rates quoted 30 41 48
Median hourly rate £42.50 £25.00 £23.96
Median hourly rate % change year-on-year +70.00% +4.35% +21.30%
10th Percentile £19.80 £15.00 £16.25
90th Percentile £70.21 £50.00 £47.50
Median hourly rate £25.05 £19.09 £18.75
% change year-on-year +31.19% +1.81% +0.47%

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Contractor Daily Rate Trend in London

3-month moving average daily rate quoted in jobs citing Apache Spark in London.

Daily rate trend for Apache Spark in London

Apache Spark
Daily Rate Histogram in London

Daily rate distribution for jobs citing Apache Spark in London over the 6 months to 5 March 2021.

Daily rate histogram for Apache Spark in London

Apache Spark
Contractor Hourly Rate Trend in London

3-month moving average hourly rates quoted in jobs citing Apache Spark in London.

Hourly rate trend for Apache Spark in London

Apache Spark
Contract Job Locations in London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the London region over the 6 months to 5 March 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +26 46 £600 +4.35% 41
West London +40 19 £620 +12.73% 22
South London +40 5 £600 +20.00%
Apache Spark
England

For the 6 months to 5 March 2021, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the London region with a requirement for Apache Spark.

1 255 (60.14%) Python
2 218 (51.42%) Hadoop
3 207 (48.82%) Big Data
4 187 (44.10%) Java
5 167 (39.39%) Scala
6 152 (35.85%) SQL
7 147 (34.67%) Finance
8 146 (34.43%) AWS
9 131 (30.90%) Agile Software Development
10 121 (28.54%) Kafka
11 110 (25.94%) Apache Hive
12 109 (25.71%) Apache
13 98 (23.11%) Analytics
14 95 (22.41%) Banking
15 92 (21.70%) Machine Learning
16 91 (21.46%) Data Science
17 83 (19.58%) Front Office
18 82 (19.34%) Azure
19 81 (19.10%) ETL
20 72 (16.98%) DevOps
21 68 (16.04%) Software Engineering
22 67 (15.80%) NoSQL
23 66 (15.57%) Docker
24 65 (15.33%) HDFS
25 61 (14.39%) Terraform
25 61 (14.39%) Data Warehouse
26 60 (14.15%) TDD
26 60 (14.15%) Kubernetes
27 59 (13.92%) Data Engineering
28 57 (13.44%) Web Services

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 109 (25.71%) Apache
2 44 (10.38%) Apache Airflow
3 19 (4.48%) Confluence
4 18 (4.25%) Apache Pig
5 14 (3.30%) Elasticsearch
6 9 (2.12%) Jupyter
7 6 (1.42%) Oracle Coherence
8 5 (1.18%) Apache Solr
9 2 (0.47%) Luigi
10 1 (0.24%) Apache Synapse
10 1 (0.24%) SAS
Applications
1 2 (0.47%) Chatbot
1 2 (0.47%) Microsoft Excel
Business Applications
1 1 (0.24%) SAP CRM
1 1 (0.24%) SAP ERP
1 1 (0.24%) SAP IS-U
Cloud Services
1 146 (34.43%) AWS
2 82 (19.34%) Azure
3 47 (11.08%) GCP
4 43 (10.14%) Amazon S3
5 34 (8.02%) Amazon EMR
6 25 (5.90%) Serverless
7 18 (4.25%) Amazon EC2
7 18 (4.25%) Amazon Kinesis
8 16 (3.77%) Azure Data Factory
9 13 (3.07%) AWS CloudFormation
9 13 (3.07%) AWS Lambda
10 12 (2.83%) GitHub
11 10 (2.36%) Amazon SageMaker
12 9 (2.12%) Power Platform
13 8 (1.89%) Azure Functions
14 7 (1.65%) AWS CodePipeline
14 7 (1.65%) Cloud Computing
15 6 (1.42%) Amazon ECS
16 4 (0.94%) AWS CodeCommit
16 4 (0.94%) Logic Apps
Communications & Networking
1 17 (4.01%) HTTP
2 5 (1.18%) Network Security
3 3 (0.71%) DNS
3 3 (0.71%) Firewall
3 3 (0.71%) VPN
4 2 (0.47%) TCP/IP
5 1 (0.24%) Internet
5 1 (0.24%) Kerberos
5 1 (0.24%) Software-Defined Networking
Database & Business Intelligence
1 218 (51.42%) Hadoop
2 207 (48.82%) Big Data
3 110 (25.94%) Apache Hive
4 67 (15.80%) NoSQL
5 61 (14.39%) Data Warehouse
6 50 (11.79%) Data Lake
7 43 (10.14%) HBase
8 37 (8.73%) Impala
9 36 (8.49%) Apache Ignite
10 31 (7.31%) Relational Database
11 29 (6.84%) Amazon Redshift
12 27 (6.37%) Spark SQL
13 25 (5.90%) Apache Cassandra
14 23 (5.42%) Parquet
14 23 (5.42%) SQL Server
14 23 (5.42%) Tableau
15 20 (4.72%) MongoDB
16 19 (4.48%) Amazon Athena
16 19 (4.48%) Amazon RDS
17 18 (4.25%) BigQuery
Development Applications
1 47 (11.08%) Git (software)
2 45 (10.61%) Jenkins
3 22 (5.19%) JIRA
4 14 (3.30%) TeamCity
5 10 (2.36%) Selenium
6 9 (2.12%) Atlassian Bamboo
7 8 (1.89%) MLflow
8 7 (1.65%) Cucumber
9 6 (1.42%) Bitbucket
9 6 (1.42%) Sonatype Nexus
10 5 (1.18%) Maven
11 4 (0.94%) Artifactory
11 4 (0.94%) ProC
12 2 (0.47%) kappa
13 1 (0.24%) ER Studio
13 1 (0.24%) Gerrit
13 1 (0.24%) Gradle
13 1 (0.24%) IntelliJ
13 1 (0.24%) JUnit
13 1 (0.24%) Mockito
General
1 147 (34.67%) Finance
2 95 (22.41%) Banking
3 83 (19.58%) Front Office
4 49 (11.56%) Investment Banking
5 21 (4.95%) Marketing
6 15 (3.54%) Public Sector
7 14 (3.30%) Retail
8 5 (1.18%) Financial Institution
9 4 (0.94%) Games
9 4 (0.94%) Legal
9 4 (0.94%) Manufacturing
10 3 (0.71%) Automotive
10 3 (0.71%) French Language
10 3 (0.71%) Retail Banking
10 3 (0.71%) Telecoms
11 2 (0.47%) Electronics
11 2 (0.47%) Military
12 1 (0.24%) Hebrew Language
12 1 (0.24%) Local Government
12 1 (0.24%) Swedish Language
Job Titles
1 156 (36.79%) Data Engineer
2 155 (36.56%) Developer
3 59 (13.92%) Python Engineer
4 46 (10.85%) Software Engineer
5 45 (10.61%) Java Developer
6 44 (10.38%) Big Data Engineer
7 43 (10.14%) AWS Engineer
8 37 (8.73%) Developer/Software Engineer
8 37 (8.73%) Python Developer
9 36 (8.49%) Data Scientist
10 35 (8.25%) Scala Developer
11 30 (7.08%) Architect
11 30 (7.08%) Senior Developer
12 22 (5.19%) Apache Spark Developer
12 22 (5.19%) Scala Engineer
13 20 (4.72%) Senior Data Engineer
14 19 (4.48%) Software Developer
15 18 (4.25%) Big Data Developer
15 18 (4.25%) Python Software Engineer
16 16 (3.77%) Hadoop Engineer
Libraries, Frameworks & Software Standards
1 121 (28.54%) Kafka
2 57 (13.44%) Web Services
3 42 (9.91%) PySpark
4 40 (9.43%) Node.js
5 38 (8.96%) React
6 36 (8.49%) gRPC
6 36 (8.49%) Protocol Buffers
7 25 (5.90%) JSON
8 21 (4.95%) XML
9 15 (3.54%) NumPy
9 15 (3.54%) Pandas
10 14 (3.30%) Spark Streaming
11 13 (3.07%) scikit-learn
12 11 (2.59%) PyTorch
13 10 (2.36%) Akka
13 10 (2.36%) HTML
13 10 (2.36%) PostGIS
13 10 (2.36%) RESTful
13 10 (2.36%) TensorFlow
14 8 (1.89%) Spring
Miscellaneous
1 49 (11.56%) Algorithms
1 49 (11.56%) Analytical Skills
2 38 (8.96%) Equities
3 37 (8.73%) Fixed Income
4 36 (8.49%) Derivative
5 33 (7.78%) Data Structures
6 12 (2.83%) Self-Motivation
7 11 (2.59%) Foreign Exchange (FX)
8 10 (2.36%) Grid Computing
9 9 (2.12%) Cloud Native
9 9 (2.12%) Market Risk
10 8 (1.89%) Virtual Team
11 7 (1.65%) Public Cloud
11 7 (1.65%) User Experience
12 6 (1.42%) Greenfield Project
13 5 (1.18%) Analytical Mindset
13 5 (1.18%) Credit Risk
14 4 (0.94%) FMCG
15 2 (0.47%) CUDA
15 2 (0.47%) Hedge funds
Operating Systems
1 36 (8.49%) Linux
2 9 (2.12%) Unix
3 2 (0.47%) Ubuntu
4 1 (0.24%) Windows
Processes & Methodologies
1 131 (30.90%) Agile Software Development
2 98 (23.11%) Analytics
3 92 (21.70%) Machine Learning
4 91 (21.46%) Data Science
5 81 (19.10%) ETL
6 72 (16.98%) DevOps
7 68 (16.04%) Software Engineering
8 60 (14.15%) TDD
9 59 (13.92%) Data Engineering
10 57 (13.44%) Mathematics
11 53 (12.50%) Data Modelling
12 52 (12.26%) Computer Science
13 44 (10.38%) BDD
14 41 (9.67%) Scrum
15 38 (8.96%) Investment Management
16 37 (8.73%) Asset Management
16 37 (8.73%) Performance Attribution
17 36 (8.49%) Visualisation
18 35 (8.25%) MapReduce
19 34 (8.02%) Business Intelligence
Programming Languages
1 255 (60.14%) Python
2 187 (44.10%) Java
3 167 (39.39%) Scala
4 152 (35.85%) SQL
5 47 (11.08%) JavaScript
6 43 (10.14%) Kotlin
7 37 (8.73%) R
8 23 (5.42%) C#
8 23 (5.42%) C++
9 10 (2.36%) Haskell
9 10 (2.36%) Shell Script
10 7 (1.65%) Bash Shell
11 5 (1.18%) Groovy
11 5 (1.18%) PL/SQL
12 4 (0.94%) Java 8
13 3 (0.71%) Go
13 3 (0.71%) Perl
14 1 (0.24%) MATLAB
14 1 (0.24%) T-SQL
14 1 (0.24%) VBA
Qualifications
1 52 (12.26%) Degree
2 38 (8.96%) Security Cleared
3 34 (8.02%) SC Cleared
4 29 (6.84%) Numerate Degree
5 14 (3.30%) AWS Certification
6 12 (2.83%) TOGAF Certification
7 10 (2.36%) PhD
8 8 (1.89%) Computer Science Degree
9 3 (0.71%) Master's Degree
10 2 (0.47%) DV Cleared
Quality Assurance & Compliance
1 20 (4.72%) QA
2 19 (4.48%) Data Quality
3 3 (0.71%) MiFID
4 1 (0.24%) GDPR
4 1 (0.24%) ISO 20022
4 1 (0.24%) PCI DSS
4 1 (0.24%) SEPA
4 1 (0.24%) TARGET2
System Software
1 66 (15.57%) Docker
2 65 (15.33%) HDFS
3 6 (1.42%) Apache Flume
3 6 (1.42%) Apache ZooKeeper
4 3 (0.71%) Active Directory Lightweight Directory Services
5 1 (0.24%) Virtual Machines
Systems Management
1 61 (14.39%) Terraform
2 60 (14.15%) Kubernetes
3 19 (4.48%) Ansible
4 18 (4.25%) Hadoop YARN
5 10 (2.36%) Oozie
6 8 (1.89%) Opscode Chef
6 8 (1.89%) Puppet
7 5 (1.18%) Ambari
7 5 (1.18%) Grafana
8 2 (0.47%) Kibana
8 2 (0.47%) Mesos
8 2 (0.47%) Prometheus
9 1 (0.24%) Control-M
9 1 (0.24%) Kubeflow
Vendors
1 41 (9.67%) Oracle
2 37 (8.73%) Databricks
3 36 (8.49%) Bloomberg
4 33 (7.78%) Cloudera
5 31 (7.31%) Google
6 18 (4.25%) Informatica
7 17 (4.01%) Hortonworks
7 17 (4.01%) Microsoft
8 11 (2.59%) Ab Initio
8 11 (2.59%) Teradata
9 9 (2.12%) StreamSets
9 9 (2.12%) Talend
10 8 (1.89%) MapR
11 7 (1.65%) TIBCO
12 6 (1.42%) Adobe
12 6 (1.42%) Splunk
13 5 (1.18%) Atlassian
13 5 (1.18%) Sage
14 4 (0.94%) Alteryx
14 4 (0.94%) SAP