Period
to 1 October 2020

The following table provides summary statistics for contract job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 1 October 2020 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
England > London
6 months to
1 Oct 2020
Same period 2019 Same period 2018
Rank 128 97 138
Rank change year-on-year -31 +41 -10
Contract jobs citing Apache Spark 287 996 963
As % of all contract jobs advertised in London 2.12% 2.60% 2.11%
As % of the Application Platforms category 14.42% 16.79% 12.22%
Number of daily rates quoted 187 804 724
Median daily rate £550 £575 £575
Median daily rate % change year-on-year -4.35% - +4.55%
10th Percentile £413 £463 £450
90th Percentile £700 £725 £713
England median daily rate £550 £550 £550
Number of hourly rates quoted 0 0 3
Median hourly rate - - £45.00
England median hourly rate £35.00 £24.65 £33.75
% change year-on-year +41.99% -26.96% +74.51%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Contract vacancies with a requirement for application platform skills 1,990 5,933 7,879
As % of all contract IT jobs advertised in London 14.72% 15.50% 17.29%
Number of daily rates quoted 1,272 4,146 5,580
Median daily rate £500 £500 £500
10th Percentile £313 £300 £300
90th Percentile £713 £675 £663
England median daily rate £475 £463 £463
% change year-on-year +2.70% - +2.78%
Number of hourly rates quoted 21 50 103
Median hourly rate £35.00 £22.75 £21.50
Median hourly rate % change year-on-year +53.85% +5.81% +5.04%
10th Percentile £15.00 £15.59 £13.00
90th Percentile £53.75 £50.25 £69.50
Median hourly rate £27.23 £20.51 £21.50
% change year-on-year +32.76% -4.60% +2.60%

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Contractor Daily Rate Trend in London

3-month moving average daily rate quoted in jobs citing Apache Spark in London.

Daily rate trend for Apache Spark in London

Apache Spark
Daily Rate Histogram in London

Daily rate distribution for jobs citing Apache Spark in London over the 6 months to 1 October 2020.

Daily rate histogram for Apache Spark in London

Apache Spark
Contractor Hourly Rate Trend in London

3-month moving average hourly rates quoted in jobs citing Apache Spark in London.

Hourly rate trend for Apache Spark in London

Apache Spark
Contract Job Locations in London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the London region over the 6 months to 1 October 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London -2 49 £538 -5.70% 22
West London +38 16 £630 +20.00% 9
South London +36 2 £535 +1.90% 3
Apache Spark
England

For the 6 months to 1 October 2020, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the London region with a requirement for Apache Spark.

1 167 (58.19%) Big Data
2 164 (57.14%) Hadoop
3 156 (54.36%) Python
4 119 (41.46%) SQL
4 119 (41.46%) Kafka
5 113 (39.37%) Java
6 109 (37.98%) Scala
7 105 (36.59%) AWS
8 104 (36.24%) Agile Software Development
9 81 (28.22%) Azure
10 70 (24.39%) Finance
11 66 (23.00%) Apache Hive
12 62 (21.60%) Analytics
12 62 (21.60%) Kubernetes
13 61 (21.25%) NoSQL
13 61 (21.25%) GCP
14 58 (20.21%) Git (software)
15 57 (19.86%) Banking
16 56 (19.51%) Microservices
16 56 (19.51%) PySpark
17 52 (18.12%) Docker
18 51 (17.77%) Machine Learning
18 51 (17.77%) Data Science
19 46 (16.03%) Computer Science
20 45 (15.68%) DevOps
21 41 (14.29%) Data Modelling
22 39 (13.59%) ETL
23 38 (13.24%) Mathematics
23 38 (13.24%) Jenkins
24 37 (12.89%) Spring

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 35 (12.20%) Apache
2 26 (9.06%) Apache Pig
2 26 (9.06%) Elasticsearch
3 13 (4.53%) Apache Airflow
4 7 (2.44%) SAS
5 6 (2.09%) Confluence
6 5 (1.74%) Jupyter
6 5 (1.74%) Tornado
7 2 (0.70%) Luigi
8 1 (0.35%) Camunda
8 1 (0.35%) Oracle Workflow
8 1 (0.35%) R Markdown
Applications
1 1 (0.35%) Microsoft Excel
1 1 (0.35%) Microsoft PowerPoint
Business Applications
1 4 (1.39%) SAP Oil and Gas
2 1 (0.35%) FINCAD Analytics
2 1 (0.35%) Oracle BPM
2 1 (0.35%) Sparx Enterprise Architect
Cloud Services
1 105 (36.59%) AWS
2 81 (28.22%) Azure
3 61 (21.25%) GCP
4 19 (6.62%) Serverless
5 18 (6.27%) Azure Data Factory
6 16 (5.57%) Amazon EC2
7 12 (4.18%) Cloud Computing
8 10 (3.48%) Amazon S3
8 10 (3.48%) Virtual Private Cloud
9 8 (2.79%) Amazon EKS
9 8 (2.79%) Amazon EMR
9 8 (2.79%) AWS Lambda
10 6 (2.09%) Azure Functions
11 5 (1.74%) Azure Active Directory
11 5 (1.74%) Power Platform
12 4 (1.39%) AWS CloudFormation
12 4 (1.39%) SaaS
13 3 (1.05%) Amazon Kinesis
13 3 (1.05%) Azure Machine Learning
13 3 (1.05%) PaaS
Communications & Networking
1 28 (9.76%) SSL
2 23 (8.01%) HTTPS
3 6 (2.09%) HTTP
4 2 (0.70%) Kerberos
4 2 (0.70%) RTMP
4 2 (0.70%) Skype
4 2 (0.70%) VPN
5 1 (0.35%) Wi-Fi
5 1 (0.35%) Wireless
Database & Business Intelligence
1 167 (58.19%) Big Data
2 164 (57.14%) Hadoop
3 66 (23.00%) Apache Hive
4 61 (21.25%) NoSQL
5 27 (9.41%) Impala
6 21 (7.32%) HBase
7 20 (6.97%) Data Warehouse
7 20 (6.97%) MongoDB
8 19 (6.62%) Data Lake
9 18 (6.27%) Apache Cassandra
10 17 (5.92%) Amazon RDS
11 15 (5.23%) PostgreSQL
11 15 (5.23%) Spark SQL
12 14 (4.88%) BigQuery
13 12 (4.18%) Redis
13 12 (4.18%) Relational Database
14 11 (3.83%) Data Mining
15 10 (3.48%) Amazon Redshift
15 10 (3.48%) Presto
15 10 (3.48%) SQL Server
Development Applications
1 58 (20.21%) Git (software)
2 38 (13.24%) Jenkins
3 16 (5.57%) TeamCity
4 13 (4.53%) GitLab
5 8 (2.79%) JIRA
6 7 (2.44%) Artifactory
6 7 (2.44%) Cucumber
6 7 (2.44%) Gradle
6 7 (2.44%) Maven
6 7 (2.44%) Selenium
7 6 (2.09%) ProC
8 5 (1.74%) Hudson
9 3 (1.05%) SonarQube
10 2 (0.70%) JUnit
10 2 (0.70%) kappa
10 2 (0.70%) Mockito
10 2 (0.70%) NUnit
10 2 (0.70%) Visual Studio Team System
11 1 (0.35%) ERwin
11 1 (0.35%) MLflow
General
1 70 (24.39%) Finance
2 57 (19.86%) Banking
3 26 (9.06%) Front Office
4 20 (6.97%) Investment Banking
5 13 (4.53%) Marketing
5 13 (4.53%) Retail
6 9 (3.14%) Public Sector
6 9 (3.14%) Telecoms
7 6 (2.09%) Financial Institution
7 6 (2.09%) Legal
8 5 (1.74%) Electronics
8 5 (1.74%) Manufacturing
9 3 (1.05%) Games
9 3 (1.05%) Law
9 3 (1.05%) Pharmaceutical
10 2 (0.70%) Military
11 1 (0.35%) Aerospace
11 1 (0.35%) Automotive
11 1 (0.35%) Retail Banking
Job Titles
1 102 (35.54%) Developer
2 101 (35.19%) Data Engineer
3 43 (14.98%) Java Developer
4 40 (13.94%) Big Data Engineer
5 35 (12.20%) Python Engineer
6 30 (10.45%) Architect
7 21 (7.32%) Scala Developer
8 19 (6.62%) Data Scientist
8 19 (6.62%) Python Developer
8 19 (6.62%) Senior Data Engineer
8 19 (6.62%) Senior Developer
9 17 (5.92%) Azure Engineer
10 14 (4.88%) Big Data Developer
11 13 (4.53%) Data Architect
11 13 (4.53%) Solutions Architect
12 9 (3.14%) Apache Spark Developer
12 9 (3.14%) Consultant
12 9 (3.14%) Senior Java Developer
13 8 (2.79%) Agile Developer
13 8 (2.79%) Senior Data Scientist
Libraries, Frameworks & Software Standards
1 119 (41.46%) Kafka
2 56 (19.51%) PySpark
3 37 (12.89%) Spring
4 26 (9.06%) RESTful
4 26 (9.06%) Spring Security
5 23 (8.01%) JAAS
6 19 (6.62%) Guice
7 17 (5.92%) Akka
7 17 (5.92%) Spark Streaming
7 17 (5.92%) TensorFlow
8 13 (4.53%) Elastic Stack
9 12 (4.18%) .NET
10 11 (3.83%) scikit-learn
11 9 (3.14%) Anaconda
11 9 (3.14%) ASP.NET Web API
11 9 (3.14%) GraphQL
11 9 (3.14%) HTML
11 9 (3.14%) JSON
12 8 (2.79%) H2O
12 8 (2.79%) HTML5
Miscellaneous
1 29 (10.10%) Analytical Skills
2 23 (8.01%) Algorithms
3 14 (4.88%) Foreign Exchange (FX)
3 14 (4.88%) Virtual Team
4 12 (4.18%) FMCG
5 10 (3.48%) Cloud Native
5 10 (3.48%) Greenfield Project
6 9 (3.14%) Credit Risk
6 9 (3.14%) Data Structures
7 8 (2.79%) Analytical Mindset
8 7 (2.44%) Enterprise Software
8 7 (2.44%) Public Cloud
9 6 (2.09%) PMI
10 4 (1.39%) Fixed Income
10 4 (1.39%) Grid Computing
10 4 (1.39%) Management Information System
10 4 (1.39%) Market Risk
10 4 (1.39%) User Experience
11 3 (1.05%) Self-Motivation
11 3 (1.05%) Taxonomies
Operating Systems
1 21 (7.32%) Linux
2 9 (3.14%) Unix
3 4 (1.39%) Debian
3 4 (1.39%) Red Hat Enterprise Linux
3 4 (1.39%) Ubuntu
3 4 (1.39%) Windows
Processes & Methodologies
1 104 (36.24%) Agile Software Development
2 62 (21.60%) Analytics
3 56 (19.51%) Microservices
4 51 (17.77%) Data Science
4 51 (17.77%) Machine Learning
5 46 (16.03%) Computer Science
6 45 (15.68%) DevOps
7 41 (14.29%) Data Modelling
8 39 (13.59%) ETL
9 38 (13.24%) Mathematics
10 35 (12.20%) SDLC
10 35 (12.20%) Visualisation
11 34 (11.85%) Continuous Integration
12 31 (10.80%) Software Engineering
13 28 (9.76%) Business Intelligence
13 28 (9.76%) Scrum
14 27 (9.41%) Data Analytics
15 25 (8.71%) TDD
15 25 (8.71%) Test Automation
16 22 (7.67%) Statistics
Programming Languages
1 156 (54.36%) Python
2 119 (41.46%) SQL
3 113 (39.37%) Java
4 109 (37.98%) Scala
5 31 (10.80%) Kotlin
6 28 (9.76%) Java 8
7 21 (7.32%) JavaScript
8 20 (6.97%) C#
9 18 (6.27%) R
10 12 (4.18%) Haskell
11 11 (3.83%) Shell Script
12 9 (3.14%) Perl
12 9 (3.14%) Ruby
13 6 (2.09%) Groovy
14 5 (1.74%) C++
14 5 (1.74%) PHP
15 4 (1.39%) Go
15 4 (1.39%) PowerShell
15 4 (1.39%) T-SQL
16 2 (0.70%) Bash Shell
Qualifications
1 36 (12.54%) Security Cleared
2 33 (11.50%) SC Cleared
3 32 (11.15%) Degree
4 16 (5.57%) Numerate Degree
5 12 (4.18%) TOGAF Certification
6 10 (3.48%) Computer Science Degree
7 9 (3.14%) Master's Degree
8 6 (2.09%) PMI Certification
9 3 (1.05%) Microsoft Certification
10 2 (0.70%) AWS Certification
10 2 (0.70%) PhD
Quality Assurance & Compliance
1 7 (2.44%) MiFID
2 2 (0.70%) Data Quality
3 1 (0.35%) CDISC
3 1 (0.35%) ISO 20022
3 1 (0.35%) SEPA
3 1 (0.35%) SLA
3 1 (0.35%) TARGET2
System Software
1 52 (18.12%) Docker
2 23 (8.01%) HDFS
3 6 (2.09%) Apache ZooKeeper
4 5 (1.74%) Active Directory
5 3 (1.05%) Apache Flume
6 2 (0.70%) Active Directory Lightweight Directory Services
Systems Management
1 62 (21.60%) Kubernetes
2 21 (7.32%) Prometheus
2 21 (7.32%) Zipkin
3 16 (5.57%) Terraform
4 15 (5.23%) Ansible
5 5 (1.74%) Oozie
6 4 (1.39%) Hadoop YARN
6 4 (1.39%) Puppet
7 3 (1.05%) Opscode Chef
8 1 (0.35%) Grafana
8 1 (0.35%) Knative
8 1 (0.35%) vRealize Orchestrator
Vendors
1 31 (10.80%) Cloudera
2 28 (9.76%) Databricks
3 26 (9.06%) Oracle
4 24 (8.36%) Tableau
5 19 (6.62%) Google
6 16 (5.57%) Hortonworks
7 13 (4.53%) Informatica
7 13 (4.53%) Teradata
8 11 (3.83%) Microsoft
9 8 (2.79%) Ab Initio
9 8 (2.79%) SAP
10 7 (2.44%) StreamSets
10 7 (2.44%) TIBCO
11 5 (1.74%) MapR
11 5 (1.74%) MuleSoft
11 5 (1.74%) Salesforce.com
12 4 (1.39%) IBM
12 4 (1.39%) Red Hat
12 4 (1.39%) Talend
13 3 (1.05%) Splunk