Period
to 6 April 2020

The following table provides summary statistics for contract job vacancies advertised in Central London with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 6 April 2020 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
London > Central London
6 months to
6 Apr 2020
Same period 2019 Same period 2018
Rank 110 70 119
Rank change year-on-year -40 +49 -21
Contract jobs citing Apache Spark 95 324 205
As % of all contract jobs advertised in Central London 1.81% 3.60% 2.22%
As % of the Application Platforms category 12.04% 20.30% 13.29%
Number of daily rates quoted 75 274 178
Median daily rate £575 £550 £550
Median daily rate % change year-on-year +4.55% - -8.33%
10th Percentile £413 £513 £463
90th Percentile £708 £700 £691
London median daily rate £575 £575 £550
% change year-on-year - +4.55% -2.22%
Number of hourly rates quoted 0 0 0
Median hourly rate - - -
London median hourly rate £52.50 £115.75 £82.50
% change year-on-year -54.64% +40.30% -17.50%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in Central London with a requirement for application platform skills.

All Application Platform Skills
Central London
Contract vacancies with a requirement for application platform skills 789 1,596 1,542
As % of all contract IT jobs advertised in Central London 15.05% 17.75% 16.71%
Number of daily rates quoted 564 1,174 1,200
Median daily rate £500 £525 £500
Median daily rate % change year-on-year -4.76% +5.00% +5.26%
10th Percentile £263 £300 £288
90th Percentile £675 £650 £650
London median daily rate £500 £525 £500
% change year-on-year -4.76% +5.00% +7.53%
Number of hourly rates quoted 10 17 16
Median hourly rate £35.00 £19.00 £20.00
Median hourly rate % change year-on-year +84.21% -5.00% +5.26%
10th Percentile £24.50 £16.25 £12.50
90th Percentile £37.25 £34.25 £43.75
Median hourly rate £30.05 £28.50 £21.75
% change year-on-year +5.44% +31.03% +14.47%

Apache Spark
Job Vacancy Trend in Central London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in Central London.

Job vacancy trend for Apache Spark in Central London

Apache Spark
Contractor Daily Rate Trend in Central London

3-month moving average daily rate quoted in jobs citing Apache Spark in Central London.

Daily rate trend for Apache Spark in Central London

Apache Spark
Daily Rate Histogram in Central London

Daily rate distribution for jobs citing Apache Spark in Central London over the 6 months to 6 April 2020.

Daily rate histogram for Apache Spark in Central London

Apache Spark
Contract Job Locations in Central London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the Central London region over the 6 months to 6 April 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
City of London -55 71 £559 +1.64% 47
City of Westminster +22 6 £725 +20.83%
Southwark -7 3 £500 -4.76% 1
West End of London +27 1 - -
Apache Spark
London

For the 6 months to 6 April 2020, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the Central London region with a requirement for Apache Spark.

1 56 (58.95%) Python
2 52 (54.74%) Big Data
3 46 (48.42%) Kafka
4 41 (43.16%) Java
5 39 (41.05%) AWS
6 38 (40.00%) Hadoop
7 36 (37.89%) Agile Software Development
8 31 (32.63%) Scala
9 30 (31.58%) GCP
10 29 (30.53%) SQL
11 24 (25.26%) Analytics
11 24 (25.26%) Kubernetes
12 22 (23.16%) Finance
13 21 (22.11%) Machine Learning
13 21 (22.11%) Data Science
14 19 (20.00%) Microservices
14 19 (20.00%) Docker
14 19 (20.00%) Google
15 17 (17.89%) Azure
16 16 (16.84%) TDD
16 16 (16.84%) Spring
16 16 (16.84%) Computer Science
16 16 (16.84%) Apache
16 16 (16.84%) Spark Streaming
17 15 (15.79%) Use Case
17 15 (15.79%) Apache Cassandra
17 15 (15.79%) Apache Hive
18 14 (14.74%) NoSQL
18 14 (14.74%) Linux
18 14 (14.74%) Apache Airflow

Apache Spark
Co-occurring IT Skills in Central London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 16 (16.84%) Apache
2 14 (14.74%) Apache Airflow
3 5 (5.26%) Elasticsearch
4 3 (3.16%) Luigi
5 2 (2.11%) Apache Pig
5 2 (2.11%) DNN
6 1 (1.05%) Confluence
6 1 (1.05%) Oracle Workflow
6 1 (1.05%) Pachyderm
6 1 (1.05%) Tornado
Business Applications
1 2 (2.11%) SAP Oil and Gas
2 1 (1.05%) FINCAD Analytics
Cloud Services
1 39 (41.05%) AWS
2 30 (31.58%) GCP
3 17 (17.89%) Azure
4 7 (7.37%) Amazon S3
4 7 (7.37%) SaaS
4 7 (7.37%) Serverless
5 5 (5.26%) Amazon EMR
5 5 (5.26%) Amazon Kinesis
5 5 (5.26%) AWS Lambda
6 3 (3.16%) Amazon EC2
6 3 (3.16%) AWS IoT
7 2 (2.11%) Amazon CloudWatch
7 2 (2.11%) AWS CloudFormation
7 2 (2.11%) Data Lake Analytics
8 1 (1.05%) Amazon SageMaker
8 1 (1.05%) AWS Step Functions
8 1 (1.05%) Azure Active Directory
8 1 (1.05%) Azure Data Factory
8 1 (1.05%) Azure Functions
8 1 (1.05%) Virtual Private Cloud
Communications & Networking
1 2 (2.11%) TCP/IP
2 1 (1.05%) HTTPS
2 1 (1.05%) Internet
2 1 (1.05%) SSL
Database & Business Intelligence
1 52 (54.74%) Big Data
2 38 (40.00%) Hadoop
3 15 (15.79%) Apache Cassandra
3 15 (15.79%) Apache Hive
4 14 (14.74%) NoSQL
5 12 (12.63%) HBase
6 9 (9.47%) Data Warehouse
7 8 (8.42%) MongoDB
8 7 (7.37%) Amazon RDS
8 7 (7.37%) Amazon Redshift
8 7 (7.37%) BigQuery
8 7 (7.37%) PostgreSQL
9 6 (6.32%) Amazon Athena
9 6 (6.32%) Flink
9 6 (6.32%) Relational Database
10 5 (5.26%) Amazon Aurora
10 5 (5.26%) Bigtable
10 5 (5.26%) Cloud Spanner
10 5 (5.26%) Impala
11 4 (4.21%) Data Lake
Development Applications
1 12 (12.63%) Jenkins
2 6 (6.32%) Git (software)
2 6 (6.32%) Maven
3 5 (5.26%) JIRA
4 4 (4.21%) Travis CI
5 3 (3.16%) TeamCity
6 2 (2.11%) Atlassian Bamboo
7 1 (1.05%) ERwin
7 1 (1.05%) GitLab
7 1 (1.05%) Hudson
7 1 (1.05%) IntelliJ
7 1 (1.05%) ProC
General
1 22 (23.16%) Finance
2 13 (13.68%) Banking
3 6 (6.32%) Front Office
4 5 (5.26%) Investment Banking
5 4 (4.21%) Retail
5 4 (4.21%) Retail Banking
6 3 (3.16%) Telecoms
7 2 (2.11%) Marketing
Job Titles
1 35 (36.84%) Data Engineer
2 34 (35.79%) Developer
3 17 (17.89%) Java Developer
4 12 (12.63%) Big Data Engineer
4 12 (12.63%) Data Scientist
5 8 (8.42%) Scala Developer
6 7 (7.37%) Architect
7 6 (6.32%) AWS Developer
8 5 (5.26%) Senior Developer
9 4 (4.21%) AWS Engineer
9 4 (4.21%) Hadoop Developer
9 4 (4.21%) Java Hadoop Developer
9 4 (4.21%) Machine Learning Scientist
9 4 (4.21%) Python Developer
10 3 (3.16%) AWS DevOps
10 3 (3.16%) AWS DevOps Engineer
10 3 (3.16%) Backend Developer
10 3 (3.16%) E-Commerce Developer
10 3 (3.16%) Lead Architect
10 3 (3.16%) Python Engineer
Libraries, Frameworks & Software Standards
1 46 (48.42%) Kafka
2 16 (16.84%) Spark Streaming
2 16 (16.84%) Spring
3 12 (12.63%) PySpark
4 11 (11.58%) Pandas
5 10 (10.53%) Web Services
6 9 (9.47%) Akka
6 9 (9.47%) Spring Boot
6 9 (9.47%) TensorFlow
7 8 (8.42%) Apache Beam
7 8 (8.42%) RESTful
7 8 (8.42%) SciPy
8 7 (7.37%) AngularJS
8 7 (7.37%) MLlib
8 7 (7.37%) React
8 7 (7.37%) REST
9 5 (5.26%) Node.js
10 4 (4.21%) .NET
10 4 (4.21%) Backbone.js
10 4 (4.21%) HTML
Miscellaneous
1 12 (12.63%) Algorithms
2 11 (11.58%) Analytical Skills
3 7 (7.37%) Data Structures
4 6 (6.32%) Greenfield Project
4 6 (6.32%) Hedge funds
5 5 (5.26%) Market Risk
6 4 (4.21%) Cloud Native
6 4 (4.21%) Credit Risk
6 4 (4.21%) Linux Command Line
7 3 (3.16%) Fixed Income
7 3 (3.16%) Tandem
8 2 (2.11%) Equities
8 2 (2.11%) Foreign Exchange (FX)
8 2 (2.11%) Life Science
8 2 (2.11%) Management Information System
8 2 (2.11%) Self-Motivation
8 2 (2.11%) Semantic Web
9 1 (1.05%) Derivative
9 1 (1.05%) Internet of Things
Operating Systems
1 14 (14.74%) Linux
2 4 (4.21%) Unix
2 4 (4.21%) Windows
3 3 (3.16%) Debian
4 1 (1.05%) Red Hat Enterprise Linux
4 1 (1.05%) Ubuntu
Processes & Methodologies
1 36 (37.89%) Agile Software Development
2 24 (25.26%) Analytics
3 21 (22.11%) Data Science
3 21 (22.11%) Machine Learning
4 19 (20.00%) Microservices
5 16 (16.84%) Computer Science
5 16 (16.84%) TDD
6 15 (15.79%) Use Case
7 13 (13.68%) Artificial Intelligence
8 12 (12.63%) Open Source
9 11 (11.58%) DevOps
9 11 (11.58%) Mathematics
10 10 (10.53%) ETL
11 9 (9.47%) BDD
11 9 (9.47%) Data Transformation
11 9 (9.47%) E-Commerce
12 8 (8.42%) OOP
12 8 (8.42%) Physics
13 7 (7.37%) Actionable Insight
13 7 (7.37%) OO
Programming Languages
1 56 (58.95%) Python
2 41 (43.16%) Java
3 31 (32.63%) Scala
4 29 (30.53%) SQL
5 8 (8.42%) R
6 7 (7.37%) C++
7 6 (6.32%) C#
8 5 (5.26%) Java 8
8 5 (5.26%) JavaScript
8 5 (5.26%) Ruby
9 4 (4.21%) C
9 4 (4.21%) PHP
10 3 (3.16%) AWK
10 3 (3.16%) sed
11 2 (2.11%) Go
11 2 (2.11%) TypeScript
12 1 (1.05%) Julia
12 1 (1.05%) Kotlin
12 1 (1.05%) Perl
12 1 (1.05%) T-SQL
Qualifications
1 11 (11.58%) Degree
2 7 (7.37%) Master's Degree
2 7 (7.37%) PhD
3 2 (2.11%) Numerate Degree
4 1 (1.05%) Computer Science Degree
Quality Assurance & Compliance
1 2 (2.11%) QA
System Software
1 19 (20.00%) Docker
2 10 (10.53%) HDFS
3 4 (4.21%) Apache Flume
4 1 (1.05%) Active Directory
Systems Management
1 24 (25.26%) Kubernetes
2 9 (9.47%) Hadoop YARN
3 7 (7.37%) Terraform
4 4 (4.21%) Ansible
5 3 (3.16%) Graphite
6 2 (2.11%) Oozie
6 2 (2.11%) Puppet
7 1 (1.05%) Kubeflow
7 1 (1.05%) Prometheus
7 1 (1.05%) Zipkin
Vendors
1 19 (20.00%) Google
2 5 (5.26%) Confluent
3 4 (4.21%) Cloudera
3 4 (4.21%) MapR
4 3 (3.16%) IBM
4 3 (3.16%) Oracle
4 3 (3.16%) Splunk
5 2 (2.11%) Hortonworks
5 2 (2.11%) Informatica
5 2 (2.11%) New Relic
5 2 (2.11%) Palantir
5 2 (2.11%) SAP
5 2 (2.11%) Talend
6 1 (1.05%) Alteryx
6 1 (1.05%) BiZZdesign
6 1 (1.05%) Databricks
6 1 (1.05%) MuleSoft
6 1 (1.05%) Okta
6 1 (1.05%) Red Hat
6 1 (1.05%) Twitter