Period
to 18 February 2020

The following table provides summary statistics for contract job vacancies advertised in Central London with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 18 February 2020 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
London > Central London
6 months to
18 Feb 2020
Same period 2019 Same period 2018
Rank 124 69 117
Rank change year-on-year -55 +48 -21
Contract jobs citing Apache Spark 103 308 183
As % of all contract IT jobs advertised in Central London 1.73% 3.47% 2.07%
As % of the Application Platforms category 11.55% 19.69% 13.32%
Number of daily rates quoted 78 259 165
Central London median daily rate £575 £550 £575
Median daily rate % change year-on-year +4.55% -4.35% -4.17%
10th Percentile £410 £513 £464
90th Percentile £707 £700 £725
London median daily rate £575 £575 £575
Number of hourly rates quoted 0 0 0
Central London median hourly rate - - -
London median hourly rate £52.50 £50.00 £82.50
% change year-on-year +5.00% -39.39% -17.50%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in Central London with a requirement for application platform skills.

All Application Platform Skills
Central London
Contract vacancies with a requirement for application platform skills 892 1,564 1,374
As % of all contract IT jobs advertised in Central London 14.94% 17.60% 15.56%
Number of daily rates quoted 656 1,170 1,085
Central London median daily rate £500 £525 £496
Median daily rate % change year-on-year -4.76% +5.85% +4.42%
10th Percentile £263 £275 £275
90th Percentile £663 £650 £646
London median daily rate £500 £513 £500
% change year-on-year -2.44% +2.50% +11.11%
Number of hourly rates quoted 8 22 14
Central London median hourly rate £26.00 £18.88 £19.35
Median hourly rate % change year-on-year +37.71% -2.43% +1.84%
10th Percentile £20.30 £13.00 £11.88
90th Percentile £35.00 £32.93 £29.25
London median hourly rate £26.00 £26.75 £20.25
% change year-on-year -2.80% +32.10% +3.85%

Apache Spark
Job Vacancy Trend in Central London

Job postings citing Apache Spark as a percentage of all IT jobs advertised in Central London.

Job vacancy trend for Apache Spark in Central London

Apache Spark
Contractor Daily Rate Trend in Central London

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Apache Spark in Central London.

Contractor daily rate trend for Apache Spark in Central London

Apache Spark
Contractor Daily Rate Histogram in Central London

The daily rate distribution of IT jobs citing Apache Spark in Central London over the 6 months to 18 February 2020.

Contractor daily rate histogram for Apache Spark in Central London

Apache Spark
Contract Job Locations in Central London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the Central London region over the 6 months to 18 February 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
City of London -59 78 £575 +4.55% 64
City of Westminster - 4 £900 -
Southwark -13 3 £500 -4.76%
West End of London +37 1 - -
Apache Spark
London

For the 6 months to 18 February 2020, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the Central London region with a requirement for Apache Spark.

1 66 (64.08%) Python
2 55 (53.40%) Big Data
3 53 (51.46%) Hadoop
4 50 (48.54%) Kafka
5 46 (44.66%) AWS
6 43 (41.75%) Java
7 42 (40.78%) Agile Software Development
8 39 (37.86%) SQL
9 37 (35.92%) Scala
10 31 (30.10%) Kubernetes
11 30 (29.13%) Data Science
12 29 (28.16%) GCP
12 29 (28.16%) Docker
13 27 (26.21%) Analytics
13 27 (26.21%) Machine Learning
14 23 (22.33%) Apache Hive
15 22 (21.36%) Finance
15 22 (21.36%) Google
16 21 (20.39%) ETL
17 20 (19.42%) Spark Streaming
18 18 (17.48%) DevOps
19 17 (16.50%) Microservices
19 17 (16.50%) NoSQL
19 17 (16.50%) Computer Science
19 17 (16.50%) Apache Airflow
20 16 (15.53%) PostgreSQL
21 15 (14.56%) Open Source
21 15 (14.56%) Jenkins
22 14 (13.59%) Apache Cassandra
22 14 (13.59%) Algorithms

Apache Spark
Co-occurring IT Skills in Central London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 17 (16.50%) Apache Airflow
2 14 (13.59%) Apache
3 9 (8.74%) Elasticsearch
4 5 (4.85%) Luigi
5 2 (1.94%) DNN
5 2 (1.94%) Jupyter
6 1 (0.97%) Apache Pig
6 1 (0.97%) Oracle Workflow
6 1 (0.97%) Pachyderm
6 1 (0.97%) SAS
Applications
1 1 (0.97%) Microsoft Excel
Business Applications
1 1 (0.97%) FINCAD Analytics
1 1 (0.97%) SAP Oil and Gas
Cloud Services
1 46 (44.66%) AWS
2 29 (28.16%) GCP
3 14 (13.59%) Azure
4 8 (7.77%) Amazon S3
5 7 (6.80%) Amazon EMR
5 7 (6.80%) Amazon Kinesis
5 7 (6.80%) Serverless
6 6 (5.83%) AWS Lambda
7 4 (3.88%) Amazon EC2
7 4 (3.88%) AWS CloudFormation
7 4 (3.88%) SaaS
8 3 (2.91%) AWS IoT
9 2 (1.94%) Amazon CloudWatch
9 2 (1.94%) Amazon EKS
9 2 (1.94%) Data Lake Analytics
9 2 (1.94%) Virtual Private Cloud
10 1 (0.97%) Amazon SQS
10 1 (0.97%) AWS Step Functions
10 1 (0.97%) Cloud Computing
10 1 (0.97%) Power Platform
Communications & Networking
1 2 (1.94%) TCP/IP
2 1 (0.97%) Internet
Database & Business Intelligence
1 55 (53.40%) Big Data
2 53 (51.46%) Hadoop
3 23 (22.33%) Apache Hive
4 17 (16.50%) NoSQL
5 16 (15.53%) PostgreSQL
6 14 (13.59%) Apache Cassandra
6 14 (13.59%) Data Warehouse
6 14 (13.59%) HBase
7 12 (11.65%) Amazon Redshift
8 9 (8.74%) BigQuery
9 8 (7.77%) Impala
9 8 (7.77%) MongoDB
10 7 (6.80%) Amazon Athena
11 6 (5.83%) Amazon RDS
11 6 (5.83%) Flink
11 6 (5.83%) Relational Database
11 6 (5.83%) Spark SQL
12 5 (4.85%) Amazon Aurora
12 5 (4.85%) Bigtable
12 5 (4.85%) Cloud Spanner
Development Applications
1 15 (14.56%) Jenkins
2 6 (5.83%) Git (software)
3 5 (4.85%) Maven
4 4 (3.88%) JIRA
4 4 (3.88%) Travis CI
5 3 (2.91%) Atlassian Bamboo
5 3 (2.91%) TeamCity
6 1 (0.97%) ERwin
6 1 (0.97%) GitLab
6 1 (0.97%) GoCD
6 1 (0.97%) Vagrant
General
1 22 (21.36%) Finance
2 13 (12.62%) Banking
3 7 (6.80%) Front Office
4 4 (3.88%) Retail Banking
5 3 (2.91%) Marketing
6 2 (1.94%) Telecoms
7 1 (0.97%) Billing
7 1 (0.97%) Investment Banking
7 1 (0.97%) Public Sector
7 1 (0.97%) Retail
Job Titles
1 42 (40.78%) Data Engineer
2 32 (31.07%) Developer
3 12 (11.65%) Big Data Engineer
4 10 (9.71%) Data Scientist
5 9 (8.74%) Java Developer
5 9 (8.74%) Python Developer
6 8 (7.77%) Senior Developer
7 7 (6.80%) Architect
8 6 (5.83%) AWS Engineer
8 6 (5.83%) Python Engineer
8 6 (5.83%) Scala Developer
9 5 (4.85%) AWS Developer
9 5 (4.85%) DevOps Engineer
10 4 (3.88%) Cloud Engineer
10 4 (3.88%) Hadoop Developer
10 4 (3.88%) Java Hadoop Developer
10 4 (3.88%) Machine Learning Scientist
11 3 (2.91%) Backend Developer
11 3 (2.91%) Energy Trading Developer
11 3 (2.91%) Senior Scala Developer
Libraries, Frameworks & Software Standards
1 50 (48.54%) Kafka
2 20 (19.42%) Spark Streaming
3 13 (12.62%) PySpark
4 12 (11.65%) Spring
5 11 (10.68%) Pandas
5 11 (10.68%) Web Services
6 7 (6.80%) Akka
6 7 (6.80%) Apache Beam
6 7 (6.80%) RESTful
7 6 (5.83%) Node.js
7 6 (5.83%) Spring Boot
8 5 (4.85%) AngularJS
8 5 (4.85%) React
8 5 (4.85%) REST
8 5 (4.85%) scikit-learn
8 5 (4.85%) SciPy
8 5 (4.85%) TensorFlow
9 4 (3.88%) Backbone.js
9 4 (3.88%) MLlib
10 3 (2.91%) Lambda Expressions
Miscellaneous
1 14 (13.59%) Algorithms
1 14 (13.59%) Analytical Skills
2 12 (11.65%) Data Structures
3 6 (5.83%) Greenfield Project
4 4 (3.88%) Hedge funds
4 4 (3.88%) Linux Command Line
5 3 (2.91%) Cloud Native
5 3 (2.91%) Fixed Income
5 3 (2.91%) Tandem
6 2 (1.94%) Foreign Exchange (FX)
6 2 (1.94%) Life Science
6 2 (1.94%) Self-Motivation
6 2 (1.94%) Semantic Web
7 1 (0.97%) Derivative
7 1 (0.97%) Internet of Things
7 1 (0.97%) Public Cloud
Operating Systems
1 10 (9.71%) Linux
2 6 (5.83%) Unix
3 3 (2.91%) Windows
Processes & Methodologies
1 42 (40.78%) Agile Software Development
2 30 (29.13%) Data Science
3 27 (26.21%) Analytics
3 27 (26.21%) Machine Learning
4 21 (20.39%) ETL
5 18 (17.48%) DevOps
6 17 (16.50%) Computer Science
6 17 (16.50%) Microservices
7 15 (14.56%) Open Source
8 14 (13.59%) TDD
9 13 (12.62%) Use Case
10 12 (11.65%) Artificial Intelligence
11 10 (9.71%) Data Transformation
12 9 (8.74%) Data Engineering
13 8 (7.77%) BDD
13 8 (7.77%) OO
14 7 (6.80%) Business Intelligence
14 7 (6.80%) E-Commerce
14 7 (6.80%) OOP
14 7 (6.80%) Statistics
Programming Languages
1 66 (64.08%) Python
2 43 (41.75%) Java
3 39 (37.86%) SQL
4 37 (35.92%) Scala
5 9 (8.74%) R
6 8 (7.77%) C++
7 6 (5.83%) Go
8 5 (4.85%) Ruby
9 4 (3.88%) C
9 4 (3.88%) C#
9 4 (3.88%) JavaScript
10 3 (2.91%) AWK
10 3 (2.91%) PHP
10 3 (2.91%) sed
11 1 (0.97%) Bash Shell
11 1 (0.97%) HiveQL
11 1 (0.97%) Julia
11 1 (0.97%) PL/SQL
Qualifications
1 11 (10.68%) Degree
2 7 (6.80%) PhD
3 4 (3.88%) Master's Degree
4 3 (2.91%) Computer Science Degree
5 1 (0.97%) Numerate Degree
Quality Assurance & Compliance
1 3 (2.91%) QA
System Software
1 29 (28.16%) Docker
2 11 (10.68%) HDFS
3 4 (3.88%) Apache Flume
4 1 (0.97%) Apache ZooKeeper
4 1 (0.97%) VirtualBox
Systems Management
1 31 (30.10%) Kubernetes
2 11 (10.68%) Hadoop YARN
2 11 (10.68%) Terraform
3 7 (6.80%) Ansible
4 3 (2.91%) Graphite
4 3 (2.91%) Oozie
4 3 (2.91%) Puppet
5 1 (0.97%) Docker Swarm
5 1 (0.97%) Kubeflow
5 1 (0.97%) Opscode Chef
5 1 (0.97%) Prometheus
Vendors
1 22 (21.36%) Google
2 6 (5.83%) Cloudera
3 5 (4.85%) Confluent
3 5 (4.85%) Hortonworks
3 5 (4.85%) Oracle
4 4 (3.88%) IBM
4 4 (3.88%) Palantir
5 3 (2.91%) Informatica
5 3 (2.91%) New Relic
5 3 (2.91%) Splunk
5 3 (2.91%) Tableau
5 3 (2.91%) Talend
5 3 (2.91%) Teradata
6 2 (1.94%) Databricks
6 2 (1.94%) Microsoft
7 1 (0.97%) Ab Initio
7 1 (0.97%) Bloomberg
7 1 (0.97%) Datactics
7 1 (0.97%) Okta
7 1 (0.97%) SAP