Period
to 11 December 2017

The following table provides summary statistics for contract job vacancies advertised in the City of London with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 11 December 2017 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
Central London > City of London
6 months to
11 Dec 2017
Same period 2016 Same period 2015
Rank 103 86 171
Rank change year-on-year -17 +85 +99
Contract jobs citing Apache Spark 174 183 90
As % of all contract IT jobs advertised in the City of London 2.42% 2.81% 1.06%
As % of the Application Platforms category 17.03% 15.67% 6.53%
Number of daily rates quoted 157 155 83
City of London median daily rate £600 £600 £600
10th Percentile £506 £443 £422
90th Percentile £725 £688 £737
Central London median daily rate £600 £591 £600
% change year-on-year +1.48% -1.46% +9.09%
Number of hourly rates quoted 0 0 1
City of London median hourly rate - - £37.50
Central London median hourly rate - - £37.50

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in the City of London with a requirement for application platform skills.

All Application Platform Skills
City of London
Contract vacancies with a requirement for application platform skills 1022 1168 1379
As % of all contract IT jobs advertised in the City of London 14.22% 17.92% 16.22%
Number of daily rates quoted 808 919 1043
City of London median daily rate £500 £473 £425
Median daily rate % change year-on-year +5.82% +11.18% +6.25%
10th Percentile £275 £313 £263
90th Percentile £675 £664 £638
Central London median daily rate £499 £455 £425
% change year-on-year +9.62% +7.06% +6.25%
Number of hourly rates quoted 8 15 16
City of London median hourly rate £17.00 £19.00 £15.10
Median hourly rate % change year-on-year -10.53% +25.87% -7.11%
10th Percentile £10.88 £11.25 £10.63
90th Percentile £27.50 £56.12 £31.25
Central London median hourly rate £17.85 £18.45 £17.25
% change year-on-year -3.25% +6.96% +4.55%

Apache Spark
Job Vacancy Trend in the City of London

Job postings citing Apache Spark as a percentage of all IT jobs advertised in the City of London.

Job vacancy trend for Apache Spark in the City of London

Apache Spark
Contractor Daily Rate Trend in the City of London

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Apache Spark in the City of London.

Contractor daily rate trend for Apache Spark in the City of London

Apache Spark
Contractor Daily Rate Histogram in the City of London

The daily rate distribution of IT jobs citing Apache Spark in the City of London over the 6 months to 11 December 2017.

Contractor daily rate histogram for Apache Spark in the City of London

For the 6 months to 11 December 2017, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all contract ads across the City of London region with a requirement for Apache Spark.

1 143 (82.18%) Big Data
2 132 (75.86%) Hadoop
3 114 (65.52%) Java
4 96 (55.17%) Python
5 94 (54.02%) Apache Hive
6 80 (45.98%) Kafka
7 67 (38.51%) HDFS
8 63 (36.21%) Agile Software Development
9 62 (35.63%) Scala
10 54 (31.03%) MapReduce
11 53 (30.46%) HBase
11 53 (30.46%) Apache Flume
11 53 (30.46%) SQL
11 53 (30.46%) Oozie
12 51 (29.31%) DevOps
13 50 (28.74%) Banking
14 47 (27.01%) Finance
15 43 (24.71%) Amazon AWS
16 38 (21.84%) NoSQL
16 38 (21.84%) TDD
17 37 (21.26%) Open Source
18 36 (20.69%) Analytics
19 35 (20.11%) Oracle
20 34 (19.54%) Data Engineering
21 31 (17.82%) MongoDB
22 30 (17.24%) Apache Cassandra
23 29 (16.67%) Data Modelling
23 29 (16.67%) Impala
23 29 (16.67%) Ambari
23 29 (16.67%) PySpark

Apache Spark
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 25 (14.37%) Apache
2 23 (13.22%) Elasticsearch
3 11 (6.32%) Apache Pig
4 6 (3.45%) Apache Solr
5 5 (2.87%) SAS
6 4 (2.30%) nginx
7 3 (1.72%) Oracle Coherence
7 3 (1.72%) Tomcat
8 2 (1.15%) OpenStack
9 1 (0.57%) Apache Airflow
9 1 (0.57%) Confluence
9 1 (0.57%) Jetty
Applications
1 2 (1.15%) SPSS
1 2 (1.15%) Weka
2 1 (0.57%) Chatbot
Cloud Services
1 43 (24.71%) Amazon AWS
2 18 (10.34%) GitHub
3 14 (8.05%) Bluemix
3 14 (8.05%) IBM Cloud
4 6 (3.45%) Microsoft Azure
5 3 (1.72%) Google Cloud Platform
5 3 (1.72%) PaaS
6 2 (1.15%) Amazon Kinesis
6 2 (1.15%) Amazon S3
6 2 (1.15%) Cloud Computing
6 2 (1.15%) SaaS
7 1 (0.57%) Amazon EC2
7 1 (0.57%) Amazon EMR
7 1 (0.57%) AWS CloudTrail
7 1 (0.57%) AWS Lambda
7 1 (0.57%) Azure Active Directory
7 1 (0.57%) CloudFront
7 1 (0.57%) Route 53
7 1 (0.57%) Serverless
7 1 (0.57%) Virtual Private Cloud
Communications & Networking
1 6 (3.45%) Kerberos
2 5 (2.87%) HTTP
3 3 (1.72%) LDAP
4 2 (1.15%) SSL
Database & Business Intelligence
1 143 (82.18%) Big Data
2 132 (75.86%) Hadoop
3 94 (54.02%) Apache Hive
4 53 (30.46%) HBase
5 38 (21.84%) NoSQL
6 31 (17.82%) MongoDB
7 30 (17.24%) Apache Cassandra
8 29 (16.67%) Impala
9 26 (14.94%) Sqoop
10 24 (13.79%) PostgreSQL
11 19 (10.92%) Data Warehouse
12 14 (8.05%) Oracle ODI
13 12 (6.90%) Relational Database
14 10 (5.75%) Oracle Database
14 10 (5.75%) RDBMS
15 5 (2.87%) Amazon Redshift
15 5 (2.87%) CouchDB
15 5 (2.87%) SQL Server
16 4 (2.30%) QlikView
17 3 (1.72%) SAS Enterprise Guide
Development Applications
1 28 (16.09%) Jenkins
2 17 (9.77%) Git (software)
3 5 (2.87%) Maven
3 5 (2.87%) SonarQube
3 5 (2.87%) TeamCity
4 4 (2.30%) Cucumber
5 3 (1.72%) GoCD
5 3 (1.72%) Subversion
5 3 (1.72%) Talend Open Studio
5 3 (1.72%) Travis CI
6 2 (1.15%) GitLab
7 1 (0.57%) Babel
7 1 (0.57%) Bower
7 1 (0.57%) Gradle
7 1 (0.57%) Grunt
7 1 (0.57%) ProC
7 1 (0.57%) Visual Studio
7 1 (0.57%) Visual Studio Team System
General
1 50 (28.74%) Banking
2 47 (27.01%) Finance
3 15 (8.62%) Investment Banking
4 8 (4.60%) Front Office
5 5 (2.87%) Publishing
6 3 (1.72%) Marketing
6 3 (1.72%) Retail
7 2 (1.15%) Advertising
8 1 (0.57%) Telecoms
Job Titles
1 48 (27.59%) Developer
2 37 (21.26%) Data Engineer
3 26 (14.94%) Java Engineer
4 24 (13.79%) Big Data Developer
5 21 (12.07%) Scala Developer
6 19 (10.92%) Big Data Engineer
6 19 (10.92%) Hadoop Engineer
7 14 (8.05%) Data Scientist
7 14 (8.05%) Senior Hadoop Engineer
7 14 (8.05%) Senior Java Engineer
8 10 (5.75%) Apache Spark Developer
8 10 (5.75%) Java Developer
9 9 (5.17%) DevOps Engineer
10 7 (4.02%) Architect
11 5 (2.87%) AWS Engineer
12 4 (2.30%) Big Data Architect
12 4 (2.30%) Data Architect
12 4 (2.30%) Programmer
12 4 (2.30%) Senior Developer
12 4 (2.30%) Software Engineer
Libraries, Frameworks & Software Standards
1 80 (45.98%) Kafka
2 29 (16.67%) PySpark
3 21 (12.07%) Akka
4 14 (8.05%) React
5 13 (7.47%) REST
6 12 (6.90%) Redux
7 11 (6.32%) JSON
8 8 (4.60%) AngularJS
8 8 (4.60%) Twitter Bootstrap
9 6 (3.45%) Node.js
9 6 (3.45%) OpenCL
9 6 (3.45%) Spring
10 5 (2.87%) HTML
10 5 (2.87%) HTML5
10 5 (2.87%) Natural Language Processing
10 5 (2.87%) Spring MVC
10 5 (2.87%) Web Services
11 4 (2.30%) Gherkin
11 4 (2.30%) RabbitMQ
11 4 (2.30%) RESTful
Miscellaneous
1 21 (12.07%) Analytical Skills
2 13 (7.47%) Algorithms
3 11 (6.32%) Market Risk
4 10 (5.75%) Computer Science
5 9 (5.17%) Derivative
6 5 (2.87%) Greenfield Project
7 4 (2.30%) Credit Risk
7 4 (2.30%) User Experience
8 3 (1.72%) Data Centre
8 3 (1.72%) Interest Rate Derivative
9 2 (1.15%) Data Structures
9 2 (1.15%) Grid Computing
9 2 (1.15%) Linked Data
9 2 (1.15%) Semantic Web
10 1 (0.57%) Equities
10 1 (0.57%) iPad
10 1 (0.57%) Life Science
10 1 (0.57%) Replication
10 1 (0.57%) Social Network
10 1 (0.57%) Wiki
Operating Systems
1 21 (12.07%) Linux
2 10 (5.75%) Unix
3 5 (2.87%) Red Hat Enterprise Linux
4 4 (2.30%) Android
4 4 (2.30%) Apple iOS
5 1 (0.57%) CentOS
5 1 (0.57%) Windows
5 1 (0.57%) Windows Server
5 1 (0.57%) Windows Server 2008
Processes & Methodologies
1 63 (36.21%) Agile Software Development
2 54 (31.03%) MapReduce
3 51 (29.31%) DevOps
4 38 (21.84%) TDD
5 37 (21.26%) Open Source
6 36 (20.69%) Analytics
7 34 (19.54%) Data Engineering
8 29 (16.67%) Data Modelling
9 18 (10.34%) Business Intelligence
10 17 (9.77%) Machine Learning
10 17 (9.77%) Use Case
11 16 (9.20%) BDD
11 16 (9.20%) ETL
11 16 (9.20%) Multithreading
11 16 (9.20%) Software Engineering
12 15 (8.62%) Microservices
13 14 (8.05%) Continuous Integration
13 14 (8.05%) Visualisation
14 10 (5.75%) Data Analytics
14 10 (5.75%) Scrum
Programming Languages
1 114 (65.52%) Java
2 96 (55.17%) Python
3 62 (35.63%) Scala
4 53 (30.46%) SQL
5 16 (9.20%) Shell Script
6 14 (8.05%) R
7 10 (5.75%) JavaScript
7 10 (5.75%) PL/SQL
8 6 (3.45%) Bash Shell
9 4 (2.30%) Apple Swift
9 4 (2.30%) GraphQL
9 4 (2.30%) Haskell
9 4 (2.30%) Objective-C
10 2 (1.15%) AWK
10 2 (1.15%) sed
10 2 (1.15%) SPARQL
11 1 (0.57%) C#
11 1 (0.57%) C++
11 1 (0.57%) Ruby
11 1 (0.57%) SAS Macro
Qualifications
1 8 (4.60%) Degree
2 5 (2.87%) Computer Science Degree
3 3 (1.72%) Security Cleared
4 2 (1.15%) Master's Degree
4 2 (1.15%) Numerate Degree
4 2 (1.15%) SC Cleared
Quality Assurance & Compliance
1 4 (2.30%) QA
2 3 (1.72%) IFRS
2 3 (1.72%) IFRS 9
3 2 (1.15%) Data Quality
System Software
1 67 (38.51%) HDFS
2 53 (30.46%) Apache Flume
3 26 (14.94%) Docker
4 12 (6.90%) Apache ZooKeeper
5 2 (1.15%) Active Directory
6 1 (0.57%) Virtual Machines
Systems Management
1 53 (30.46%) Oozie
2 29 (16.67%) Ambari
3 21 (12.07%) Hadoop YARN
4 16 (9.20%) Kubernetes
5 11 (6.32%) Ansible
6 7 (4.02%) Mesos
7 5 (2.87%) Puppet
7 5 (2.87%) Terraform
8 3 (1.72%) Apache Slider
8 3 (1.72%) Salt
9 2 (1.15%) Rundeck
10 1 (0.57%) Opscode Chef
Vendors
1 35 (20.11%) Oracle
2 25 (14.37%) Tableau
3 14 (8.05%) IBM
4 11 (6.32%) Cloudera
5 7 (4.02%) Hortonworks
6 4 (2.30%) Google
7 3 (1.72%) SaltStack
7 3 (1.72%) Talend
7 3 (1.72%) Vertica
8 2 (1.15%) Informatica
8 2 (1.15%) Salesforce.com
9 1 (0.57%) Cognos
9 1 (0.57%) Databricks
9 1 (0.57%) Facebook
9 1 (0.57%) MapR
9 1 (0.57%) Microsoft
9 1 (0.57%) Netezza
9 1 (0.57%) Red Hat
9 1 (0.57%) SAP
9 1 (0.57%) Sybase