Period
to 16 December 2017

The following table provides summary statistics for contract job vacancies advertised in the City of London with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 16 December 2017 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
Central London > City of London
6 months to
16 Dec 2017
Same period 2016 Same period 2015
Rank 173 203 241
Rank change year-on-year +30 +38 +48
Contract jobs citing HDFS 78 37 20
As % of all contract IT jobs advertised in the City of London 1.09% 0.57% 0.24%
As % of the System Software category 8.93% 5.28% 2.42%
Number of daily rates quoted 71 24 17
City of London median daily rate £625 £609 £550
Median daily rate % change year-on-year +2.67% +10.68% -15.38%
10th Percentile £525 £458 £406
90th Percentile £725 £675 £729
Central London median daily rate £625 £580 £550
% change year-on-year +7.76% +5.45% -15.38%
Number of hourly rates quoted 0 0 1
City of London median hourly rate - - £37.50
Central London median hourly rate - - £37.50

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in the City of London with a requirement for system software skills.

All System Software Skills
City of London
Contract vacancies with a requirement for system software skills 873 701 825
As % of all contract IT jobs advertised in the City of London 12.22% 10.73% 9.88%
Number of daily rates quoted 690 543 632
City of London median daily rate £490 £450 £425
Median daily rate % change year-on-year +8.89% +5.88% +13.33%
10th Percentile £200 £206 £213
90th Percentile £656 £638 £630
Central London median daily rate £485 £450 £425
% change year-on-year +7.78% +5.88% +13.79%
Number of hourly rates quoted 26 20 26
City of London median hourly rate £16.50 £23.88 £17.49
Median hourly rate % change year-on-year -30.89% +36.51% +7.63%
10th Percentile £10.38 £12.38 £12.38
90th Percentile £26.63 £59.50 £27.50
Central London median hourly rate £16.50 £19.00 £17.00
% change year-on-year -13.16% +11.76% +6.25%

HDFS
Job Vacancy Trend in the City of London

Job postings citing HDFS as a percentage of all IT jobs advertised in the City of London.

Job vacancy trend for HDFS in the City of London

HDFS
Contractor Daily Rate Trend in the City of London

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing HDFS in the City of London.

Contractor daily rate trend for HDFS in the City of London

HDFS
Contractor Daily Rate Histogram in the City of London

The daily rate distribution of IT jobs citing HDFS in the City of London over the 6 months to 16 December 2017.

Contractor daily rate histogram for HDFS in the City of London

For the 6 months to 16 December 2017, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all contract ads across the City of London region with a requirement for HDFS.

1 78 (100.00%) Hadoop
2 71 (91.03%) Big Data
3 70 (89.74%) Apache Hive
4 65 (83.33%) Apache Spark
5 62 (79.49%) Java
6 54 (69.23%) Apache Flume
7 52 (66.67%) Oozie
8 48 (61.54%) Kafka
9 43 (55.13%) Python
10 42 (53.85%) HBase
11 40 (51.28%) MapReduce
12 34 (43.59%) Oracle
13 30 (38.46%) Finance
13 30 (38.46%) PySpark
14 28 (35.90%) DevOps
14 28 (35.90%) Data Modelling
15 27 (34.62%) Ambari
16 26 (33.33%) Impala
17 25 (32.05%) Data Engineering
18 24 (30.77%) PostgreSQL
18 24 (30.77%) Open Source
19 22 (28.21%) NoSQL
19 22 (28.21%) Agile Software Development
20 21 (26.92%) Hadoop YARN
20 21 (26.92%) Jenkins
20 21 (26.92%) Sqoop
21 20 (25.64%) SQL
22 19 (24.36%) Amazon AWS
23 18 (23.08%) Analytical Skills
24 17 (21.79%) Elasticsearch

HDFS
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 65 (83.33%) Apache Spark
2 17 (21.79%) Apache
2 17 (21.79%) Elasticsearch
3 7 (8.97%) Apache Pig
4 4 (5.13%) Apache Solr
5 3 (3.85%) Oracle Coherence
6 2 (2.56%) Tomcat
7 1 (1.28%) nginx
7 1 (1.28%) OpenStack
7 1 (1.28%) WebLogic
Cloud Services
1 19 (24.36%) Amazon AWS
2 14 (17.95%) Bluemix
2 14 (17.95%) GitHub
2 14 (17.95%) IBM Cloud
3 1 (1.28%) PaaS
Communications & Networking
1 4 (5.13%) Kerberos
1 4 (5.13%) SSL
2 2 (2.56%) Network Security
3 1 (1.28%) LDAP
Database & Business Intelligence
1 78 (100.00%) Hadoop
2 71 (91.03%) Big Data
3 70 (89.74%) Apache Hive
4 42 (53.85%) HBase
5 26 (33.33%) Impala
6 24 (30.77%) PostgreSQL
7 22 (28.21%) NoSQL
8 21 (26.92%) Sqoop
9 14 (17.95%) Data Warehouse
9 14 (17.95%) Oracle ODI
10 13 (16.67%) Apache Cassandra
11 12 (15.38%) RDBMS
11 12 (15.38%) Relational Database
12 11 (14.10%) Oracle Database
13 10 (12.82%) MongoDB
14 4 (5.13%) SQL Server
15 3 (3.85%) Cloudera CDH
15 3 (3.85%) Spark SQL
16 2 (2.56%) SQLite
17 1 (1.28%) MySQL
Development Applications
1 21 (26.92%) Jenkins
2 5 (6.41%) Git (software)
3 4 (5.13%) Cucumber
4 3 (3.85%) Maven
4 3 (3.85%) SonarQube
4 3 (3.85%) TeamCity
5 1 (1.28%) Codeship
5 1 (1.28%) ERwin
5 1 (1.28%) Subversion
5 1 (1.28%) Travis CI
General
1 30 (38.46%) Finance
2 17 (21.79%) Banking
3 3 (3.85%) Front Office
4 1 (1.28%) Marketing
4 1 (1.28%) Retail
4 1 (1.28%) Retail Banking
4 1 (1.28%) Telecoms
Job Titles
1 17 (21.79%) Hadoop Engineer
2 14 (17.95%) Java Engineer
2 14 (17.95%) Senior Hadoop Engineer
2 14 (17.95%) Senior Java Engineer
3 13 (16.67%) Developer
4 12 (15.38%) Big Data Developer
5 7 (8.97%) Scala Developer
6 6 (7.69%) Administrator
6 6 (7.69%) Apache Spark Developer
6 6 (7.69%) Data Engineer
7 4 (5.13%) Hadoop Administrator
8 3 (3.85%) Architect
8 3 (3.85%) Data Architect
8 3 (3.85%) Senior Software Engineer
8 3 (3.85%) Software Engineer
9 2 (2.56%) Analyst
9 2 (2.56%) Big Data Architect
9 2 (2.56%) Consultant
9 2 (2.56%) Senior Big Data Developer
9 2 (2.56%) Senior Developer
Libraries, Frameworks & Software Standards
1 48 (61.54%) Kafka
2 30 (38.46%) PySpark
3 10 (12.82%) Akka
4 5 (6.41%) RESTful
5 4 (5.13%) Apache Avro
5 4 (5.13%) Gherkin
5 4 (5.13%) OpenCL
6 2 (2.56%) .NET
6 2 (2.56%) JSON
6 2 (2.56%) RabbitMQ
6 2 (2.56%) scikit-learn
6 2 (2.56%) XML
7 1 (1.28%) D3.js
7 1 (1.28%) LEMP Stack
7 1 (1.28%) Natural Language Processing
7 1 (1.28%) REST
7 1 (1.28%) Sensu
Miscellaneous
1 18 (23.08%) Analytical Skills
2 9 (11.54%) Market Risk
3 5 (6.41%) Derivative
4 3 (3.85%) Credit Risk
4 3 (3.85%) Interest Rate Derivative
5 2 (2.56%) Computer Science
5 2 (2.56%) Data Centre
5 2 (2.56%) Greenfield Project
5 2 (2.56%) Replication
6 1 (1.28%) Algorithms
6 1 (1.28%) Credit Derivative
6 1 (1.28%) Equities
6 1 (1.28%) Foreign Exchange (FX)
6 1 (1.28%) Grid Computing
6 1 (1.28%) Life Science
6 1 (1.28%) Social Network
Operating Systems
1 12 (15.38%) Linux
2 11 (14.10%) Unix
3 3 (3.85%) Red Hat Enterprise Linux
3 3 (3.85%) Windows
4 2 (2.56%) Android
4 2 (2.56%) Apple iOS
5 1 (1.28%) AIX
5 1 (1.28%) Windows Server
5 1 (1.28%) Windows Server 2008
Processes & Methodologies
1 40 (51.28%) MapReduce
2 28 (35.90%) Data Modelling
2 28 (35.90%) DevOps
3 25 (32.05%) Data Engineering
4 24 (30.77%) Open Source
5 22 (28.21%) Agile Software Development
6 14 (17.95%) Business Intelligence
6 14 (17.95%) Multithreading
6 14 (17.95%) Use Case
7 9 (11.54%) Analytics
7 9 (11.54%) BDD
8 8 (10.26%) SDLC
9 7 (8.97%) TDD
10 5 (6.41%) Continuous Integration
10 5 (6.41%) Risk Management
11 4 (5.13%) Data Analytics
11 4 (5.13%) Data Integration
11 4 (5.13%) ETL
11 4 (5.13%) PnL Attribution
11 4 (5.13%) Software Engineering
Programming Languages
1 62 (79.49%) Java
2 43 (55.13%) Python
3 20 (25.64%) SQL
4 17 (21.79%) Scala
4 17 (21.79%) Shell Script
5 11 (14.10%) PL/SQL
6 4 (5.13%) Bash Shell
7 3 (3.85%) R
8 2 (2.56%) C#
8 2 (2.56%) COBOL
9 1 (1.28%) C++
9 1 (1.28%) Go
9 1 (1.28%) JavaScript
9 1 (1.28%) Julia
9 1 (1.28%) Ruby
9 1 (1.28%) T-SQL
Qualifications
1 1 (1.28%) Computer Science Degree
1 1 (1.28%) Degree
1 1 (1.28%) Numerate Degree
1 1 (1.28%) SC Cleared
1 1 (1.28%) Security Cleared
Quality Assurance & Compliance
1 2 (2.56%) Dodd-Frank
1 2 (2.56%) EMIR
1 2 (2.56%) MiFID
System Software
1 54 (69.23%) Apache Flume
2 14 (17.95%) Docker
3 10 (12.82%) Apache ZooKeeper
4 2 (2.56%) Active Directory
Systems Management
1 52 (66.67%) Oozie
2 27 (34.62%) Ambari
3 21 (26.92%) Hadoop YARN
4 14 (17.95%) Kubernetes
5 7 (8.97%) Ansible
6 6 (7.69%) Puppet
7 5 (6.41%) Opscode Chef
8 4 (5.13%) Mesos
9 2 (2.56%) Apache Slider
9 2 (2.56%) Control-M
10 1 (1.28%) Salt
10 1 (1.28%) ZABBIX
Vendors
1 34 (43.59%) Oracle
2 15 (19.23%) Tableau
3 14 (17.95%) IBM
4 9 (11.54%) Cloudera
5 4 (5.13%) Hortonworks
6 3 (3.85%) SaltStack
7 2 (2.56%) Palantir
8 1 (1.28%) Informatica
8 1 (1.28%) SAP
8 1 (1.28%) Splunk