Period
to 21 August 2018

The following table provides summary statistics for contract job vacancies advertised in the City of London with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 21 August 2018 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
Central London > City of London
6 months to
21 Aug 2018
Same period 2017 Same period 2016
Rank 212 201 233
Rank change year-on-year -11 +32 +40
Contract jobs citing HDFS 48 49 15
As % of all contract IT jobs advertised in the City of London 0.68% 0.74% 0.22%
As % of the System Software category 4.26% 6.55% 2.02%
Number of daily rates quoted 42 44 11
City of London median daily rate £550 £625 £463
Median daily rate % change year-on-year -12.00% +35.14% -17.78%
10th Percentile £513 £416 £375
90th Percentile £675 £725 £825
Central London median daily rate £550 £625 £425
% change year-on-year -12.00% +47.06% -24.44%
Number of hourly rates quoted 0 0 0
City of London median hourly rate - - -
Central London median hourly rate - - -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in the City of London with a requirement for system software skills.

All System Software Skills
City of London
Contract vacancies with a requirement for system software skills 1,127 748 741
As % of all contract IT jobs advertised in the City of London 16.06% 11.29% 10.96%
Number of daily rates quoted 900 601 600
City of London median daily rate £500 £475 £450
Median daily rate % change year-on-year +5.26% +5.56% +12.50%
10th Percentile £200 £185 £240
90th Percentile £650 £638 £625
Central London median daily rate £500 £475 £450
% change year-on-year +5.26% +5.56% +12.50%
Number of hourly rates quoted 42 21 26
City of London median hourly rate £19.25 £21.25 £16.63
Median hourly rate % change year-on-year -9.41% +27.82% -7.64%
10th Percentile £13.31 £13.31 £11.25
90th Percentile £25.50 £29.62 £23.00
Central London median hourly rate £19.25 £19.00 £15.63
% change year-on-year +1.32% +21.60% -13.10%

HDFS
Job Vacancy Trend in the City of London

Job postings citing HDFS as a percentage of all IT jobs advertised in the City of London.

Job vacancy trend for HDFS in the City of London

HDFS
Contractor Daily Rate Trend in the City of London

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing HDFS in the City of London.

Contractor daily rate trend for HDFS in the City of London

HDFS
Contractor Daily Rate Histogram in the City of London

The daily rate distribution of IT jobs citing HDFS in the City of London over the 6 months to 21 August 2018.

Contractor daily rate histogram for HDFS in the City of London

HDFS
Contract Job Locations in the City of London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the City of London region over the 6 months to 21 August 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London Bridge - 15 £525 -
HDFS
Central London

For the 6 months to 21 August 2018, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the City of London region with a requirement for HDFS.

1 48 (100.00%) Hadoop
2 42 (87.50%) Apache Hive
3 40 (83.33%) Big Data
4 34 (70.83%) Apache Spark
5 29 (60.42%) Python
6 27 (56.25%) ETL
6 27 (56.25%) SQL
7 26 (54.17%) Amazon AWS
8 23 (47.92%) Hadoop YARN
9 19 (39.58%) Sqoop
10 18 (37.50%) Impala
10 18 (37.50%) SQL Server
11 16 (33.33%) Jenkins
12 15 (31.25%) Unix
12 15 (31.25%) Agile Software Development
13 14 (29.17%) Scrum
13 14 (29.17%) Linux
13 14 (29.17%) Amazon S3
13 14 (29.17%) Continuous Integration
13 14 (29.17%) Cloudera
13 14 (29.17%) Ansible
14 13 (27.08%) Virtual Private Cloud
14 13 (27.08%) Oozie
14 13 (27.08%) Opscode Chef
14 13 (27.08%) Build Automation
14 13 (27.08%) Puppet
14 13 (27.08%) Bash Shell
14 13 (27.08%) Data Extraction
14 13 (27.08%) PowerShell
14 13 (27.08%) Wiki

HDFS
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 34 (70.83%) Apache Spark
2 10 (20.83%) Apache Solr
3 8 (16.67%) Apache Pig
4 1 (2.08%) Apache
4 1 (2.08%) Confluence
4 1 (2.08%) Elasticsearch
4 1 (2.08%) Tomcat
Applications
1 13 (27.08%) Microsoft Excel
1 13 (27.08%) MS Visio
Cloud Services
1 26 (54.17%) Amazon AWS
2 14 (29.17%) Amazon S3
3 13 (27.08%) Amazon EC2
3 13 (27.08%) Virtual Private Cloud
4 7 (14.58%) Microsoft Azure
5 2 (4.17%) GitHub
5 2 (4.17%) IBM Cloud
6 1 (2.08%) Amazon EMR
6 1 (2.08%) Azure Data Factory
6 1 (2.08%) Google Cloud Platform
Communications & Networking
1 8 (16.67%) Kerberos
2 6 (12.50%) SSL
3 1 (2.08%) HTTP
Database & Business Intelligence
1 48 (100.00%) Hadoop
2 42 (87.50%) Apache Hive
3 40 (83.33%) Big Data
4 19 (39.58%) Sqoop
5 18 (37.50%) Impala
5 18 (37.50%) SQL Server
6 10 (20.83%) HBase
7 7 (14.58%) Amazon Redshift
7 7 (14.58%) Apache Cassandra
8 5 (10.42%) PostgreSQL
9 4 (8.33%) Data Warehouse
9 4 (8.33%) SQL Server Integration Services
10 3 (6.25%) MongoDB
10 3 (6.25%) NoSQL
10 3 (6.25%) Relational Database
11 2 (4.17%) Apache Kudu
11 2 (4.17%) Data Lake
11 2 (4.17%) SQL Server 2008
11 2 (4.17%) SQL Server 2012
11 2 (4.17%) SQL Server 2014
Development Applications
1 16 (33.33%) Jenkins
2 2 (4.17%) Git (software)
2 2 (4.17%) Subversion
3 1 (2.08%) Atlassian Bamboo
3 1 (2.08%) FishEye
3 1 (2.08%) GitLab
3 1 (2.08%) JIRA
3 1 (2.08%) jMock
3 1 (2.08%) JUnit
3 1 (2.08%) Maven
General
1 9 (18.75%) Finance
2 3 (6.25%) Banking
3 2 (4.17%) Marketing
4 1 (2.08%) Investment Banking
Job Titles
1 14 (29.17%) Big Data Engineer
1 14 (29.17%) Data Engineer
2 13 (27.08%) Platform Engineer
3 11 (22.92%) Developer
4 9 (18.75%) Big Data Developer
5 7 (14.58%) Apache Spark Developer
6 6 (12.50%) AWS Engineer
6 6 (12.50%) Consultant
6 6 (12.50%) Python Engineer
6 6 (12.50%) Senior Big Data Developer
6 6 (12.50%) Senior Developer
7 5 (10.42%) AWS Developer
7 5 (10.42%) Principal Consultant
8 4 (8.33%) Hadoop Consultant
9 2 (4.17%) Database Engineer
9 2 (4.17%) Java Developer
9 2 (4.17%) Scala Developer
9 2 (4.17%) Senior Database Engineer
9 2 (4.17%) Site Engineer
9 2 (4.17%) Site Reliability Engineer
Libraries, Frameworks & Software Standards
1 11 (22.92%) Kafka
2 2 (4.17%) PySpark
2 2 (4.17%) REST
2 2 (4.17%) XML
3 1 (2.08%) CSS
3 1 (2.08%) CSS3
3 1 (2.08%) Hibernate
3 1 (2.08%) HTML
3 1 (2.08%) HTML5
3 1 (2.08%) J2EE
3 1 (2.08%) Java EE
3 1 (2.08%) JAXB
3 1 (2.08%) JMS
3 1 (2.08%) JSON
3 1 (2.08%) JTA
3 1 (2.08%) JTS
3 1 (2.08%) Lucene
3 1 (2.08%) RabbitMQ
3 1 (2.08%) Spring
3 1 (2.08%) XSLT
Miscellaneous
1 13 (27.08%) Wiki
2 6 (12.50%) Greenfield Project
3 2 (4.17%) Algorithms
3 2 (4.17%) Analytical Skills
4 1 (2.08%) Clustering
4 1 (2.08%) Data Centre
4 1 (2.08%) Fixed Income
4 1 (2.08%) Market Risk
Operating Systems
1 15 (31.25%) Unix
2 14 (29.17%) Linux
3 13 (27.08%) Windows
4 5 (10.42%) Red Hat Enterprise Linux
5 2 (4.17%) DC/OS
Processes & Methodologies
1 27 (56.25%) ETL
2 15 (31.25%) Agile Software Development
3 14 (29.17%) Continuous Integration
3 14 (29.17%) Scrum
4 13 (27.08%) Build Automation
4 13 (27.08%) Data Analysis
4 13 (27.08%) Data Extraction
4 13 (27.08%) Test Automation
5 12 (25.00%) Analytics
6 7 (14.58%) DevOps
7 6 (12.50%) E-Commerce
8 5 (10.42%) Capacity Management
8 5 (10.42%) Change Management
8 5 (10.42%) Data Engineering
8 5 (10.42%) MapReduce
8 5 (10.42%) Mentoring
9 4 (8.33%) Data Modelling
10 3 (6.25%) ETL Development
10 3 (6.25%) Machine Learning
10 3 (6.25%) SDLC
Programming Languages
1 29 (60.42%) Python
2 27 (56.25%) SQL
3 13 (27.08%) Bash Shell
3 13 (27.08%) Java
3 13 (27.08%) Korn
3 13 (27.08%) PowerShell
3 13 (27.08%) Shell Script
4 12 (25.00%) Scala
5 3 (6.25%) T-SQL
6 2 (4.17%) Java 8
7 1 (2.08%) JavaScript
7 1 (2.08%) R
System Software
1 9 (18.75%) Apache ZooKeeper
2 8 (16.67%) Apache Flume
3 6 (12.50%) Active Directory
Systems Management
1 23 (47.92%) Hadoop YARN
2 14 (29.17%) Ansible
3 13 (27.08%) Oozie
3 13 (27.08%) Opscode Chef
3 13 (27.08%) Puppet
4 8 (16.67%) Ambari
5 6 (12.50%) Apache Slider
6 2 (4.17%) Kubernetes
6 2 (4.17%) Mesos
Vendors
1 14 (29.17%) Cloudera
2 13 (27.08%) Oracle
3 7 (14.58%) Hortonworks
4 4 (8.33%) Tableau
5 3 (6.25%) Informatica
6 2 (4.17%) IBM
7 1 (2.08%) Atlassian
7 1 (2.08%) Google
7 1 (2.08%) Salesforce.com
7 1 (2.08%) Sybase
7 1 (2.08%) Talend