Period
to 17 September 2019

The following table provides summary statistics for contract job vacancies advertised in the City of London with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 17 September 2019 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
Central London > City of London
6 months to
17 Sep 2019
Same period 2018 Same period 2017
Rank 204 218 190
Rank change year-on-year +14 -28 +28
Contract jobs citing HDFS 26 42 57
As % of all contract IT jobs advertised in the City of London 0.41% 0.61% 0.85%
As % of the System Software category 2.39% 3.65% 7.61%
Number of daily rates quoted 19 37 51
City of London median daily rate £650 £525 £625
Median daily rate % change year-on-year +23.81% -16.00% -3.85%
10th Percentile £525 £510 £425
90th Percentile £700 £685 £725
Central London median daily rate £650 £525 £625
% change year-on-year +23.81% -16.00% +5.49%
Number of hourly rates quoted 0 0 0
City of London median hourly rate - - -
Central London median hourly rate - - -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in the City of London with a requirement for system software skills.

All System Software Skills
City of London
Contract vacancies with a requirement for system software skills 1,086 1,151 749
As % of all contract IT jobs advertised in the City of London 17.31% 16.58% 11.19%
Number of daily rates quoted 868 927 601
City of London median daily rate £513 £500 £488
Median daily rate % change year-on-year +2.50% +2.56% +8.33%
10th Percentile £213 £197 £198
90th Percentile £675 £650 £638
Central London median daily rate £525 £500 £475
% change year-on-year +5.00% +5.26% +5.56%
Number of hourly rates quoted 22 38 24
City of London median hourly rate £21.00 £19.25 £19.13
Median hourly rate % change year-on-year +9.09% +0.65% -1.92%
10th Percentile £13.33 £13.31 £13.01
90th Percentile £46.42 £25.50 £28.98
Central London median hourly rate £21.00 £19.63 £18.85
% change year-on-year +7.01% +4.14% -0.82%

HDFS
Job Vacancy Trend in the City of London

Job postings citing HDFS as a percentage of all IT jobs advertised in the City of London.

Job vacancy trend for HDFS in the City of London

HDFS
Contractor Daily Rate Trend in the City of London

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing HDFS in the City of London.

Contractor daily rate trend for HDFS in the City of London

HDFS
Contractor Daily Rate Histogram in the City of London

The daily rate distribution of IT jobs citing HDFS in the City of London over the 6 months to 17 September 2019.

Contractor daily rate histogram for HDFS in the City of London

For the 6 months to 17 September 2019, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the City of London region with a requirement for HDFS.

1 26 (100.00%) Apache Spark
2 23 (88.46%) Hadoop
3 19 (73.08%) Apache Hive
3 19 (73.08%) Python
4 18 (69.23%) Java
5 17 (65.38%) HBase
6 15 (57.69%) Hadoop YARN
7 13 (50.00%) SQL
8 12 (46.15%) Finance
8 12 (46.15%) Agile Software Development
8 12 (46.15%) Scala
8 12 (46.15%) Performance Tuning
8 12 (46.15%) Impala
9 11 (42.31%) Big Data
10 10 (38.46%) Unix
11 9 (34.62%) TDD
11 9 (34.62%) Amazon S3
11 9 (34.62%) Spring
11 9 (34.62%) JIRA
11 9 (34.62%) PySpark
12 8 (30.77%) Data Science
12 8 (30.77%) Data Lake
12 8 (30.77%) Cloudera
12 8 (30.77%) Jenkins
12 8 (30.77%) DevOps
12 8 (30.77%) Data Modelling
13 7 (26.92%) OOP
13 7 (26.92%) Oracle
13 7 (26.92%) Shell Script
13 7 (26.92%) Computer Science

HDFS
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 26 (100.00%) Apache Spark
2 3 (11.54%) Jupyter
3 2 (7.69%) Confluence
4 1 (3.85%) Apache
4 1 (3.85%) Apache Solr
Cloud Services
1 9 (34.62%) Amazon S3
2 7 (26.92%) AWS
3 6 (23.08%) Amazon EMR
4 4 (15.38%) GCP
5 3 (11.54%) Cloud Computing
6 2 (7.69%) Amazon CloudWatch
6 2 (7.69%) Amazon Kinesis
6 2 (7.69%) AWS CloudFormation
6 2 (7.69%) AWS CloudTrail
6 2 (7.69%) AWS Lambda
6 2 (7.69%) Azure
6 2 (7.69%) Serverless
7 1 (3.85%) AWS Elastic Beanstalk
Database & Business Intelligence
1 23 (88.46%) Hadoop
2 19 (73.08%) Apache Hive
3 17 (65.38%) HBase
4 12 (46.15%) Impala
5 11 (42.31%) Big Data
6 8 (30.77%) Data Lake
7 5 (19.23%) Data Warehouse
7 5 (19.23%) Parquet
7 5 (19.23%) SQL Server
7 5 (19.23%) Sqoop
8 4 (15.38%) MongoDB
8 4 (15.38%) MySQL
8 4 (15.38%) NoSQL
9 3 (11.54%) OLAP
9 3 (11.54%) OLTP
9 3 (11.54%) Oracle OLAP
10 2 (7.69%) Amazon Athena
10 2 (7.69%) Apache Zeppelin
10 2 (7.69%) Presto
11 1 (3.85%) Cloudera CDH
Development Applications
1 9 (34.62%) JIRA
2 8 (30.77%) Jenkins
3 7 (26.92%) Travis CI
4 3 (11.54%) Git (software)
5 2 (7.69%) Atlassian Bamboo
General
1 12 (46.15%) Finance
2 7 (26.92%) Retail Banking
3 5 (19.23%) Banking
4 3 (11.54%) Retail
Job Titles
1 9 (34.62%) Data Engineer
2 8 (30.77%) Developer
3 7 (26.92%) Hadoop Developer
3 7 (26.92%) Java Developer
3 7 (26.92%) Java Hadoop Developer
4 4 (15.38%) Architect
5 3 (11.54%) Administrator
5 3 (11.54%) Data Architect
5 3 (11.54%) Hadoop Administrator
6 2 (7.69%) AWS Engineer
6 2 (7.69%) Cloud Engineer
7 1 (3.85%) Big Data Engineer
7 1 (3.85%) Cloud Architect
7 1 (3.85%) Systems Architect
Libraries, Frameworks & Software Standards
1 9 (34.62%) PySpark
1 9 (34.62%) Spring
2 7 (26.92%) Kafka
3 4 (15.38%) Web Services
4 3 (11.54%) LAMP
5 2 (7.69%) Apache Avro
5 2 (7.69%) JSON
5 2 (7.69%) Spark Streaming
5 2 (7.69%) Spring Boot
5 2 (7.69%) XML
6 1 (3.85%) Anaconda
6 1 (3.85%) Apache NiFi
6 1 (3.85%) Apache Tez
6 1 (3.85%) Play Framework
6 1 (3.85%) Regular Expression
Miscellaneous
1 7 (26.92%) Linux Command Line
2 3 (11.54%) Algorithms
2 3 (11.54%) Data Structures
3 2 (7.69%) Cloud Native
Operating Systems
1 10 (38.46%) Unix
2 5 (19.23%) Linux
3 1 (3.85%) Red Hat Enterprise Linux
3 1 (3.85%) Windows
Processes & Methodologies
1 12 (46.15%) Agile Software Development
1 12 (46.15%) Performance Tuning
2 9 (34.62%) TDD
3 8 (30.77%) Data Modelling
3 8 (30.77%) Data Science
3 8 (30.77%) DevOps
4 7 (26.92%) Computer Science
4 7 (26.92%) Functional Programming
4 7 (26.92%) OOP
4 7 (26.92%) Performance Optimisation
4 7 (26.92%) Performance Testing
5 6 (23.08%) Data Ingestion
5 6 (23.08%) ETL
6 3 (11.54%) Data Architecture
6 3 (11.54%) Data Cleansing
6 3 (11.54%) Data Enrichment
6 3 (11.54%) ETL Development
6 3 (11.54%) Logical Data Model
6 3 (11.54%) Physical Data Model
6 3 (11.54%) Single Customer View
Programming Languages
1 19 (73.08%) Python
2 18 (69.23%) Java
3 13 (50.00%) SQL
4 12 (46.15%) Scala
5 7 (26.92%) R
5 7 (26.92%) Shell Script
Qualifications
1 1 (3.85%) SC Cleared
1 1 (3.85%) Security Cleared
Quality Assurance & Compliance
1 1 (3.85%) GDPR
System Software
1 3 (11.54%) Apache Flume
2 1 (3.85%) Apache ZooKeeper
2 1 (3.85%) Docker
Systems Management
1 15 (57.69%) Hadoop YARN
2 2 (7.69%) Oozie
3 1 (3.85%) Ansible
Vendors
1 8 (30.77%) Cloudera
2 7 (26.92%) Oracle
3 5 (19.23%) Google
4 3 (11.54%) Informatica
4 3 (11.54%) Talend
4 3 (11.54%) Teradata
5 2 (7.69%) Tableau
6 1 (3.85%) Datameer