Period
to 9 December 2019

The following table provides summary statistics for contract job vacancies advertised in London with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 9 December 2019 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
England > London
6 months to
9 Dec 2019
Same period 2018 Same period 2017
Rank 456 444 341
Rank change year-on-year -12 -103 +80
Contract jobs citing HDFS 87 196 334
As % of all contract IT jobs advertised in London 0.25% 0.42% 0.71%
As % of the System Software category 1.74% 2.71% 5.71%
Number of daily rates quoted 53 149 251
London median daily rate £550 £550 £575
Median daily rate % change year-on-year - -4.35% -
10th Percentile £421 £461 £406
90th Percentile £700 £725 £719
England median daily rate £545 £550 £550
% change year-on-year -0.91% - -
Number of hourly rates quoted 0 0 0
London median hourly rate - - -
England median hourly rate - - -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in London with a requirement for system software skills.

All System Software Skills
London
Contract vacancies with a requirement for system software skills 5,013 7,232 5,848
As % of all contract IT jobs advertised in London 14.58% 15.51% 12.41%
Number of daily rates quoted 3,639 5,316 4,218
London median daily rate £500 £525 £488
Median daily rate % change year-on-year -4.76% +7.69% +8.33%
10th Percentile £185 £200 £185
90th Percentile £663 £656 £638
England median daily rate £450 £463 £425
% change year-on-year -2.70% +8.82% +3.03%
Number of hourly rates quoted 123 141 141
London median hourly rate £19.00 £19.25 £20.50
Median hourly rate % change year-on-year -1.30% -6.10% +24.24%
10th Percentile £12.44 £12.75 £12.00
90th Percentile £50.00 £47.36 £57.50
England median hourly rate £16.50 £16.00 £16.50
% change year-on-year +3.13% -3.03% +3.13%

HDFS
Job Vacancy Trend in London

Job postings citing HDFS as a percentage of all IT jobs advertised in London.

Job vacancy trend for HDFS in London

HDFS
Contractor Daily Rate Trend in London

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing HDFS in London.

Contractor daily rate trend for HDFS in London

HDFS
Contractor Daily Rate Histogram in London

The daily rate distribution of IT jobs citing HDFS in London over the 6 months to 9 December 2019.

Contractor daily rate histogram for HDFS in London

HDFS
Contract Job Locations in London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the London region over the 6 months to 9 December 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +31 17 £656 +14.13% 2
East London +26 3 £507 +12.67%
HDFS
England

For the 6 months to 9 December 2019, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the London region with a requirement for HDFS.

1 79 (90.80%) Hadoop
2 67 (77.01%) Apache Spark
3 58 (66.67%) Python
4 49 (56.32%) Apache Hive
5 44 (50.57%) SQL
6 43 (49.43%) Java
7 42 (48.28%) Scala
8 40 (45.98%) Big Data
9 37 (42.53%) Hadoop YARN
10 35 (40.23%) Kafka
10 35 (40.23%) Agile Software Development
11 30 (34.48%) HBase
12 29 (33.33%) Impala
13 26 (29.89%) ETL
13 26 (29.89%) DevOps
13 26 (29.89%) AWS
14 24 (27.59%) R
15 23 (26.44%) Azure
16 22 (25.29%) Unix
16 22 (25.29%) PySpark
17 21 (24.14%) Finance
18 20 (22.99%) Data Modelling
18 20 (22.99%) Oracle
18 20 (22.99%) Cloudera
19 17 (19.54%) Analytics
19 17 (19.54%) Performance Tuning
19 17 (19.54%) SQL Server
19 17 (19.54%) Continuous Integration
20 16 (18.39%) Visualisation
20 16 (18.39%) Data Lake

HDFS
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 67 (77.01%) Apache Spark
2 10 (11.49%) Jupyter
3 6 (6.90%) Apache
4 5 (5.75%) Apache Solr
5 4 (4.60%) Confluence
6 3 (3.45%) Elasticsearch
7 2 (2.30%) Apache Airflow
7 2 (2.30%) SAS
7 2 (2.30%) Tornado
8 1 (1.15%) OpenStack
Applications
1 6 (6.90%) Microsoft Excel
Business Applications
1 1 (1.15%) SAP MDM
Cloud Services
1 26 (29.89%) AWS
2 23 (26.44%) Azure
3 15 (17.24%) Amazon Kinesis
4 12 (13.79%) AWS Lambda
4 12 (13.79%) Serverless
5 11 (12.64%) GCP
6 10 (11.49%) Amazon EMR
7 9 (10.34%) Amazon S3
8 8 (9.20%) Power Platform
9 2 (2.30%) Amazon CloudWatch
9 2 (2.30%) AWS CloudFormation
9 2 (2.30%) AWS CloudTrail
9 2 (2.30%) AWS Step Functions
9 2 (2.30%) GitHub
10 1 (1.15%) OpenShift
Communications & Networking
1 3 (3.45%) Kerberos
2 1 (1.15%) SSL
2 1 (1.15%) VPN
Database & Business Intelligence
1 79 (90.80%) Hadoop
2 49 (56.32%) Apache Hive
3 40 (45.98%) Big Data
4 30 (34.48%) HBase
5 29 (33.33%) Impala
6 17 (19.54%) SQL Server
7 16 (18.39%) Data Lake
8 14 (16.09%) NoSQL
9 12 (13.79%) Data Warehouse
9 12 (13.79%) Spark SQL
10 11 (12.64%) Sqoop
11 10 (11.49%) MongoDB
12 8 (9.20%) Parquet
12 8 (9.20%) Power BI
13 6 (6.90%) Cloudera CDH
13 6 (6.90%) Flink
13 6 (6.90%) Neo4j
14 5 (5.75%) Apache Cassandra
14 5 (5.75%) Oracle Database
14 5 (5.75%) Relational Database
Development Applications
1 12 (13.79%) JIRA
2 10 (11.49%) Jenkins
3 8 (9.20%) Travis CI
4 6 (6.90%) Git (software)
4 6 (6.90%) Jasmine
5 3 (3.45%) Atlassian Bamboo
5 3 (3.45%) kappa
6 1 (1.15%) Bitbucket
6 1 (1.15%) Bitbucket Server
6 1 (1.15%) ER Studio
6 1 (1.15%) ERwin
6 1 (1.15%) FishEye
6 1 (1.15%) JUnit
General
1 21 (24.14%) Finance
2 10 (11.49%) Retail Banking
3 5 (5.75%) Banking
4 3 (3.45%) Investment Banking
4 3 (3.45%) Telecoms
5 2 (2.30%) Retail
6 1 (1.15%) Games
Job Titles
1 27 (31.03%) Developer
2 22 (25.29%) Data Engineer
3 20 (22.99%) Architect
4 14 (16.09%) Hadoop Developer
5 12 (13.79%) Data Architect
6 10 (11.49%) Big Data Engineer
6 10 (11.49%) Hadoop Architect
7 9 (10.34%) Java Developer
7 9 (10.34%) Java Hadoop Developer
8 8 (9.20%) Python Developer
9 7 (8.05%) Administrator
10 5 (5.75%) Big Data Architect
10 5 (5.75%) Software Engineer
11 4 (4.60%) Cloud Architect
11 4 (4.60%) Python Engineer
11 4 (4.60%) Scala Developer
12 3 (3.45%) AWS Engineer
12 3 (3.45%) Big Data Developer
12 3 (3.45%) ETL Developer
12 3 (3.45%) Java Engineer
Libraries, Frameworks & Software Standards
1 35 (40.23%) Kafka
2 22 (25.29%) PySpark
3 8 (9.20%) Pandas
4 7 (8.05%) Flask
4 7 (8.05%) React
4 7 (8.05%) Spring
5 6 (6.90%) RabbitMQ
6 5 (5.75%) Web Services
7 4 (4.60%) HTML
7 4 (4.60%) Natural Language Processing
7 4 (4.60%) XML
8 3 (3.45%) CSS
8 3 (3.45%) JSON
8 3 (3.45%) Spark Streaming
9 2 (2.30%) Apache NiFi
9 2 (2.30%) Highcharts JS
9 2 (2.30%) REST
10 1 (1.15%) Play Framework
10 1 (1.15%) RESTful
10 1 (1.15%) TensorFlow
Miscellaneous
1 10 (11.49%) Algorithms
2 8 (9.20%) Analytical Skills
2 8 (9.20%) Linux Command Line
3 2 (2.30%) Cloud Native
3 2 (2.30%) Greenfield Project
4 1 (1.15%) Data Structures
4 1 (1.15%) Derivative
4 1 (1.15%) Equities
4 1 (1.15%) Public Cloud
4 1 (1.15%) Smart Energy
Operating Systems
1 22 (25.29%) Unix
2 14 (16.09%) Linux
3 9 (10.34%) Windows
4 6 (6.90%) Ubuntu
5 1 (1.15%) Android
5 1 (1.15%) Apple iOS
5 1 (1.15%) Red Hat Enterprise Linux
Processes & Methodologies
1 35 (40.23%) Agile Software Development
2 26 (29.89%) DevOps
2 26 (29.89%) ETL
3 20 (22.99%) Data Modelling
4 17 (19.54%) Analytics
4 17 (19.54%) Continuous Integration
4 17 (19.54%) Performance Tuning
5 16 (18.39%) Visualisation
6 14 (16.09%) Data Engineering
6 14 (16.09%) Migration
6 14 (16.09%) OOP
7 13 (14.94%) Computer Science
7 13 (14.94%) Data Visualisation
7 13 (14.94%) Performance Testing
8 12 (13.79%) Business Intelligence
8 12 (13.79%) TDD
9 11 (12.64%) Data Architecture
9 11 (12.64%) Problem-Solving
10 10 (11.49%) Software Engineering
11 9 (10.34%) Use Case
Programming Languages
1 58 (66.67%) Python
2 44 (50.57%) SQL
3 43 (49.43%) Java
4 42 (48.28%) Scala
5 24 (27.59%) R
6 10 (11.49%) JavaScript
7 8 (9.20%) Go
7 8 (9.20%) Kotlin
8 6 (6.90%) C#
8 6 (6.90%) PL/SQL
8 6 (6.90%) Shell Script
9 4 (4.60%) Bash Shell
10 2 (2.30%) HiveQL
11 1 (1.15%) Java 8
11 1 (1.15%) Perl
Qualifications
1 6 (6.90%) Security Cleared
2 5 (5.75%) SC Cleared
3 1 (1.15%) DV Cleared
3 1 (1.15%) Microsoft Certification
Quality Assurance & Compliance
1 2 (2.30%) Data Quality
2 1 (1.15%) QA
System Software
1 5 (5.75%) Apache ZooKeeper
2 1 (1.15%) Active Directory
2 1 (1.15%) Apache Flume
2 1 (1.15%) VMware ESXi
2 1 (1.15%) vSphere
Systems Management
1 37 (42.53%) Hadoop YARN
2 8 (9.20%) Kubernetes
3 7 (8.05%) Oozie
4 6 (6.90%) Grafana
5 5 (5.75%) Ansible
6 1 (1.15%) Opscode Chef
6 1 (1.15%) Puppet
Vendors
1 20 (22.99%) Cloudera
1 20 (22.99%) Oracle
2 14 (16.09%) Tableau
3 12 (13.79%) Microsoft
4 9 (10.34%) Hortonworks
5 8 (9.20%) Google
6 7 (8.05%) Foundry
6 7 (8.05%) Palantir
6 7 (8.05%) Teradata
7 6 (6.90%) Splunk
8 4 (4.60%) Talend
9 2 (2.30%) Ab Initio
9 2 (2.30%) Atlassian
9 2 (2.30%) Sybase
10 1 (1.15%) Informatica
10 1 (1.15%) MapR
10 1 (1.15%) NVIDIA
10 1 (1.15%) Red Hat
10 1 (1.15%) SAP
10 1 (1.15%) VMware