Period
to 24 May 2020

The following table provides summary statistics for contract job vacancies advertised in London with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 24 May 2020 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
England > London
6 months to
24 May 2020
Same period 2019 Same period 2018
Rank 381 440 437
Rank change year-on-year +59 -3 -34
Contract jobs citing HDFS 45 151 211
As % of all contract jobs advertised in London 0.23% 0.37% 0.46%
As % of the System Software category 1.52% 2.36% 3.40%
Number of daily rates quoted 31 112 147
Median daily rate £600 £600 £575
Median daily rate % change year-on-year - +4.35% -
10th Percentile £494 £513 £463
90th Percentile £700 £715 £675
England median daily rate £575 £600 £575
% change year-on-year -4.17% +4.35% +8.24%
Number of hourly rates quoted 0 0 0
Median hourly rate - - -
England median hourly rate £35.00 - -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in London with a requirement for system software skills.

All System Software Skills
London
Contract vacancies with a requirement for system software skills 2,962 6,394 6,212
As % of all contract IT jobs advertised in London 14.90% 15.77% 13.62%
Number of daily rates quoted 2,103 4,642 4,580
Median daily rate £513 £525 £500
Median daily rate % change year-on-year -2.38% +5.00% +6.95%
10th Percentile £213 £213 £203
90th Percentile £675 £675 £650
England median daily rate £475 £475 £450
% change year-on-year - +5.56% +5.88%
Number of hourly rates quoted 76 109 160
Median hourly rate £25.74 £21.00 £20.25
Median hourly rate % change year-on-year +22.57% +3.70% -2.41%
10th Percentile £12.88 £13.00 £12.03
90th Percentile £44.00 £47.09 £47.83
Median hourly rate £18.50 £16.60 £16.19
% change year-on-year +11.45% +2.53% +1.19%

HDFS
Job Vacancy Trend in London

Job postings citing HDFS as a proportion of all IT jobs advertised in London.

Job vacancy trend for HDFS in London

HDFS
Contractor Daily Rate Trend in London

3-month moving average daily rate quoted in jobs citing HDFS in London.

Daily rate trend for HDFS in London

HDFS
Daily Rate Histogram in London

Daily rate distribution for jobs citing HDFS in London over the 6 months to 24 May 2020.

Daily rate histogram for HDFS in London

HDFS
Contract Job Locations in London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the London region over the 6 months to 24 May 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +89 9 £600 -7.69%
East London +48 1 £700 - 1
West London +38 1 £575 +9.52%
HDFS
England

For the 6 months to 24 May 2020, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the London region with a requirement for HDFS.

1 44 (97.78%) Hadoop
2 42 (93.33%) Apache Spark
3 35 (77.78%) HBase
4 28 (62.22%) Big Data
4 28 (62.22%) Apache Hive
4 28 (62.22%) Python
5 25 (55.56%) Java
6 23 (51.11%) Kafka
7 22 (48.89%) Hadoop YARN
7 22 (48.89%) Agile Software Development
8 20 (44.44%) Cloudera
9 17 (37.78%) SQL
9 17 (37.78%) Scala
10 16 (35.56%) Spark Streaming
10 16 (35.56%) DevOps
10 16 (35.56%) PySpark
11 13 (28.89%) Analytics
11 13 (28.89%) Sqoop
11 13 (28.89%) NoSQL
11 13 (28.89%) Linux
11 13 (28.89%) ETL
12 12 (26.67%) Apache Cassandra
12 12 (26.67%) MapReduce
12 12 (26.67%) Oozie
12 12 (26.67%) Performance Tuning
13 11 (24.44%) Impala
13 11 (24.44%) MongoDB
13 11 (24.44%) AWS
13 11 (24.44%) Data Management
14 10 (22.22%) TDD

HDFS
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 42 (93.33%) Apache Spark
2 5 (11.11%) Apache Pig
3 4 (8.89%) Apache Airflow
3 4 (8.89%) Jupyter
4 3 (6.67%) Apache Pulsar
4 3 (6.67%) Oracle Workflow
5 2 (4.44%) SAS
6 1 (2.22%) Apache
6 1 (2.22%) Apache Solr
6 1 (2.22%) Confluence
6 1 (2.22%) Luigi
6 1 (2.22%) SharePoint
Applications
1 1 (2.22%) Chatbot
1 1 (2.22%) MS Visio
Cloud Services
1 11 (24.44%) AWS
2 6 (13.33%) Amazon Kinesis
2 6 (13.33%) Azure
2 6 (13.33%) GCP
3 4 (8.89%) IaaS
3 4 (8.89%) PaaS
3 4 (8.89%) Serverless
4 3 (6.67%) Amazon EC2
4 3 (6.67%) Amazon S3
4 3 (6.67%) Virtual Private Cloud
5 2 (4.44%) Amazon CloudWatch
5 2 (4.44%) AWS CloudTrail
5 2 (4.44%) AWS Lambda
6 1 (2.22%) Amazon SQS
6 1 (2.22%) AWS CloudFormation
Communications & Networking
1 8 (17.78%) Kerberos
2 6 (13.33%) VPN
3 5 (11.11%) SSL
4 4 (8.89%) HTTP
4 4 (8.89%) HTTPS
5 3 (6.67%) Firewall
6 2 (4.44%) TCP/IP
Database & Business Intelligence
1 44 (97.78%) Hadoop
2 35 (77.78%) HBase
3 28 (62.22%) Apache Hive
3 28 (62.22%) Big Data
4 13 (28.89%) NoSQL
4 13 (28.89%) Sqoop
5 12 (26.67%) Apache Cassandra
6 11 (24.44%) Impala
6 11 (24.44%) MongoDB
7 6 (13.33%) Data Warehouse
8 3 (6.67%) Amazon Aurora
8 3 (6.67%) Amazon RDS
8 3 (6.67%) DB2
8 3 (6.67%) MemSQL
8 3 (6.67%) Presto
8 3 (6.67%) Relational Database
9 2 (4.44%) Amazon Athena
9 2 (4.44%) Amazon DynamoDB
9 2 (4.44%) Spark SQL
10 1 (2.22%) Amazon Redshift
Development Applications
1 8 (17.78%) Jenkins
2 6 (13.33%) JIRA
3 5 (11.11%) Git (software)
3 5 (11.11%) Travis CI
4 4 (8.89%) Artifactory
4 4 (8.89%) GitLab
5 2 (4.44%) Atlassian Bamboo
5 2 (4.44%) TeamCity
6 1 (2.22%) Bitbucket
6 1 (2.22%) Cucumber
6 1 (2.22%) JUnit
General
1 8 (17.78%) Finance
2 5 (11.11%) Retail Banking
3 4 (8.89%) Public Sector
4 3 (6.67%) Banking
5 2 (4.44%) Investment Banking
6 1 (2.22%) Front Office
6 1 (2.22%) Legal
6 1 (2.22%) Manufacturing
6 1 (2.22%) Pharmaceutical
Job Titles
1 16 (35.56%) Data Engineer
2 12 (26.67%) Developer
3 9 (20.00%) DevOps Engineer
4 8 (17.78%) Architect
4 8 (17.78%) Hadoop Architect
4 8 (17.78%) Hadoop Developer
5 7 (15.56%) Big Data Architect
5 7 (15.56%) Big Data Engineer
5 7 (15.56%) Data Architect
6 6 (13.33%) Java Developer
6 6 (13.33%) Java Hadoop Developer
7 5 (11.11%) Big Data Developer
7 5 (11.11%) Cloud Engineer
8 4 (8.89%) Infrastructure Architect
8 4 (8.89%) Senior DevOps
8 4 (8.89%) Senior DevOps Engineer
9 3 (6.67%) Java Engineer
9 3 (6.67%) Scala Developer
10 2 (4.44%) AWS DevOps
10 2 (4.44%) AWS Engineer
Libraries, Frameworks & Software Standards
1 23 (51.11%) Kafka
2 16 (35.56%) PySpark
2 16 (35.56%) Spark Streaming
3 7 (15.56%) LDAP
4 6 (13.33%) OAuth
4 6 (13.33%) SAML
5 5 (11.11%) ADO
5 5 (11.11%) Elastic Stack
5 5 (11.11%) RESTful
6 4 (8.89%) Apache NiFi
7 3 (6.67%) Spring
8 2 (4.44%) Web Services
9 1 (2.22%) CSS
9 1 (2.22%) DOM
9 1 (2.22%) HTML
9 1 (2.22%) Natural Language Processing
9 1 (2.22%) RabbitMQ
9 1 (2.22%) SAX
9 1 (2.22%) XML
Miscellaneous
1 5 (11.11%) Data Centre
1 5 (11.11%) Linux Command Line
2 4 (8.89%) Analytical Skills
2 4 (8.89%) Enterprise Software
2 4 (8.89%) PMI
3 2 (4.44%) Algorithms
3 2 (4.44%) Credit Risk
3 2 (4.44%) Market Risk
3 2 (4.44%) User Experience
4 1 (2.22%) Analytical Mindset
4 1 (2.22%) Self-Motivation
4 1 (2.22%) Wiki
Operating Systems
1 13 (28.89%) Linux
2 8 (17.78%) Unix
Processes & Methodologies
1 22 (48.89%) Agile Software Development
2 16 (35.56%) DevOps
3 13 (28.89%) Analytics
3 13 (28.89%) ETL
4 12 (26.67%) MapReduce
4 12 (26.67%) Performance Tuning
5 11 (24.44%) Data Management
6 10 (22.22%) Business Intelligence
6 10 (22.22%) TDD
7 9 (20.00%) Scrum
7 9 (20.00%) Use Case
8 8 (17.78%) Computer Science
8 8 (17.78%) Microservices
9 7 (15.56%) Workflow
10 6 (13.33%) Functional Programming
10 6 (13.33%) Software Engineering
11 5 (11.11%) Data Analytics
11 5 (11.11%) Data Engineering
11 5 (11.11%) Performance Optimisation
11 5 (11.11%) Performance Testing
Programming Languages
1 28 (62.22%) Python
2 25 (55.56%) Java
3 17 (37.78%) Scala
3 17 (37.78%) SQL
4 8 (17.78%) Bash Shell
4 8 (17.78%) Ruby
5 7 (15.56%) R
6 3 (6.67%) Go
6 3 (6.67%) Shell Script
7 1 (2.22%) PL/SQL
7 1 (2.22%) PowerShell
Qualifications
1 4 (8.89%) PMI Certification
2 3 (6.67%) Computer Science Degree
3 2 (4.44%) Degree
4 1 (2.22%) Numerate Degree
Quality Assurance & Compliance
1 3 (6.67%) Data Quality
2 1 (2.22%) ISO/IEC 27001
System Software
1 7 (15.56%) Apache Flume
2 4 (8.89%) Docker
3 1 (2.22%) Apache ZooKeeper
Systems Management
1 22 (48.89%) Hadoop YARN
2 12 (26.67%) Oozie
3 7 (15.56%) Ambari
4 6 (13.33%) Single Sign-On
5 5 (11.11%) Ansible
5 5 (11.11%) Puppet
5 5 (11.11%) Terraform
6 3 (6.67%) Kubernetes
7 1 (2.22%) Docker Swarm
7 1 (2.22%) Opscode Chef
Vendors
1 20 (44.44%) Cloudera
2 8 (17.78%) Hortonworks
3 6 (13.33%) Ab Initio
3 6 (13.33%) MapR
4 5 (11.11%) Google
5 4 (8.89%) Splunk
6 3 (6.67%) Oracle
6 3 (6.67%) Sybase
7 2 (4.44%) New Relic
8 1 (2.22%) Foundry
8 1 (2.22%) Palantir
8 1 (2.22%) StreamSets