Period
to 20 September 2021

The following table provides summary statistics for contract job vacancies advertised in London with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 20 September 2021 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
England > London
6 months to
20 Sep 2021
Same period 2020 Same period 2019
Rank 483 333 485
Rank change year-on-year -150 +152 -60
Contract jobs citing HDFS 59 19 112
As % of all contract jobs advertised in London 0.18% 0.14% 0.29%
As % of the System Software category 1.29% 1.01% 1.89%
Number of daily rates quoted 38 10 74
10th Percentile £339 £519 £441
25th Percentile £506 £531 £521
Median daily rate (50th Percentile) £600 £550 £575
Median % change year-on-year +9.09% -4.35% +9.52%
75th Percentile £675 £643 £673
90th Percentile £699 £666 £700
England median daily rate £550 £450 £575
% change year-on-year +22.22% -21.74% +9.52%

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in London with a requirement for system software skills.

All System Software Skills
London
Contract vacancies with a requirement for system software skills 4,576 1,885 5,929
As % of all contract IT jobs advertised in London 13.71% 14.36% 15.27%
Number of daily rates quoted 3,398 1,351 4,396
10th Percentile £344 £295 £200
25th Percentile £455 £413 £400
Median daily rate (50th Percentile) £550 £500 £513
Median % change year-on-year +10.00% -2.44% +2.50%
75th Percentile £638 £600 £600
90th Percentile £700 £675 £675
England median daily rate £500 £485 £475
% change year-on-year +3.20% +2.00% +5.56%
Number of hourly rates quoted 38 22 100
10th Percentile £15.00 £12.76 £12.76
25th Percentile £20.50 £15.78 £16.08
Median hourly rate £25.00 £23.75 £21.00
Median % change year-on-year +5.26% +13.10% +9.80%
75th Percentile £45.89 £43.00 £35.26
90th Percentile £53.92 £47.77 £50.00
England median hourly rate £17.00 £17.75 £16.00
% change year-on-year -4.23% +10.94% -

HDFS
Job Vacancy Trend in London

Job postings citing HDFS as a proportion of all IT jobs advertised in London.

Job vacancy trend for HDFS in London

HDFS
Contractor Daily Rate Trend in London

3-month moving average daily rate quoted in jobs citing HDFS in London.

Daily rate trend for HDFS in London

HDFS
Daily Rate Histogram in London

Daily rate distribution for jobs citing HDFS in London over the 6 months to 20 September 2021.

Daily rate histogram for HDFS in London

HDFS
Contract Job Locations in London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the London region over the 6 months to 20 September 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
South London - 5 £575 - 1
Central London -56 3 £500 -9.09% 2
HDFS
England

For the 6 months to 20 September 2021, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the London region with a requirement for HDFS.

1 55 (93.22%) Apache Spark
2 51 (86.44%) Hadoop
3 42 (71.19%) Apache Hive
4 38 (64.41%) Big Data
5 31 (52.54%) Kafka
6 30 (50.85%) HBase
7 24 (40.68%) Java
8 22 (37.29%) Kubernetes
9 21 (35.59%) DevOps
9 21 (35.59%) Linux
10 20 (33.90%) Banking
10 20 (33.90%) AWS
10 20 (33.90%) Analytics
11 19 (32.20%) Python
12 17 (28.81%) Git (software)
13 16 (27.12%) Terraform
14 15 (25.42%) Jenkins
14 15 (25.42%) ETL
14 15 (25.42%) SQL
14 15 (25.42%) Scala
15 14 (23.73%) Apache Cassandra
16 13 (22.03%) Use Case
16 13 (22.03%) Data Warehouse
16 13 (22.03%) GCP
16 13 (22.03%) Social Skills
17 12 (20.34%) Microservices
17 12 (20.34%) Docker
17 12 (20.34%) NoSQL
17 12 (20.34%) MongoDB
17 12 (20.34%) Data Analytics

HDFS
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 55 (93.22%) Apache Spark
2 3 (5.08%) Apache
3 2 (3.39%) Apache Pig
4 1 (1.69%) Apache Airflow
4 1 (1.69%) Confluence
4 1 (1.69%) PowerExchange
Cloud Services
1 20 (33.90%) AWS
2 13 (22.03%) GCP
3 11 (18.64%) Amazon S3
3 11 (18.64%) Azure
4 6 (10.17%) Azure Data Factory
4 6 (10.17%) Azure Synapse Analytics
5 4 (6.78%) Amazon EMR
5 4 (6.78%) OpenShift
6 3 (5.08%) Amazon Kinesis
6 3 (5.08%) Amazon SQS
6 3 (5.08%) GitHub
7 2 (3.39%) Google Compute Engine
8 1 (1.69%) Amazon CloudWatch
8 1 (1.69%) Amazon EC2
8 1 (1.69%) Power Platform
8 1 (1.69%) Snowflake
Communications & Networking
1 6 (10.17%) FTP
2 3 (5.08%) HTTP
Database & Business Intelligence
1 51 (86.44%) Hadoop
2 42 (71.19%) Apache Hive
3 38 (64.41%) Big Data
4 30 (50.85%) HBase
5 14 (23.73%) Apache Cassandra
6 13 (22.03%) Data Warehouse
7 12 (20.34%) MongoDB
7 12 (20.34%) NoSQL
8 10 (16.95%) Data Lake
9 7 (11.86%) PostgreSQL
9 7 (11.86%) Relational Database
10 6 (10.17%) Azure SQL Database
10 6 (10.17%) Metadata
10 6 (10.17%) RDBMS
10 6 (10.17%) SAP HANA
10 6 (10.17%) SQL Server
11 5 (8.47%) BigQuery
12 4 (6.78%) Spark SQL
12 4 (6.78%) Sqoop
13 3 (5.08%) Impala
Development Applications
1 17 (28.81%) Git (software)
2 15 (25.42%) Jenkins
3 6 (10.17%) ER Studio
3 6 (10.17%) ERwin
3 6 (10.17%) Maven
3 6 (10.17%) Subversion
4 4 (6.78%) TeamCity
5 3 (5.08%) Artifactory
5 3 (5.08%) Bitbucket
5 3 (5.08%) Eclipse
5 3 (5.08%) IntelliJ
5 3 (5.08%) JUnit
5 3 (5.08%) Mockito
5 3 (5.08%) Sonatype Nexus
6 2 (3.39%) IBM UrbanCode
6 2 (3.39%) JIRA
General
1 20 (33.90%) Banking
2 13 (22.03%) Social Skills
3 10 (16.95%) Investment Banking
4 9 (15.25%) Finance
5 2 (3.39%) Analytical Skills
Job Titles
1 16 (27.12%) Developer
2 15 (25.42%) Data Engineer
3 10 (16.95%) Big Data Manager
3 10 (16.95%) Data Manager
3 10 (16.95%) Platform Manager
4 8 (13.56%) DevOps Engineer
5 6 (10.17%) Azure Developer
6 5 (8.47%) Big Data Engineer
7 4 (6.78%) Hadoop Engineer
7 4 (6.78%) Kubernetes Engineer
8 3 (5.08%) Java Developer
8 3 (5.08%) Java Engineer
8 3 (5.08%) Platform Engineer
8 3 (5.08%) Scala Developer
9 2 (3.39%) Architect
9 2 (3.39%) Data Architect
9 2 (3.39%) Hadoop Developer
9 2 (3.39%) Real Time Engineer
9 2 (3.39%) Senior Developer
9 2 (3.39%) Software Engineer
Libraries, Frameworks & Software Standards
1 31 (52.54%) Kafka
2 10 (16.95%) Spark Streaming
2 10 (16.95%) Spring
3 6 (10.17%) YAML
4 5 (8.47%) Spring Boot
5 4 (6.78%) Akka
5 4 (6.78%) PySpark
6 3 (5.08%) RESTful
6 3 (5.08%) WireMock
7 2 (3.39%) JSON
7 2 (3.39%) Node.js
8 1 (1.69%) Apache NiFi
8 1 (1.69%) Matplotlib
8 1 (1.69%) NumPy
8 1 (1.69%) Pandas
8 1 (1.69%) PyArrow
8 1 (1.69%) REST
8 1 (1.69%) scikit-learn
Miscellaneous
1 2 (3.39%) Cloud Native
1 2 (3.39%) Data Structures
2 1 (1.69%) Equities
2 1 (1.69%) Foreign Exchange (FX)
2 1 (1.69%) Reference Data
2 1 (1.69%) Virtual Team
Operating Systems
1 21 (35.59%) Linux
2 6 (10.17%) Unix
Processes & Methodologies
1 21 (35.59%) DevOps
2 20 (33.90%) Analytics
3 15 (25.42%) ETL
4 13 (22.03%) Use Case
5 12 (20.34%) Data Analytics
5 12 (20.34%) Microservices
6 11 (18.64%) Performance Tuning
7 10 (16.95%) Business Intelligence
7 10 (16.95%) Data Management
8 9 (15.25%) Data Modelling
8 9 (15.25%) Problem-Solving
9 8 (13.56%) Computer Science
9 8 (13.56%) Continuous Improvement
10 7 (11.86%) Infrastructure as Code
11 6 (10.17%) Configuration Management
11 6 (10.17%) Market Research
11 6 (10.17%) Metadata Management
11 6 (10.17%) Scrum
12 5 (8.47%) MapReduce
13 4 (6.78%) Test Automation
Programming Languages
1 24 (40.68%) Java
2 19 (32.20%) Python
3 15 (25.42%) Scala
3 15 (25.42%) SQL
4 5 (8.47%) Groovy
4 5 (8.47%) Shell Script
5 4 (6.78%) Go
5 4 (6.78%) Ruby
6 3 (5.08%) C
7 1 (1.69%) Bash Shell
7 1 (1.69%) C#
7 1 (1.69%) Java 8
Qualifications
1 6 (10.17%) Security Cleared
2 5 (8.47%) Computer Science Degree
2 5 (8.47%) Degree
3 2 (3.39%) SC Cleared
4 1 (1.69%) CCA
Quality Assurance & Compliance
1 1 (1.69%) Data Quality
1 1 (1.69%) QA
System Software
1 12 (20.34%) Docker
2 1 (1.69%) Apache Flume
Systems Management
1 22 (37.29%) Kubernetes
2 16 (27.12%) Terraform
3 9 (15.25%) Ansible
4 7 (11.86%) Hadoop YARN
5 6 (10.17%) Ambari
5 6 (10.17%) Puppet
6 4 (6.78%) Oozie
7 3 (5.08%) Istio
8 1 (1.69%) Control-M
Vendors
1 10 (16.95%) Ab Initio
1 10 (16.95%) Google
1 10 (16.95%) MapR
2 7 (11.86%) Databricks
3 6 (10.17%) Salesforce.com
3 6 (10.17%) SAP
4 4 (6.78%) Informatica
5 3 (5.08%) Microsoft
5 3 (5.08%) Oracle
6 2 (3.39%) Cloudera
6 2 (3.39%) Hortonworks
7 1 (1.69%) Alteryx
7 1 (1.69%) Dataiku
7 1 (1.69%) IBM
7 1 (1.69%) StreamSets
7 1 (1.69%) Teradata