Period
to 12 May 2021

The following table provides summary statistics for contract job vacancies advertised in London with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 12 May 2021 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
England > London
6 months to
12 May 2021
Same period 2020 Same period 2019
Rank 408 385 458
Rank change year-on-year -23 +73 -14
Contract jobs citing HDFS 60 48 156
As % of all contract jobs advertised in London 0.25% 0.23% 0.38%
As % of the System Software category 1.79% 1.53% 2.45%
Number of daily rates quoted 37 35 118
10th Percentile £432 £496 £500
25th Percentile £500 £525 £551
Median daily rate (50th Percentile) £575 £600 £600
Median % change year-on-year -4.17% - +4.35%
75th Percentile £613 £694 £675
90th Percentile £625 £740 £712
England median daily rate £540 £558 £575
% change year-on-year -3.14% -3.04% -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in London with a requirement for system software skills.

All System Software Skills
London
Contract vacancies with a requirement for system software skills 3,356 3,135 6,357
As % of all contract IT jobs advertised in London 14.24% 14.89% 15.57%
Number of daily rates quoted 2,504 2,306 4,640
10th Percentile £350 £213 £210
25th Percentile £444 £400 £413
Median daily rate (50th Percentile) £525 £500 £525
Median % change year-on-year +5.00% -4.76% +5.00%
75th Percentile £613 £600 £600
90th Percentile £700 £675 £675
England median daily rate £490 £474 £475
% change year-on-year +3.31% -0.14% +5.56%
Number of hourly rates quoted 20 68 109
10th Percentile £19.51 £12.42 £13.00
25th Percentile £36.50 £15.38 £16.08
Median hourly rate £48.50 £19.00 £20.33
Median % change year-on-year +155.26% -6.52% +1.63%
75th Percentile £61.25 £26.81 £36.67
90th Percentile £76.03 £47.83 £47.09
England median hourly rate £20.00 £16.90 £16.50
% change year-on-year +18.36% +2.41% +4.04%

HDFS
Job Vacancy Trend in London

Job postings citing HDFS as a proportion of all IT jobs advertised in London.

Job vacancy trend for HDFS in London

HDFS
Contractor Daily Rate Trend in London

3-month moving average daily rate quoted in jobs citing HDFS in London.

Daily rate trend for HDFS in London

HDFS
Daily Rate Histogram in London

Daily rate distribution for jobs citing HDFS in London over the 6 months to 12 May 2021.

Daily rate histogram for HDFS in London

HDFS
Contract Job Locations in London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the London region over the 6 months to 12 May 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +28 6 £443 -30.59% 1
East London +17 1 £405 -42.14%
West London +17 1 £500 -13.04% 1
HDFS
England

For the 6 months to 12 May 2021, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the London region with a requirement for HDFS.

1 58 (96.67%) Hadoop
2 52 (86.67%) Apache Spark
3 49 (81.67%) Apache Hive
4 37 (61.67%) Big Data
5 30 (50.00%) Python
6 29 (48.33%) Scala
7 27 (45.00%) Java
7 27 (45.00%) Kafka
7 27 (45.00%) HBase
8 25 (41.67%) SQL
9 23 (38.33%) ETL
10 22 (36.67%) Agile Software Development
11 21 (35.00%) Linux
12 20 (33.33%) NoSQL
13 19 (31.67%) Analytics
14 18 (30.00%) DevOps
14 18 (30.00%) Impala
14 18 (30.00%) AWS
15 17 (28.33%) Azure
16 16 (26.67%) Cloudera
16 16 (26.67%) Finance
17 15 (25.00%) Data Warehouse
17 15 (25.00%) Banking
17 15 (25.00%) Apache Cassandra
18 13 (21.67%) Data Management
19 11 (18.33%) Scrum
19 11 (18.33%) Oracle
19 11 (18.33%) Amazon S3
19 11 (18.33%) SC Cleared
19 11 (18.33%) Security Cleared

HDFS
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 52 (86.67%) Apache Spark
2 5 (8.33%) Apache
2 5 (8.33%) Apache Airflow
2 5 (8.33%) Apache Pig
2 5 (8.33%) Confluence
3 4 (6.67%) Apache Solr
4 3 (5.00%) SAS
Applications
1 3 (5.00%) Microsoft Excel
2 1 (1.67%) Spreadsheet
Cloud Services
1 18 (30.00%) AWS
2 17 (28.33%) Azure
3 11 (18.33%) Amazon S3
4 10 (16.67%) GCP
5 9 (15.00%) Amazon EMR
6 6 (10.00%) Power Platform
7 4 (6.67%) Amazon EC2
7 4 (6.67%) Amazon Kinesis
8 3 (5.00%) Amazon ElastiCache
8 3 (5.00%) AWS CloudFormation
9 2 (3.33%) Azure Storage
10 1 (1.67%) Amazon CloudWatch
10 1 (1.67%) Amazon SQS
10 1 (1.67%) AWS Lambda
10 1 (1.67%) Azure Cosmos DB
10 1 (1.67%) Azure Data Factory
10 1 (1.67%) Azure Synapse Analytics
10 1 (1.67%) SaaS
10 1 (1.67%) Serverless
Communications & Networking
1 3 (5.00%) Network Security
2 1 (1.67%) FTP
Database & Business Intelligence
1 58 (96.67%) Hadoop
2 49 (81.67%) Apache Hive
3 37 (61.67%) Big Data
4 27 (45.00%) HBase
5 20 (33.33%) NoSQL
6 18 (30.00%) Impala
7 15 (25.00%) Apache Cassandra
7 15 (25.00%) Data Warehouse
8 11 (18.33%) MongoDB
9 9 (15.00%) Relational Database
9 9 (15.00%) SQL Server
10 8 (13.33%) MySQL
10 8 (13.33%) Parquet
10 8 (13.33%) Spark SQL
11 7 (11.67%) Tableau
12 6 (10.00%) Power BI
13 5 (8.33%) Amazon Redshift
13 5 (8.33%) Data Lake
14 4 (6.67%) Amazon DynamoDB
14 4 (6.67%) Amazon RDS
Development Applications
1 9 (15.00%) Bitbucket
2 8 (13.33%) Git (software)
3 7 (11.67%) Jenkins
4 6 (10.00%) Artifactory
4 6 (10.00%) JIRA
5 3 (5.00%) Maven
6 1 (1.67%) ER Studio
6 1 (1.67%) ERwin
6 1 (1.67%) kappa
6 1 (1.67%) Sonatype Nexus
6 1 (1.67%) TeamCity
General
1 16 (26.67%) Finance
2 15 (25.00%) Banking
3 10 (16.67%) Social Skills
4 8 (13.33%) Investment Banking
5 5 (8.33%) Analytical Skills
5 5 (8.33%) French Language
6 2 (3.33%) Games
6 2 (3.33%) Presentation Skills
7 1 (1.67%) Influencing Skills
7 1 (1.67%) Organisational Skills
7 1 (1.67%) Public Sector
7 1 (1.67%) Retail
Job Titles
1 18 (30.00%) Data Engineer
2 16 (26.67%) Developer
3 8 (13.33%) Big Data Developer
3 8 (13.33%) Big Data Engineer
4 6 (10.00%) Python Developer
5 5 (8.33%) Apache Spark Developer
5 5 (8.33%) DevOps Engineer
5 5 (8.33%) ETL Developer
5 5 (8.33%) Scala Engineer
6 4 (6.67%) Architect
7 3 (5.00%) Cloud Engineer
7 3 (5.00%) Hadoop Engineer
7 3 (5.00%) Kubernetes Engineer
7 3 (5.00%) Python Engineer
8 2 (3.33%) AWS DevOps
8 2 (3.33%) Data Migration Developer
8 2 (3.33%) Senior Big Data Developer
8 2 (3.33%) Senior Developer
8 2 (3.33%) Software Engineer
8 2 (3.33%) Tester
Libraries, Frameworks & Software Standards
1 27 (45.00%) Kafka
2 9 (15.00%) Spark Streaming
3 7 (11.67%) JSON
4 6 (10.00%) Anaconda
5 5 (8.33%) XML
6 3 (5.00%) Akka
6 3 (5.00%) GraphX
6 3 (5.00%) Lucene
6 3 (5.00%) MLlib
6 3 (5.00%) PySpark
7 2 (3.33%) Spring
8 1 (1.67%) Apache NiFi
8 1 (1.67%) Matplotlib
8 1 (1.67%) NumPy
8 1 (1.67%) Pandas
8 1 (1.67%) React
8 1 (1.67%) scikit-learn
8 1 (1.67%) Spring Boot
8 1 (1.67%) Web Services
8 1 (1.67%) YAML
Miscellaneous
1 4 (6.67%) VSAM
2 3 (5.00%) Data Structures
3 1 (1.67%) Cloud Native
3 1 (1.67%) Clustering
3 1 (1.67%) Equities
3 1 (1.67%) Foreign Exchange (FX)
3 1 (1.67%) Linux Command Line
3 1 (1.67%) Management Information System
3 1 (1.67%) Reference Data
3 1 (1.67%) Virtual Team
Operating Systems
1 21 (35.00%) Linux
2 6 (10.00%) Unix
3 3 (5.00%) Windows
Processes & Methodologies
1 23 (38.33%) ETL
2 22 (36.67%) Agile Software Development
3 19 (31.67%) Analytics
4 18 (30.00%) DevOps
5 13 (21.67%) Data Management
6 11 (18.33%) Data Analytics
6 11 (18.33%) Scrum
7 10 (16.67%) Data Engineering
7 10 (16.67%) Data Ingestion
7 10 (16.67%) Data Modelling
7 10 (16.67%) Data Pipeline
7 10 (16.67%) MapReduce
8 9 (15.00%) Business Intelligence
8 9 (15.00%) Data Science
8 9 (15.00%) Migration
9 8 (13.33%) Configuration Management
9 8 (13.33%) Data Migration
10 7 (11.67%) Continuous Integration
10 7 (11.67%) Microservices
10 7 (11.67%) Software Engineering
Programming Languages
1 30 (50.00%) Python
2 29 (48.33%) Scala
3 27 (45.00%) Java
4 25 (41.67%) SQL
5 6 (10.00%) R
6 4 (6.67%) Shell Script
7 3 (5.00%) Go
7 3 (5.00%) HiveQL
7 3 (5.00%) Perl
7 3 (5.00%) PL/SQL
7 3 (5.00%) Ruby
7 3 (5.00%) VBA
8 1 (1.67%) Bash Shell
8 1 (1.67%) COBOL
8 1 (1.67%) Groovy
8 1 (1.67%) JavaScript
8 1 (1.67%) Kotlin
Qualifications
1 11 (18.33%) SC Cleared
1 11 (18.33%) Security Cleared
2 3 (5.00%) AWS Certification
3 2 (3.33%) Degree
4 1 (1.67%) Computer Science Degree
Quality Assurance & Compliance
1 3 (5.00%) Data Quality
1 3 (5.00%) QA
2 2 (3.33%) GDPR
3 1 (1.67%) PCI DSS
System Software
1 4 (6.67%) Apache Flume
2 3 (5.00%) Docker
3 2 (3.33%) Apache ZooKeeper
Systems Management
1 8 (13.33%) Hadoop YARN
2 7 (11.67%) Ansible
2 7 (11.67%) Oozie
3 6 (10.00%) Ambari
3 6 (10.00%) Kubernetes
3 6 (10.00%) Terraform
4 5 (8.33%) Grafana
4 5 (8.33%) Puppet
5 1 (1.67%) Control-M
5 1 (1.67%) Istio
Vendors
1 16 (26.67%) Cloudera
2 11 (18.33%) Oracle
3 10 (16.67%) MapR
4 7 (11.67%) Ab Initio
5 6 (10.00%) Google
5 6 (10.00%) StreamSets
6 4 (6.67%) Databricks
6 4 (6.67%) Hortonworks
6 4 (6.67%) Informatica
6 4 (6.67%) Salesforce.com
6 4 (6.67%) Teradata
7 3 (5.00%) Microsoft
7 3 (5.00%) Talend
7 3 (5.00%) Vertica
8 1 (1.67%) Alteryx
8 1 (1.67%) Dataiku
8 1 (1.67%) IBM
8 1 (1.67%) MicroStrategy
8 1 (1.67%) Netezza
8 1 (1.67%) Rackspace