Period
to 25 October 2021

The following table provides summary statistics for contract job vacancies advertised in England with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 25 October 2021 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
UK > England
6 months to
25 Oct 2021
Same period 2020 Same period 2019
Rank 564 416 625
Rank change year-on-year -148 +209 -57
Contract jobs citing HDFS 188 81 155
As % of all contract jobs advertised in England 0.28% 0.30% 0.20%
As % of the System Software category 1.95% 2.07% 1.29%
Number of daily rates quoted 116 57 83
10th Percentile £400 £356 £429
25th Percentile £483 £413 £469
Median daily rate (50th Percentile) £540 £450 £550
Median % change year-on-year +20.00% -18.18% +4.76%
75th Percentile £675 £613 £659
90th Percentile £713 £675 £700
UK median daily rate £545 £450 £540
% change year-on-year +21.11% -16.67% +2.86%
Number of hourly rates quoted 3 0 0
Median hourly rate £26.00 - -
UK median hourly rate £26.00 - -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in England with a requirement for system software skills.

All System Software Skills
England
Contract vacancies with a requirement for system software skills 9,647 3,922 11,997
As % of all contract IT jobs advertised in England 14.22% 14.34% 15.58%
Number of daily rates quoted 6,931 2,659 8,183
10th Percentile £200 £225 £170
25th Percentile £384 £363 £325
Median daily rate (50th Percentile) £500 £475 £463
Median % change year-on-year +5.26% +2.70% +2.78%
75th Percentile £600 £575 £575
90th Percentile £688 £650 £638
UK median daily rate £500 £475 £450
% change year-on-year +5.26% +5.56% -
Number of hourly rates quoted 354 149 546
10th Percentile £12.02 £11.17 £11.17
25th Percentile £13.25 £12.70 £13.23
Median hourly rate £16.98 £17.50 £16.00
Median % change year-on-year -3.00% +9.38% -
75th Percentile £25.88 £35.75 £23.33
90th Percentile £63.35 £49.20 £47.25
UK median hourly rate £17.50 £17.50 £16.50
% change year-on-year - +6.06% +3.13%

HDFS
Job Vacancy Trend in England

Job postings citing HDFS as a proportion of all IT jobs advertised in England.

Job vacancy trend for HDFS in England

HDFS
Contractor Daily Rate Trend in England

3-month moving average daily rate quoted in jobs citing HDFS in England.

Daily rate trend for HDFS in England

HDFS
Daily Rate Histogram in England

Daily rate distribution for jobs citing HDFS in England over the 6 months to 25 October 2021.

Daily rate histogram for HDFS in England

HDFS
Contractor Hourly Rate Trend in England

3-month moving average hourly rates quoted in jobs citing HDFS in England.

Hourly rate trend for HDFS in England

HDFS
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the England region over the 6 months to 25 October 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London -139 76 £613 +2.08% 18
Midlands -48 40 £514 +18.98% 17
West Midlands -44 37 £518 +19.91% 15
South East -80 27 £425 -5.56% 9
North of England -91 13 £725 +67.82% 3
South West -39 9 £575 - 2
East of England - 9 £540 - 1
North East - 8 £750 - 1
Yorkshire - 4 £570 - 1
East Midlands -38 3 £500 +32.28% 2
North West -74 2 £492 +13.95% 1
HDFS
UK

For the 6 months to 25 October 2021, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region with a requirement for HDFS.

1 174 (92.55%) Hadoop
2 165 (87.77%) Apache Hive
3 126 (67.02%) Apache Spark
4 101 (53.72%) HBase
5 93 (49.47%) Python
6 89 (47.34%) AWS
7 85 (45.21%) ETL
8 84 (44.68%) SQL
9 83 (44.15%) Java
10 77 (40.96%) Jenkins
11 76 (40.43%) Analytics
12 75 (39.89%) Big Data
13 66 (35.11%) Linux
14 63 (33.51%) Git (software)
14 63 (33.51%) Kafka
15 61 (32.45%) Scala
16 57 (30.32%) Unix
17 55 (29.26%) Oracle
17 55 (29.26%) Azure
18 53 (28.19%) Agile Software Development
19 52 (27.66%) Data Analytics
20 48 (25.53%) Data Modelling
21 47 (25.00%) Business Intelligence
21 47 (25.00%) Informatica
22 46 (24.47%) MySQL
22 46 (24.47%) Security Cleared
23 43 (22.87%) Impala
24 42 (22.34%) Pentaho
25 41 (21.81%) Software Engineering
25 41 (21.81%) Social Skills

HDFS
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 126 (67.02%) Apache Spark
2 37 (19.68%) Confluence
3 34 (18.09%) SAS
4 17 (9.04%) Apache
5 7 (3.72%) Elasticsearch
6 6 (3.19%) Apache Pig
7 5 (2.66%) nginx
8 2 (1.06%) Apache Airflow
9 1 (0.53%) Apache Solr
9 1 (0.53%) Mule
9 1 (0.53%) PowerExchange
9 1 (0.53%) WebLogic
Applications
1 23 (12.23%) Microsoft Excel
2 9 (4.79%) Spreadsheet
Business Applications
1 1 (0.53%) SAP FI-CA
1 1 (0.53%) SAP PSCD
Cloud Services
1 89 (47.34%) AWS
2 55 (29.26%) Azure
3 23 (12.23%) Amazon S3
4 19 (10.11%) GCP
5 15 (7.98%) Power Platform
6 9 (4.79%) Azure Data Factory
6 9 (4.79%) Azure Synapse Analytics
6 9 (4.79%) Snowflake
7 8 (4.26%) GitHub
8 7 (3.72%) Tricentis Tosca
9 5 (2.66%) Amazon EMR
9 5 (2.66%) Azure App Service
9 5 (2.66%) Azure Key Vault
9 5 (2.66%) Azure Storage
10 4 (2.13%) OpenShift
11 3 (1.60%) Amazon Kinesis
11 3 (1.60%) Amazon SQS
12 2 (1.06%) Azure Active Directory
12 2 (1.06%) Google Compute Engine
12 2 (1.06%) Microsoft 365
Communications & Networking
1 9 (4.79%) FTP
1 9 (4.79%) WAN
2 8 (4.26%) HTTP
3 7 (3.72%) Firewall
3 7 (3.72%) Network Security
4 5 (2.66%) HTTPS
4 5 (2.66%) SSL
5 2 (1.06%) Ethernet
5 2 (1.06%) Kerberos
5 2 (1.06%) PuTTY
5 2 (1.06%) VLAN
5 2 (1.06%) VSAN
Database & Business Intelligence
1 174 (92.55%) Hadoop
2 165 (87.77%) Apache Hive
3 101 (53.72%) HBase
4 75 (39.89%) Big Data
5 46 (24.47%) MySQL
6 43 (22.87%) Impala
7 42 (22.34%) Pentaho
8 40 (21.28%) Relational Database
9 39 (20.74%) Data Warehouse
10 30 (15.96%) PostgreSQL
11 29 (15.43%) Metadata
12 27 (14.36%) RDBMS
13 24 (12.77%) NoSQL
14 21 (11.17%) Apache Cassandra
14 21 (11.17%) MongoDB
15 16 (8.51%) Data Lake
16 15 (7.98%) Power BI
17 13 (6.91%) SQL Server
17 13 (6.91%) Sqoop
18 11 (5.85%) Amazon RDS
Development Applications
1 77 (40.96%) Jenkins
2 63 (33.51%) Git (software)
3 39 (20.74%) JIRA
4 38 (20.21%) Bitbucket
5 15 (7.98%) ERwin
6 13 (6.91%) GitLab
7 11 (5.85%) ER Studio
7 11 (5.85%) Maven
7 11 (5.85%) Sonatype Nexus
8 9 (4.79%) Subversion
9 8 (4.26%) Artifactory
10 5 (2.66%) IBM UrbanCode
11 4 (2.13%) Atlassian Bamboo
12 3 (1.60%) Eclipse
12 3 (1.60%) IntelliJ
12 3 (1.60%) JUnit
12 3 (1.60%) Mockito
12 3 (1.60%) TeamCity
13 2 (1.06%) Gradle
13 2 (1.06%) ProC
General
1 41 (21.81%) Social Skills
2 32 (17.02%) Banking
3 27 (14.36%) Public Sector
4 22 (11.70%) Finance
5 14 (7.45%) Investment Banking
6 11 (5.85%) Presentation Skills
7 10 (5.32%) Analytical Skills
8 4 (2.13%) Publishing
8 4 (2.13%) Retail
9 2 (1.06%) Pharmaceutical
9 2 (1.06%) Telecoms
10 1 (0.53%) Digital Economy
Job Titles
1 58 (30.85%) Data Engineer
2 43 (22.87%) Developer
3 17 (9.04%) Big Data Manager
3 17 (9.04%) Data Manager
3 17 (9.04%) Software Engineer
4 16 (8.51%) Platform Manager
5 12 (6.38%) Hadoop Engineer
6 9 (4.79%) DevOps Engineer
6 9 (4.79%) Support Engineer
7 8 (4.26%) Big Data Engineer
7 8 (4.26%) Data Modeller
7 8 (4.26%) Scala Developer
8 7 (3.72%) Azure Developer
8 7 (3.72%) Senior Data Engineer
9 6 (3.19%) Ab Initio Developer
9 6 (3.19%) Cloud Engineer
9 6 (3.19%) Java Developer
9 6 (3.19%) Java Engineer
10 5 (2.66%) Administrator
10 5 (2.66%) Python Engineer
Libraries, Frameworks & Software Standards
1 63 (33.51%) Kafka
2 20 (10.64%) Spark Streaming
3 15 (7.98%) Apache NiFi
3 15 (7.98%) JSON
4 13 (6.91%) PySpark
5 11 (5.85%) Spring
6 8 (4.26%) JDBC
7 7 (3.72%) ODBC
7 7 (3.72%) OLE DB
7 7 (3.72%) Regular Expression
8 6 (3.19%) YAML
9 5 (2.66%) Spring Boot
10 4 (2.13%) Node.js
11 3 (1.60%) Akka
11 3 (1.60%) Apache Avro
11 3 (1.60%) RESTful
11 3 (1.60%) WireMock
11 3 (1.60%) XML
12 2 (1.06%) Dojo
12 2 (1.06%) Pandas
Miscellaneous
1 5 (2.66%) Algorithms
1 5 (2.66%) Foreign Exchange (FX)
2 4 (2.13%) Cloud Native
2 4 (2.13%) Data Structures
2 4 (2.13%) Greenfield Project
3 3 (1.60%) Data Protection Act
3 3 (1.60%) Hybrid Cloud
3 3 (1.60%) Public Cloud
3 3 (1.60%) Reference Data
4 1 (0.53%) Equities
4 1 (0.53%) Linux Command Line
4 1 (0.53%) PKI
4 1 (0.53%) Self-Motivation
4 1 (0.53%) Wiki
Operating Systems
1 66 (35.11%) Linux
2 57 (30.32%) Unix
3 26 (13.83%) Windows
4 9 (4.79%) Red Hat Enterprise Linux
5 2 (1.06%) Ubuntu
6 1 (0.53%) Windows Server
6 1 (0.53%) Windows Server 2012
Processes & Methodologies
1 85 (45.21%) ETL
2 76 (40.43%) Analytics
3 53 (28.19%) Agile Software Development
4 52 (27.66%) Data Analytics
5 48 (25.53%) Data Modelling
6 47 (25.00%) Business Intelligence
7 41 (21.81%) Software Engineering
8 38 (20.21%) Continuous Integration
8 38 (20.21%) Scrum
9 37 (19.68%) DevOps
10 33 (17.55%) Configuration Management
10 33 (17.55%) Data Virtualisation
11 32 (17.02%) Data Management
12 25 (13.30%) Use Case
13 24 (12.77%) Problem-Solving
14 22 (11.70%) Data Extraction
14 22 (11.70%) Requirements Gathering
15 21 (11.17%) Test Automation
16 20 (10.64%) Process Improvement
17 19 (10.11%) Test Strategy
Programming Languages
1 93 (49.47%) Python
2 84 (44.68%) SQL
3 83 (44.15%) Java
4 61 (32.45%) Scala
5 36 (19.15%) HiveQL
6 35 (18.62%) PL/SQL
7 31 (16.49%) Shell Script
8 28 (14.89%) R
9 24 (12.77%) Perl
10 23 (12.23%) VBA
11 18 (9.57%) Groovy
12 17 (9.04%) Bash Shell
13 8 (4.26%) C++
14 7 (3.72%) C
15 5 (2.66%) Ruby
16 3 (1.60%) PowerShell
17 2 (1.06%) COBOL
17 2 (1.06%) Go
17 2 (1.06%) Pig Latin
18 1 (0.53%) PROC SQL
Qualifications
1 46 (24.47%) Security Cleared
2 31 (16.49%) SC Cleared
3 12 (6.38%) Degree
4 5 (2.66%) Computer Science Degree
4 5 (2.66%) DV Cleared
5 3 (1.60%) BPSS Clearance
6 1 (0.53%) CCA
6 1 (0.53%) ISEB
6 1 (0.53%) ISTQB
6 1 (0.53%) ISTQB Foundation Certification
Quality Assurance & Compliance
1 9 (4.79%) Data Quality
2 8 (4.26%) QA
3 3 (1.60%) Sarbanes-Oxley
4 2 (1.06%) Disclosure Scotland
5 1 (0.53%) GDPR
System Software
1 23 (12.23%) Docker
2 17 (9.04%) Apache ZooKeeper
3 7 (3.72%) Apache Flume
4 3 (1.60%) VMware Infrastructure
5 2 (1.06%) Active Directory
5 2 (1.06%) VMware NSX
5 2 (1.06%) vSphere
6 1 (0.53%) Hyper-V
Systems Management
1 36 (19.15%) Kubernetes
2 26 (13.83%) Ansible
3 25 (13.30%) Oozie
4 24 (12.77%) Terraform
5 22 (11.70%) Hadoop YARN
6 11 (5.85%) Ambari
7 7 (3.72%) Kibana
7 7 (3.72%) Puppet
8 6 (3.19%) Rancher
9 5 (2.66%) Grafana
9 5 (2.66%) Prometheus
10 3 (1.60%) Istio
10 3 (1.60%) logstash
11 2 (1.06%) Microsoft Intune
11 2 (1.06%) vCenter Server
12 1 (0.53%) Nagios
12 1 (0.53%) Opsview
12 1 (0.53%) QRadar
Vendors
1 55 (29.26%) Oracle
2 47 (25.00%) Informatica
3 36 (19.15%) Cloudera
4 33 (17.55%) Talend
5 24 (12.77%) Ab Initio
6 18 (9.57%) MapR
7 11 (5.85%) MicroStrategy
8 10 (5.32%) Databricks
8 10 (5.32%) Hortonworks
9 9 (4.79%) Red Hat
10 8 (4.26%) Google
10 8 (4.26%) SAP
11 7 (3.72%) Salesforce.com
11 7 (3.72%) Teradata
12 6 (3.19%) Microsoft
13 5 (2.66%) Dynatrace
14 3 (1.60%) VMware
15 2 (1.06%) Cisco
15 2 (1.06%) Dell
15 2 (1.06%) IBM