Period
to 1 August 2021

The following table provides summary statistics for contract job vacancies advertised in England with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 1 August 2021 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
UK > England
6 months to
1 Aug 2021
Same period 2020 Same period 2019
Rank 542 469 611
Rank change year-on-year -73 +142 -49
Contract jobs citing HDFS 157 56 223
As % of all contract jobs advertised in England 0.27% 0.17% 0.27%
As % of the System Software category 1.90% 1.15% 1.72%
Number of daily rates quoted 88 37 139
10th Percentile £360 £375 £467
25th Percentile £457 £413 £525
Median daily rate (50th Percentile) £550 £525 £600
Median % change year-on-year +4.76% -12.50% +9.09%
75th Percentile £625 £650 £675
90th Percentile £729 £680 £700
UK median daily rate £550 £505 £575
% change year-on-year +8.91% -12.17% +4.55%
Number of hourly rates quoted 2 0 0
Median hourly rate £16.75 - -
UK median hourly rate £21.75 - -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in England with a requirement for system software skills.

All System Software Skills
England
Contract vacancies with a requirement for system software skills 8,262 4,855 12,932
As % of all contract IT jobs advertised in England 14.43% 15.04% 15.52%
Number of daily rates quoted 5,793 3,319 8,792
10th Percentile £213 £200 £181
25th Percentile £376 £363 £353
Median daily rate (50th Percentile) £500 £475 £475
Median % change year-on-year +5.26% - +5.56%
75th Percentile £588 £575 £575
90th Percentile £675 £638 £638
UK median daily rate £500 £475 £475
% change year-on-year +5.26% - +5.56%
Number of hourly rates quoted 260 208 501
10th Percentile £11.17 £11.25 £11.17
25th Percentile £13.25 £13.00 £13.00
Median hourly rate £17.50 £17.42 £16.50
Median % change year-on-year +0.44% +5.59% +2.48%
75th Percentile £31.63 £22.81 £25.00
90th Percentile £63.08 £47.50 £47.69
UK median hourly rate £18.27 £17.50 £16.95
% change year-on-year +4.40% +3.24% +5.94%

HDFS
Job Vacancy Trend in England

Job postings citing HDFS as a proportion of all IT jobs advertised in England.

Job vacancy trend for HDFS in England

HDFS
Contractor Daily Rate Trend in England

3-month moving average daily rate quoted in jobs citing HDFS in England.

Daily rate trend for HDFS in England

HDFS
Daily Rate Histogram in England

Daily rate distribution for jobs citing HDFS in England over the 6 months to 1 August 2021.

Daily rate histogram for HDFS in England

HDFS
Contractor Hourly Rate Trend in England

3-month moving average hourly rates quoted in jobs citing HDFS in England.

Hourly rate trend for HDFS in England

HDFS
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the England region over the 6 months to 1 August 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London -98 65 £580 +5.41% 9
Midlands -13 33 £500 +15.34% 3
West Midlands -10 31 £500 +15.34% 2
South East -38 19 £360 -24.21% 3
North of England -31 15 £725 +67.82% 3
North East - 8 £750 -
South West -15 7 £550 +4.27%
Yorkshire -18 5 £589 - 1
East of England -21 3 £321 -
North West -34 3 £423 -2.08% 2
East Midlands - 2 £500 - 1
HDFS
UK

For the 6 months to 1 August 2021, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region with a requirement for HDFS.

1 146 (92.99%) Hadoop
2 137 (87.26%) Apache Hive
3 110 (70.06%) Apache Spark
4 84 (53.50%) Python
5 76 (48.41%) AWS
6 75 (47.77%) HBase
7 70 (44.59%) ETL
8 68 (43.31%) Jenkins
9 67 (42.68%) SQL
10 61 (38.85%) Java
10 61 (38.85%) Big Data
11 60 (38.22%) Analytics
12 53 (33.76%) Scala
12 53 (33.76%) Linux
13 51 (32.48%) Git (software)
14 49 (31.21%) Azure
14 49 (31.21%) Agile Software Development
15 48 (30.57%) Unix
16 44 (28.03%) Kafka
17 42 (26.75%) Security Cleared
17 42 (26.75%) Oracle
18 41 (26.11%) DevOps
19 37 (23.57%) Cloudera
20 36 (22.93%) Kubernetes
21 35 (22.29%) Data Analytics
21 35 (22.29%) Pentaho
21 35 (22.29%) MySQL
21 35 (22.29%) Data Modelling
22 34 (21.66%) Informatica
22 34 (21.66%) Scrum

HDFS
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 110 (70.06%) Apache Spark
2 31 (19.75%) Confluence
3 26 (16.56%) Apache
3 26 (16.56%) SAS
4 10 (6.37%) Apache Airflow
5 4 (2.55%) Apache Pig
5 4 (2.55%) Apache Solr
5 4 (2.55%) nginx
6 3 (1.91%) Elasticsearch
7 1 (0.64%) Mule
7 1 (0.64%) PowerExchange
7 1 (0.64%) WebLogic
Applications
1 14 (8.92%) Microsoft Excel
2 9 (5.73%) Spreadsheet
Business Applications
1 1 (0.64%) SAP FI-CA
1 1 (0.64%) SAP PSCD
Cloud Services
1 76 (48.41%) AWS
2 49 (31.21%) Azure
3 23 (14.65%) GCP
4 21 (13.38%) Amazon S3
5 16 (10.19%) Power Platform
6 7 (4.46%) GitHub
6 7 (4.46%) Tricentis Tosca
7 6 (3.82%) Azure Data Factory
7 6 (3.82%) Azure Storage
8 5 (3.18%) Amazon EMR
8 5 (3.18%) Azure Synapse Analytics
9 4 (2.55%) Azure App Service
9 4 (2.55%) Azure Key Vault
9 4 (2.55%) Snowflake
10 3 (1.91%) Amazon Kinesis
10 3 (1.91%) Amazon SQS
10 3 (1.91%) OpenShift
10 3 (1.91%) PaaS
10 3 (1.91%) SaaS
11 2 (1.27%) Microsoft 365
Communications & Networking
1 6 (3.82%) Firewall
1 6 (3.82%) HTTP
2 4 (2.55%) FTP
2 4 (2.55%) HTTPS
2 4 (2.55%) SSL
3 3 (1.91%) Kerberos
4 2 (1.27%) Ethernet
4 2 (1.27%) VLAN
4 2 (1.27%) VSAN
4 2 (1.27%) WAN
5 1 (0.64%) PuTTY
Database & Business Intelligence
1 146 (92.99%) Hadoop
2 137 (87.26%) Apache Hive
3 75 (47.77%) HBase
4 61 (38.85%) Big Data
5 35 (22.29%) MySQL
5 35 (22.29%) Pentaho
6 33 (21.02%) Impala
6 33 (21.02%) Relational Database
7 29 (18.47%) Data Warehouse
8 27 (17.20%) PostgreSQL
9 19 (12.10%) RDBMS
10 18 (11.46%) Data Lake
10 18 (11.46%) Metadata
10 18 (11.46%) Spark SQL
11 17 (10.83%) NoSQL
12 16 (10.19%) Power BI
13 15 (9.55%) Apache Cassandra
14 14 (8.92%) MongoDB
15 13 (8.28%) Amazon Redshift
16 12 (7.64%) SQL Server
Development Applications
1 68 (43.31%) Jenkins
2 51 (32.48%) Git (software)
3 33 (21.02%) JIRA
4 32 (20.38%) Bitbucket
5 16 (10.19%) GitLab
6 15 (9.55%) Sonatype Nexus
7 12 (7.64%) Artifactory
8 10 (6.37%) Maven
9 7 (4.46%) IBM UrbanCode
10 6 (3.82%) ERwin
11 5 (3.18%) ER Studio
12 3 (1.91%) Gradle
12 3 (1.91%) JUnit
12 3 (1.91%) Subversion
12 3 (1.91%) TeamCity
13 2 (1.27%) Cucumber
13 2 (1.27%) Eclipse
13 2 (1.27%) IntelliJ
13 2 (1.27%) Mockito
14 1 (0.64%) SoapUI
General
1 27 (17.20%) Banking
2 25 (15.92%) Social Skills
3 20 (12.74%) Finance
4 18 (11.46%) Public Sector
5 9 (5.73%) Investment Banking
6 6 (3.82%) Analytical Skills
7 5 (3.18%) French Language
8 2 (1.27%) Documentation Skills
8 2 (1.27%) Games
8 2 (1.27%) Presentation Skills
8 2 (1.27%) Publishing
9 1 (0.64%) Digital Economy
9 1 (0.64%) Retail
Job Titles
1 45 (28.66%) Data Engineer
2 40 (25.48%) Developer
3 13 (8.28%) DevOps Engineer
4 12 (7.64%) Hadoop Engineer
5 10 (6.37%) Big Data Developer
5 10 (6.37%) Software Engineer
6 8 (5.10%) Scala Engineer
6 8 (5.10%) Support Engineer
7 7 (4.46%) Apache Spark Developer
7 7 (4.46%) Architect
7 7 (4.46%) Big Data Engineer
7 7 (4.46%) Cloud Engineer
7 7 (4.46%) ETL Developer
7 7 (4.46%) Scala Developer
8 6 (3.82%) Big Data Manager
8 6 (3.82%) Data Manager
8 6 (3.82%) Hadoop Developer
8 6 (3.82%) Python Developer
8 6 (3.82%) Senior Data Engineer
9 5 (3.18%) Platform Manager
Libraries, Frameworks & Software Standards
1 44 (28.03%) Kafka
2 16 (10.19%) JSON
3 14 (8.92%) Spring
4 12 (7.64%) PySpark
4 12 (7.64%) Spark Streaming
5 6 (3.82%) Akka
5 6 (3.82%) Anaconda
5 6 (3.82%) Apache NiFi
5 6 (3.82%) YAML
6 5 (3.18%) Spring Boot
7 4 (2.55%) MLlib
8 3 (1.91%) GraphX
8 3 (1.91%) Lucene
8 3 (1.91%) Node.js
9 2 (1.27%) Pandas
9 2 (1.27%) RESTful
9 2 (1.27%) WireMock
9 2 (1.27%) XML
10 1 (0.64%) AngularJS
10 1 (0.64%) scikit-learn
Miscellaneous
1 7 (4.46%) Data Structures
1 7 (4.46%) Linux Command Line
2 3 (1.91%) Cloud Native
2 3 (1.91%) Hybrid Cloud
2 3 (1.91%) Public Cloud
3 1 (0.64%) Algorithms
3 1 (0.64%) Analytical Mindset
3 1 (0.64%) Equities
3 1 (0.64%) Foreign Exchange (FX)
3 1 (0.64%) Reference Data
3 1 (0.64%) Virtual Team
Operating Systems
1 53 (33.76%) Linux
2 48 (30.57%) Unix
3 17 (10.83%) Windows
4 2 (1.27%) Ubuntu
5 1 (0.64%) Red Hat Enterprise Linux
Processes & Methodologies
1 70 (44.59%) ETL
2 60 (38.22%) Analytics
3 49 (31.21%) Agile Software Development
4 41 (26.11%) DevOps
5 35 (22.29%) Data Analytics
5 35 (22.29%) Data Modelling
6 34 (21.66%) Scrum
7 29 (18.47%) Software Engineering
8 28 (17.83%) Business Intelligence
9 27 (17.20%) Configuration Management
10 26 (16.56%) Continuous Integration
11 24 (15.29%) Data Virtualisation
12 20 (12.74%) Test Automation
13 18 (11.46%) Problem-Solving
14 14 (8.92%) Microservices
14 14 (8.92%) Requirements Gathering
15 13 (8.28%) Data Acquisition
15 13 (8.28%) Data Engineering
15 13 (8.28%) Data Extraction
16 12 (7.64%) Data Integration
Programming Languages
1 84 (53.50%) Python
2 67 (42.68%) SQL
3 61 (38.85%) Java
4 53 (33.76%) Scala
5 26 (16.56%) PL/SQL
6 24 (15.29%) HiveQL
7 23 (14.65%) Shell Script
8 17 (10.83%) R
9 15 (9.55%) Groovy
10 14 (8.92%) Perl
10 14 (8.92%) VBA
11 6 (3.82%) Bash Shell
11 6 (3.82%) C
12 4 (2.55%) Go
12 4 (2.55%) Ruby
13 2 (1.27%) Elm
13 2 (1.27%) Haskell
13 2 (1.27%) PowerShell
13 2 (1.27%) T-SQL
14 1 (0.64%) PROC SQL
Qualifications
1 42 (26.75%) Security Cleared
2 27 (17.20%) SC Cleared
3 4 (2.55%) Computer Science Degree
3 4 (2.55%) Degree
4 3 (1.91%) BPSS Clearance
5 1 (0.64%) CCA
5 1 (0.64%) ISEB
5 1 (0.64%) ISTQB
5 1 (0.64%) ISTQB Foundation Certification
Quality Assurance & Compliance
1 12 (7.64%) QA
2 11 (7.01%) Data Quality
3 2 (1.27%) Disclosure Scotland
3 2 (1.27%) Sarbanes-Oxley
4 1 (0.64%) GDPR
System Software
1 21 (13.38%) Docker
2 12 (7.64%) Apache ZooKeeper
3 2 (1.27%) Active Directory
3 2 (1.27%) VMware Infrastructure
3 2 (1.27%) VMware NSX
3 2 (1.27%) vSphere
4 1 (0.64%) Apache Flume
Systems Management
1 36 (22.93%) Kubernetes
2 31 (19.75%) Ansible
3 26 (16.56%) Hadoop YARN
4 25 (15.92%) Terraform
5 18 (11.46%) Oozie
6 17 (10.83%) Ambari
7 9 (5.73%) Grafana
8 8 (5.10%) Puppet
9 4 (2.55%) Prometheus
9 4 (2.55%) Rancher
10 3 (1.91%) Istio
10 3 (1.91%) Kibana
11 2 (1.27%) Microsoft Intune
11 2 (1.27%) vCenter Server
12 1 (0.64%) Control-M
12 1 (0.64%) logstash
Vendors
1 42 (26.75%) Oracle
2 37 (23.57%) Cloudera
3 34 (21.66%) Informatica
4 24 (15.29%) Talend
5 13 (8.28%) Ab Initio
6 11 (7.01%) Hortonworks
6 11 (7.01%) MapR
7 10 (6.37%) Google
8 8 (5.10%) Microsoft
9 7 (4.46%) Databricks
10 5 (3.18%) MicroStrategy
11 4 (2.55%) Dynatrace
11 4 (2.55%) IBM
11 4 (2.55%) SAP
11 4 (2.55%) StreamSets
12 3 (1.91%) Salesforce.com
12 3 (1.91%) Teradata
13 2 (1.27%) Cisco
13 2 (1.27%) Dell
13 2 (1.27%) VMware