Period
to

The following table provides summary statistics for contract job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 21 May 2022 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
UK
6 months to
21 May 2022
Same period 2021 Same period 2020
Rank 697 526 512
Rank change year-on-year -171 -14 +74
Contract jobs citing HDFS 146 169 126
As % of all contract jobs advertised in the UK 0.17% 0.29% 0.26%
As % of the System Software category 1.23% 2.03% 1.68%
Number of daily rates quoted 96 96 91
10th Percentile £424 £358 £409
25th Percentile £477 £455 £474
Median daily rate (50th Percentile) £544 £545 £557
Median % change year-on-year -0.16% -2.15% -3.13%
75th Percentile £638 £591 £647
90th Percentile £750 £644 £700
UK excluding London median daily rate £495 £500 £520
% change year-on-year -1.00% -3.85% -0.95%
Number of hourly rates quoted 0 3 0
Median hourly rate - £24.50 -
UK excluding London median hourly rate - £19.50 -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for system software skills.

All System Software Skills
UK
Contract vacancies with a requirement for system software skills 11,828 8,342 7,513
As % of all contract IT jobs advertised in the UK 13.40% 14.27% 15.51%
Number of daily rates quoted 8,479 5,788 5,099
10th Percentile £200 £230 £180
25th Percentile £400 £385 £338
Median daily rate (50th Percentile) £525 £490 £463
Median % change year-on-year +7.14% +5.95% -2.63%
75th Percentile £625 £575 £563
90th Percentile £700 £660 £638
UK excluding London median daily rate £450 £425 £425
% change year-on-year +5.88% - -
Number of hourly rates quoted 309 295 379
10th Percentile £13.00 £11.64 £11.26
25th Percentile £15.00 £14.00 £13.31
Median hourly rate £19.00 £21.50 £17.50
Median % change year-on-year -11.63% +22.86% +7.97%
75th Percentile £40.00 £50.00 £24.21
90th Percentile £57.75 £64.20 £51.82
UK excluding London median hourly rate £17.50 £18.27 £16.33
% change year-on-year -4.21% +11.91% +7.47%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a proportion of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Contractor Daily Rate Trend

3-month moving average daily rate quoted in jobs citing HDFS.

Daily rate trend for HDFS in the UK

HDFS
Daily Rate Histogram

Daily rate distribution for jobs citing HDFS over the 6 months to 21 May 2022.

Daily rate histogram for HDFS in the UK

HDFS
Contractor Hourly Rate Trend

3-month moving average hourly rates quoted in jobs citing HDFS.

Hourly rate trend for HDFS in the UK

HDFS
Top 14 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the UK over the 6 months to 21 May 2022. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Jobs
England -165 111 £550 +1.85% 8
Work from Home -146 71 £545 +0.93% 5
London -120 59 £590 +2.61% 6
UK excluding London -86 50 £495 -1.00% 2
Midlands -39 24 £495 +1.02% 1
West Midlands -49 22 £495 +1.02% 1
South East -51 10 £525 +31.25%
South West -30 5 - -
Scotland - 4 £558 -
Wales -11 3 £548 +15.42%
Northern Ireland - 3 £475 - 1
East Midlands - 2 - -
North West -40 1 - -
North of England -59 1 - -

For the 6 months to 21 May 2022, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for HDFS.

1 131 (89.73%) Hadoop
2 95 (65.07%) Apache Hive
3 93 (63.70%) SQL
4 91 (62.33%) Apache Spark
5 89 (60.96%) Python
6 74 (50.68%) Agile
7 73 (50.00%) Analytics
8 69 (47.26%) AWS
9 67 (45.89%) Scala
10 66 (45.21%) Git (software)
10 66 (45.21%) HBase
11 63 (43.15%) Big Data
11 63 (43.15%) Software Engineering
11 63 (43.15%) Linux
12 54 (36.99%) Java
13 53 (36.30%) Apache
14 51 (34.93%) ETL
15 50 (34.25%) Azure
16 46 (31.51%) Jenkins
17 43 (29.45%) Finance
18 40 (27.40%) Continuous Integration
19 38 (26.03%) R
20 37 (25.34%) Kafka
20 37 (25.34%) Shell Script
20 37 (25.34%) Oracle
20 37 (25.34%) Unix
21 36 (24.66%) NoSQL
22 35 (23.97%) JIRA
22 35 (23.97%) MySQL
22 35 (23.97%) Data Science

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 91 (62.33%) Apache Spark
2 53 (36.30%) Apache
3 33 (22.60%) Confluence
4 24 (16.44%) Apache Airflow
5 20 (13.70%) SAS
6 8 (5.48%) SharePoint
7 5 (3.42%) Apache Pig
8 3 (2.05%) Mule
8 3 (2.05%) OpenStack
8 3 (2.05%) WebLogic
9 2 (1.37%) Elasticsearch
9 2 (1.37%) Oracle Coherence
10 1 (0.68%) Jupyter
10 1 (0.68%) PowerExchange
Applications
1 24 (16.44%) Microsoft Excel
2 6 (4.11%) Spreadsheet
Business Applications
1 3 (2.05%) SAP FI-CA
1 3 (2.05%) SAP PSCD
Cloud Services
1 69 (47.26%) AWS
2 50 (34.25%) Azure
3 25 (17.12%) GCP
4 18 (12.33%) Amazon S3
5 8 (5.48%) Azure Data Factory
5 8 (5.48%) Power Platform
5 8 (5.48%) Vertex AI
6 7 (4.79%) Amazon Kinesis
6 7 (4.79%) Azure Synapse Analytics
7 3 (2.05%) Amazon EMR
8 2 (1.37%) AWS CloudFormation
8 2 (1.37%) Power Automate
8 2 (1.37%) PowerApps
8 2 (1.37%) Serverless
9 1 (0.68%) Amazon EKS
9 1 (0.68%) Amazon MSK
9 1 (0.68%) Amazon SageMaker
9 1 (0.68%) Snowflake
9 1 (0.68%) Tricentis Tosca
Communications & Networking
1 7 (4.79%) FTP
Database & Business Intelligence
1 131 (89.73%) Hadoop
2 95 (65.07%) Apache Hive
3 66 (45.21%) HBase
4 63 (43.15%) Big Data
5 36 (24.66%) NoSQL
6 35 (23.97%) MySQL
7 34 (23.29%) Data Warehouse
7 34 (23.29%) Impala
8 32 (21.92%) Pentaho
9 15 (10.27%) MongoDB
9 15 (10.27%) SQL Server
10 13 (8.90%) Parquet
11 12 (8.22%) Apache Cassandra
11 12 (8.22%) Relational Database
12 11 (7.53%) Tableau
13 10 (6.85%) Data Lake
13 10 (6.85%) Metadata
14 8 (5.48%) BigQuery
14 8 (5.48%) Flink
14 8 (5.48%) Sqoop
Development Applications
1 66 (45.21%) Git (software)
2 46 (31.51%) Jenkins
3 35 (23.97%) JIRA
4 25 (17.12%) Bitbucket
5 18 (12.33%) GitLab
6 16 (10.96%) MLflow
7 8 (5.48%) ER Studio
7 8 (5.48%) ERwin
8 7 (4.79%) Subversion
9 5 (3.42%) Artifactory
10 4 (2.74%) IBM UrbanCode
10 4 (2.74%) kappa
10 4 (2.74%) Maven
10 4 (2.74%) Sonatype Nexus
11 3 (2.05%) Cucumber
11 3 (2.05%) Postman
11 3 (2.05%) Selenium
11 3 (2.05%) SoapUI
11 3 (2.05%) Toad
12 2 (1.37%) Mockito
General
1 43 (29.45%) Finance
2 28 (19.18%) Social Skills
3 22 (15.07%) Banking
4 17 (11.64%) Analytical Skills
5 16 (10.96%) Public Sector
6 6 (4.11%) Front Office
7 5 (3.42%) Investment Banking
7 5 (3.42%) Presentation Skills
7 5 (3.42%) Telecoms
8 4 (2.74%) Retail
9 3 (2.05%) Financial Institution
9 3 (2.05%) Legal
10 2 (1.37%) Influencing Skills
10 2 (1.37%) Manufacturing
11 1 (0.68%) Documentation Skills
11 1 (0.68%) Dutch Language
11 1 (0.68%) French Language
11 1 (0.68%) Law
11 1 (0.68%) Publishing
Job Titles
1 43 (29.45%) Developer
2 30 (20.55%) Software Engineer
3 26 (17.81%) Data Engineer
4 16 (10.96%) Machine Learning Engineer
5 13 (8.90%) Java Developer
5 13 (8.90%) Senior Developer
6 10 (6.85%) Python Developer
6 10 (6.85%) Scala Developer
6 10 (6.85%) Senior Java Developer
7 9 (6.16%) Senior Python Developer
8 8 (5.48%) Senior Software Engineer
9 7 (4.79%) Analyst
9 7 (4.79%) Azure Developer
9 7 (4.79%) Hadoop Engineer
10 6 (4.11%) Apache Spark Developer
10 6 (4.11%) Python Engineer
11 5 (3.42%) Architect
11 5 (3.42%) Big Data Engineer
11 5 (3.42%) Market Risk Java Developer
11 5 (3.42%) Software Developer
Libraries, Frameworks & Software Standards
1 37 (25.34%) Kafka
2 19 (13.01%) Apache Beam
3 16 (10.96%) BentoML
3 16 (10.96%) PyTorch
3 16 (10.96%) TensorFlow
4 13 (8.90%) Apache Avro
5 9 (6.16%) PySpark
6 6 (4.11%) Apache NiFi
6 6 (4.11%) Spark Streaming
7 5 (3.42%) JSON
8 4 (2.74%) RabbitMQ
8 4 (2.74%) Spring
9 3 (2.05%) PyArrow
9 3 (2.05%) SOAP
9 3 (2.05%) XML
10 2 (1.37%) H2O
10 2 (1.37%) J2EE
10 2 (1.37%) REST
Miscellaneous
1 8 (5.48%) Market Risk
2 7 (4.79%) Self-Motivation
3 5 (3.42%) Data Structures
3 5 (3.42%) Public Cloud
3 5 (3.42%) Virtual Team
4 4 (2.74%) Equities
5 3 (2.05%) Hedge funds
6 2 (1.37%) Cloud Native
6 2 (1.37%) Data Fabric
6 2 (1.37%) Linux Command Line
6 2 (1.37%) Replication
7 1 (0.68%) Mainframe
7 1 (0.68%) Management Information System
Operating Systems
1 63 (43.15%) Linux
2 37 (25.34%) Unix
3 28 (19.18%) Windows
Processes & Methodologies
1 74 (50.68%) Agile
2 73 (50.00%) Analytics
3 63 (43.15%) Software Engineering
4 51 (34.93%) ETL
5 40 (27.40%) Continuous Integration
6 35 (23.97%) Data Pipeline
6 35 (23.97%) Data Science
7 34 (23.29%) Data Analytics
8 32 (21.92%) Data Modelling
8 32 (21.92%) Scrum
9 28 (19.18%) Requirements Gathering
10 27 (18.49%) Configuration Management
11 25 (17.12%) Test Strategy
12 24 (16.44%) Process Improvement
13 22 (15.07%) CI/CD
13 22 (15.07%) Data Extraction
14 21 (14.38%) Data Governance
15 20 (13.70%) Computer Science
15 20 (13.70%) Data Engineering
16 19 (13.01%) Machine Learning
Programming Languages
1 93 (63.70%) SQL
2 89 (60.96%) Python
3 67 (45.89%) Scala
4 54 (36.99%) Java
5 38 (26.03%) R
6 37 (25.34%) Shell Script
7 31 (21.23%) PL/SQL
8 27 (18.49%) HiveQL
8 27 (18.49%) Perl
9 25 (17.12%) VBA
10 5 (3.42%) SAS Macro
11 4 (2.74%) C
12 3 (2.05%) Bash
12 3 (2.05%) PowerShell
13 2 (1.37%) C++
13 2 (1.37%) Go
13 2 (1.37%) Kotlin
13 2 (1.37%) T-SQL
14 1 (0.68%) Data Analysis Expressions (DAX)
Qualifications
1 22 (15.07%) Security Cleared
2 19 (13.01%) SC Cleared
3 16 (10.96%) PhD
4 5 (3.42%) Degree
5 4 (2.74%) Computer Science Degree
6 3 (2.05%) ISEB
6 3 (2.05%) ISTQB
6 3 (2.05%) ISTQB Foundation Certification
7 2 (1.37%) AWS Certification
7 2 (1.37%) Master's Degree
8 1 (0.68%) DBS Check
8 1 (0.68%) DV Cleared
Quality Assurance & Compliance
1 14 (9.59%) QA
2 10 (6.85%) Data Quality
3 3 (2.05%) GDPR
System Software
1 23 (15.75%) Docker
2 10 (6.85%) Apache ZooKeeper
3 5 (3.42%) Apache Flume
Systems Management
1 17 (11.64%) Kubeflow
2 12 (8.22%) Hadoop YARN
3 8 (5.48%) Ansible
4 5 (3.42%) Kubernetes
5 4 (2.74%) Terraform
6 3 (2.05%) Oozie
7 1 (0.68%) Kibana
Vendors
1 37 (25.34%) Oracle
2 29 (19.86%) Talend
3 25 (17.12%) Cloudera
3 25 (17.12%) Informatica
4 12 (8.22%) Databricks
5 11 (7.53%) SAP
6 10 (6.85%) Hortonworks
7 7 (4.79%) Salesforce.com
8 5 (3.42%) MapR
8 5 (3.42%) Teradata
9 4 (2.74%) Ab Initio
10 3 (2.05%) Google
11 2 (1.37%) Bloomberg
11 2 (1.37%) Microsoft
11 2 (1.37%) Refinitiv
11 2 (1.37%) ServiceNow
11 2 (1.37%) StreamSets
12 1 (0.68%) Alteryx
12 1 (0.68%) Atlassian
12 1 (0.68%) MicroStrategy