Period
to 12 May 2021

The following table provides summary statistics for permanent job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited HDFS over the 6 months to 12 May 2021 with a comparison to the same period in the previous 2 years.

Hadoop Distributed File System (HDFS)
UK
6 months to
12 May 2021
Same period 2020 Same period 2019
Rank 780 828 864
Rank change year-on-year +48 +36 -66
Permanent jobs citing HDFS 105 100 258
As % of all permanent jobs advertised in the UK 0.12% 0.10% 0.18%
As % of the System Software category 0.64% 0.59% 1.08%
Number of salaries quoted 44 83 205
10th Percentile £49,050 £50,348 £46,250
25th Percentile £50,000 £60,250 £50,000
Median annual salary (50th Percentile) £60,000 £75,000 £65,000
Median % change year-on-year -20.00% +15.38% -7.14%
75th Percentile £72,813 £81,875 £81,250
90th Percentile £82,850 £105,000 £92,500
UK excluding London median annual salary £55,500 £75,000 £52,500
% change year-on-year -26.00% +42.86% -12.50%

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for system software skills.

All System Software Skills
UK
Permanent vacancies with a requirement for system software skills 16,533 17,035 23,796
As % of all permanent IT jobs advertised in the UK 18.65% 17.28% 16.36%
Number of salaries quoted 13,077 14,045 19,567
10th Percentile £30,000 £26,750 £25,750
25th Percentile £39,000 £36,250 £33,750
Median annual salary (50th Percentile) £55,000 £52,500 £50,000
Median % change year-on-year +4.76% +5.00% +5.26%
75th Percentile £75,000 £72,500 £68,750
90th Percentile £90,000 £87,500 £85,000
UK excluding London median annual salary £47,500 £45,000 £42,500
% change year-on-year +5.56% +5.88% +6.25%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a proportion of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Salary Trend

3-month moving average salary quoted in jobs citing HDFS.

Salary trend for HDFS in the UK

HDFS
Salary Histogram

Salary distribution for jobs citing HDFS over the 6 months to 12 May 2021.

Salary histogram for HDFS in the UK

HDFS
Top 15 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing HDFS within the UK over the 6 months to 12 May 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +68 103 £60,000 -20.00% 18
UK excluding London +44 61 £55,500 -26.00% 12
London +66 41 £65,000 -13.33% 6
South East +72 35 £65,000 -13.33% 1
Work from Home -159 14 £65,000 -10.34% 1
North of England 0 11 £55,000 +0.60%
Yorkshire -7 9 £55,000 +0.60%
Midlands -24 9 £55,500 -26.00% 10
East Midlands -15 7 £55,500 -26.00% 2
South West -14 3 - -
West Midlands -17 2 £58,750 +11.90% 8
Northern Ireland - 2 £63,750 -
North West +25 1 - -
East of England +19 1 - - 1
North East +1 1 £56,914 +4.10%

For the 6 months to 12 May 2021, IT jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for HDFS.

1 72 (68.57%) Hadoop
2 70 (66.67%) Java
3 69 (65.71%) Apache Spark
4 64 (60.95%) HBase
4 64 (60.95%) Big Data
5 60 (57.14%) Apache Hive
6 59 (56.19%) Python
7 57 (54.29%) Kafka
8 52 (49.52%) Degree
8 52 (49.52%) Computer Science
9 50 (47.62%) Agile Software Development
9 50 (47.62%) SQL
10 49 (46.67%) AWS
10 49 (46.67%) Git (software)
11 47 (44.76%) Ansible
11 47 (44.76%) Unix
11 47 (44.76%) Shell Script
12 45 (42.86%) Linux
12 45 (42.86%) Apache
12 45 (42.86%) Azure
13 44 (41.90%) Finance
13 44 (41.90%) Software Engineering
14 42 (40.00%) DevOps
15 41 (39.05%) AngularJS
15 41 (39.05%) Jenkins
16 39 (37.14%) MariaDB
17 38 (36.19%) PostgreSQL
17 38 (36.19%) ETL
17 38 (36.19%) Spark Streaming
18 37 (35.24%) Banking

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 69 (65.71%) Apache Spark
2 45 (42.86%) Apache
3 33 (31.43%) GemFire
4 14 (13.33%) SAS
5 3 (2.86%) Elasticsearch
6 2 (1.90%) Apache Airflow
6 2 (1.90%) Apache Pig
6 2 (1.90%) Oracle SOA Suite
7 1 (0.95%) BizTalk Server
Business Applications
1 1 (0.95%) Dynamics CRM
Cloud Services
1 49 (46.67%) AWS
2 45 (42.86%) Azure
3 33 (31.43%) GitHub
4 21 (20.00%) Power Platform
5 20 (19.05%) GCP
6 10 (9.52%) Amazon S3
7 9 (8.57%) Amazon Kinesis
7 9 (8.57%) Azure Synapse Analytics
7 9 (8.57%) OpenShift
8 6 (5.71%) Azure Data Factory
9 4 (3.81%) AWS CloudFormation
9 4 (3.81%) PaaS
9 4 (3.81%) SaaS
10 3 (2.86%) Cloud Computing
11 2 (1.90%) IaaS
11 2 (1.90%) Matillion
12 1 (0.95%) Amazon EMR
12 1 (0.95%) AWS Data Pipeline
Communications & Networking
1 8 (7.62%) FTP
2 2 (1.90%) Kerberos
2 2 (1.90%) SMS
Database & Business Intelligence
1 72 (68.57%) Hadoop
2 64 (60.95%) Big Data
2 64 (60.95%) HBase
3 60 (57.14%) Apache Hive
4 39 (37.14%) MariaDB
5 38 (36.19%) PostgreSQL
6 33 (31.43%) Redis
7 30 (28.57%) NoSQL
7 30 (28.57%) Relational Database
8 22 (20.95%) Impala
9 21 (20.00%) Power BI
10 19 (18.10%) Data Lake
11 18 (17.14%) Oracle Database
12 14 (13.33%) BusinessObjects
12 14 (13.33%) Pentaho
12 14 (13.33%) SQL Server
13 11 (10.48%) Sqoop
14 10 (9.52%) MongoDB
15 9 (8.57%) MySQL
15 9 (8.57%) Presto
Development Applications
1 49 (46.67%) Git (software)
2 41 (39.05%) Jenkins
3 35 (33.33%) Eclipse
3 35 (33.33%) IntelliJ
4 33 (31.43%) Cucumber
4 33 (31.43%) Cypress.io
4 33 (31.43%) Gradle
4 33 (31.43%) JUnit
4 33 (31.43%) Mockito
4 33 (31.43%) Protractor
4 33 (31.43%) Sonatype Nexus
4 33 (31.43%) Storybook
5 9 (8.57%) Artifactory
6 8 (7.62%) ER Studio
6 8 (7.62%) ERwin
6 8 (7.62%) Maven
6 8 (7.62%) SonarQube
6 8 (7.62%) Subversion
7 4 (3.81%) Bitbucket
8 2 (1.90%) Postman
General
1 44 (41.90%) Finance
2 37 (35.24%) Banking
3 33 (31.43%) Law
4 15 (14.29%) Analytical Skills
5 6 (5.71%) Advertising
5 6 (5.71%) Marketing
5 6 (5.71%) Social Skills
6 2 (1.90%) Back Office
6 2 (1.90%) Presentation Skills
7 1 (0.95%) Pharmaceutical
7 1 (0.95%) Retail
Job Titles
1 26 (24.76%) Data Engineer
2 21 (20.00%) Developer
3 20 (19.05%) Architect
4 19 (18.10%) Software Engineer
5 12 (11.43%) ETL Developer
5 12 (11.43%) Solutions Architect
6 9 (8.57%) Principal Engineer
6 9 (8.57%) Principal Software Engineer
7 8 (7.62%) Administrator
7 8 (7.62%) Lead Developer
7 8 (7.62%) Lead Software Developer
7 8 (7.62%) Lead Software Engineer
7 8 (7.62%) Principal Architect
7 8 (7.62%) Software Developer
8 7 (6.67%) Big Data Engineer
9 6 (5.71%) DBA
9 6 (5.71%) DevOps Engineer
9 6 (5.71%) Scala Engineer
9 6 (5.71%) Senior DBA
10 5 (4.76%) Data Architect
Libraries, Frameworks & Software Standards
1 57 (54.29%) Kafka
2 41 (39.05%) AngularJS
3 38 (36.19%) Spark Streaming
4 35 (33.33%) JSON
4 35 (33.33%) Spring
5 33 (31.43%) CSS
5 33 (31.43%) Express
5 33 (31.43%) Node.js
5 33 (31.43%) React
5 33 (31.43%) Redux
5 33 (31.43%) RESTful
5 33 (31.43%) Tailwind CSS
5 33 (31.43%) Twitter Bootstrap
5 33 (31.43%) WireMock
6 16 (15.24%) LAMP
6 16 (15.24%) LAPP Stack
6 16 (15.24%) Play Framework
7 11 (10.48%) RabbitMQ
8 8 (7.62%) Java EE
8 8 (7.62%) Web Services
Miscellaneous
1 33 (31.43%) Linux Command Line
2 20 (19.05%) Self-Motivation
3 10 (9.52%) User Experience
3 10 (9.52%) Virtual Team
4 6 (5.71%) Replication
5 5 (4.76%) Social Media
6 3 (2.86%) Cloud Native
6 3 (2.86%) Data Structures
7 2 (1.90%) Enterprise Software
8 1 (0.95%) Biotechnology
8 1 (0.95%) Driving Licence
8 1 (0.95%) NHS
Operating Systems
1 47 (44.76%) Unix
2 45 (42.86%) Linux
3 6 (5.71%) Red Hat Enterprise Linux
Processes & Methodologies
1 52 (49.52%) Computer Science
2 50 (47.62%) Agile Software Development
3 44 (41.90%) Software Engineering
4 42 (40.00%) DevOps
5 38 (36.19%) ETL
6 34 (32.38%) Microservices
7 33 (31.43%) Cross-Browser Testing
7 33 (31.43%) Digital Banking
7 33 (31.43%) Validation
8 30 (28.57%) Data Engineering
9 28 (26.67%) Analytics
9 28 (26.67%) Open Source
10 24 (22.86%) Scrum
11 18 (17.14%) Problem-Solving
11 18 (17.14%) TDD
12 17 (16.19%) Business Continuity
12 17 (16.19%) Data Analytics
12 17 (16.19%) High Availability
12 17 (16.19%) System Testing
13 16 (15.24%) BDD
Programming Languages
1 70 (66.67%) Java
2 59 (56.19%) Python
3 50 (47.62%) SQL
4 47 (44.76%) Shell Script
5 33 (31.43%) Groovy
6 21 (20.00%) Scala
7 16 (15.24%) JavaScript
7 16 (15.24%) PL/SQL
8 15 (14.29%) HiveQL
9 12 (11.43%) Bash Shell
10 6 (5.71%) Perl
10 6 (5.71%) PHP
11 3 (2.86%) R
12 1 (0.95%) C#
12 1 (0.95%) C++
12 1 (0.95%) Julia
12 1 (0.95%) T-SQL
Qualifications
1 52 (49.52%) Degree
2 33 (31.43%) OCP
2 33 (31.43%) Oracle Certification
3 8 (7.62%) Computer Science Degree
4 5 (4.76%) BPSS Clearance
5 4 (3.81%) AWS Certification
5 4 (3.81%) Security Cleared
5 4 (3.81%) TOGAF Certification
6 2 (1.90%) SC Cleared
7 1 (0.95%) PhD
Quality Assurance & Compliance
1 3 (2.86%) Data Quality
2 2 (1.90%) QA
3 1 (0.95%) GDPR
System Software
1 25 (23.81%) VMware Infrastructure
2 15 (14.29%) Docker
3 8 (7.62%) Apache ZooKeeper
4 4 (3.81%) Virtual Servers
5 2 (1.90%) Apache Flume
Systems Management
1 47 (44.76%) Ansible
2 20 (19.05%) Opscode Chef
3 16 (15.24%) Hadoop YARN
4 15 (14.29%) Kubernetes
5 12 (11.43%) Puppet
6 10 (9.52%) Terraform
7 8 (7.62%) Oozie
8 6 (5.71%) Graphite
8 6 (5.71%) Nagios
8 6 (5.71%) Opsview
Vendors
1 33 (31.43%) Okta
1 33 (31.43%) VMware
2 29 (27.62%) Cloudera
3 20 (19.05%) Oracle
4 17 (16.19%) Talend
5 16 (15.24%) Informatica
6 14 (13.33%) Databricks
7 13 (12.38%) Google
8 8 (7.62%) Salesforce.com
8 8 (7.62%) SAP
9 6 (5.71%) Facebook
9 6 (5.71%) Marin
10 5 (4.76%) Ab Initio
10 5 (4.76%) Microsoft
11 3 (2.86%) Hortonworks
12 2 (1.90%) Cognos
12 2 (1.90%) MicroStrategy
12 2 (1.90%) Splunk