Period
to 28 October 2021

The following table provides summary statistics for permanent job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited HDFS over the 6 months to 28 October 2021 with a comparison to the same period in the previous 2 years.

Hadoop Distributed File System (HDFS)
UK
6 months to
28 Oct 2021
Same period 2020 Same period 2019
Rank 808 614 853
Rank change year-on-year -194 +239 -40
Permanent jobs citing HDFS 246 88 224
As % of all permanent jobs advertised in the UK 0.19% 0.18% 0.16%
As % of the System Software category 1.08% 0.96% 0.95%
Number of salaries quoted 49 43 205
10th Percentile £47,500 £45,250 £47,600
25th Percentile £48,750 £50,500 £56,550
Median annual salary (50th Percentile) £60,000 £70,000 £72,500
Median % change year-on-year -14.29% -3.45% +3.57%
75th Percentile £80,000 £95,000 £83,750
90th Percentile £96,500 £110,500 £92,000
UK excluding London median annual salary £56,914 £55,500 £58,100
% change year-on-year +2.55% -4.48% +1.04%

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for system software skills.

All System Software Skills
UK
Permanent vacancies with a requirement for system software skills 22,681 9,156 23,536
As % of all permanent IT jobs advertised in the UK 17.84% 18.45% 17.18%
Number of salaries quoted 15,553 7,862 19,698
10th Percentile £28,500 £30,000 £25,500
25th Percentile £38,750 £40,000 £32,500
Median annual salary (50th Percentile) £55,000 £55,000 £50,000
Median % change year-on-year - +10.00% -
75th Percentile £77,500 £72,500 £70,000
90th Percentile £95,000 £86,250 £85,000
UK excluding London median annual salary £47,500 £45,000 £42,500
% change year-on-year +5.56% +5.88% -

HDFS
Job Vacancy Trend

Job postings citing HDFS as a proportion of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Salary Trend

3-month moving average salary quoted in jobs citing HDFS.

Salary trend for HDFS in the UK

HDFS
Salary Histogram

Salary distribution for jobs citing HDFS over the 6 months to 28 October 2021.

Salary histogram for HDFS in the UK

HDFS
Top 11 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing HDFS within the UK over the 6 months to 28 October 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -184 244 £60,000 -19.19% 43
UK excluding London -79 193 £56,914 +2.55% 27
South East 0 184 £72,500 +3.57% 6
London -254 49 £60,000 -33.33% 17
Work from Home -257 7 £56,914 -30.38% 5
Midlands -135 5 £55,000 +3.77% 14
West Midlands -95 4 £55,000 +10.00% 13
North of England -132 4 £56,914 +8.41% 3
North East - 2 £56,914 - 1
Yorkshire - 2 - - 2
East Midlands -87 1 £65,000 +17.12% 1

For the 6 months to 28 October 2021, IT jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for HDFS.

1 227 (92.28%) Java
2 215 (87.40%) Apache Spark
3 213 (86.59%) Agile Software Development
4 208 (84.55%) Software Engineering
5 207 (84.15%) Unix
6 204 (82.93%) HBase
7 201 (81.71%) Kafka
8 197 (80.08%) DevOps
9 196 (79.67%) Computer Science
10 195 (79.27%) JSON
10 195 (79.27%) Jenkins
11 192 (78.05%) Git (software)
11 192 (78.05%) Banking
11 192 (78.05%) PostgreSQL
11 192 (78.05%) Finance
12 191 (77.64%) Spring
13 190 (77.24%) Linux
14 189 (76.83%) Shell Script
15 187 (76.02%) IntelliJ
15 187 (76.02%) Eclipse
16 186 (75.61%) Degree
16 186 (75.61%) Linux Command Line
17 184 (74.80%) Apache
17 184 (74.80%) Ansible
17 184 (74.80%) Microservices
18 182 (73.98%) Spark Streaming
18 182 (73.98%) Inclusion and Diversity
19 180 (73.17%) Digital Banking
19 180 (73.17%) Law
19 180 (73.17%) RESTful

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 215 (87.40%) Apache Spark
2 184 (74.80%) Apache
3 178 (72.36%) GemFire
4 18 (7.32%) SAS
5 10 (4.07%) Elasticsearch
6 7 (2.85%) Cloud Foundry
7 3 (1.22%) Confluence
8 1 (0.41%) Apache Pig
Applications
1 1 (0.41%) Chatbot
1 1 (0.41%) Microsoft Excel
Cloud Services
1 178 (72.36%) GitHub
2 142 (57.72%) AWS
3 124 (50.41%) Amazon Kinesis
4 112 (45.53%) Azure
4 112 (45.53%) GCP
5 40 (16.26%) OpenShift
6 7 (2.85%) PaaS
7 6 (2.44%) Amazon EMR
7 6 (2.44%) Power Platform
8 4 (1.63%) Amazon EC2
8 4 (1.63%) Amazon ELB
8 4 (1.63%) Amazon S3
9 2 (0.81%) AWS Data Pipeline
9 2 (0.81%) Azure Cosmos DB
9 2 (0.81%) Azure Data Factory
9 2 (0.81%) Azure Synapse Analytics
9 2 (0.81%) Tricentis Tosca
10 1 (0.41%) Azure Machine Learning
10 1 (0.41%) Cloud Computing
Communications & Networking
1 10 (4.07%) Internet
2 9 (3.66%) SMS
3 4 (1.63%) Sourcefire
Database & Business Intelligence
1 204 (82.93%) HBase
2 192 (78.05%) PostgreSQL
3 175 (71.14%) MariaDB
4 171 (69.51%) Redis
5 165 (67.07%) NoSQL
6 158 (64.23%) Relational Database
7 142 (57.72%) Big Data
8 73 (29.67%) Hadoop
9 59 (23.98%) Apache Hive
10 25 (10.16%) Impala
11 23 (9.35%) MySQL
12 22 (8.94%) Data Warehouse
13 18 (7.32%) Amazon Redshift
13 18 (7.32%) Pentaho
14 11 (4.47%) DB2
15 9 (3.66%) OLTP
16 7 (2.85%) Apache Cassandra
16 7 (2.85%) Data Lake
16 7 (2.85%) MongoDB
17 6 (2.44%) Power BI
Development Applications
1 195 (79.27%) Jenkins
2 192 (78.05%) Git (software)
3 187 (76.02%) Eclipse
3 187 (76.02%) IntelliJ
4 178 (72.36%) Cucumber
4 178 (72.36%) Cypress.io
4 178 (72.36%) Gradle
4 178 (72.36%) JUnit
4 178 (72.36%) Protractor
4 178 (72.36%) Sonatype Nexus
4 178 (72.36%) Storybook
5 171 (69.51%) Mockito
6 124 (50.41%) Artifactory
7 83 (33.74%) SonarQube
8 9 (3.66%) IBM UrbanCode
8 9 (3.66%) Maven
8 9 (3.66%) Postman
8 9 (3.66%) SoapUI
9 7 (2.85%) Appium
10 4 (1.63%) GitLab
General
1 192 (78.05%) Banking
1 192 (78.05%) Finance
2 182 (73.98%) Inclusion and Diversity
3 180 (73.17%) Law
4 15 (6.10%) Public Sector
5 6 (2.44%) Retail Banking
5 6 (2.44%) Social Skills
6 3 (1.22%) Retail
6 3 (1.22%) Telecoms
7 2 (0.81%) Analytical Skills
7 2 (0.81%) Financial Institution
7 2 (0.81%) Marketing
8 1 (0.41%) Documentation Skills
8 1 (0.41%) Legal
8 1 (0.41%) Presentation Skills
Job Titles
1 149 (60.57%) Software Engineer
2 76 (30.89%) Principal Engineer
2 76 (30.89%) Principal Software Engineer
3 56 (22.76%) Lead Software Engineer
4 46 (18.70%) Developer
5 40 (16.26%) Software Developer
6 39 (15.85%) Lead Developer
6 39 (15.85%) Lead Software Developer
7 31 (12.60%) Data Engineer
8 12 (4.88%) Informatica Engineer
9 11 (4.47%) Senior Software Engineer
10 10 (4.07%) DevOps Engineer
11 6 (2.44%) Java Engineer
11 6 (2.44%) Senior Java Engineer
12 5 (2.03%) Big Data Engineer
12 5 (2.03%) Cloud Engineer
12 5 (2.03%) Senior Data Engineer
13 4 (1.63%) Big Data Consultant
13 4 (1.63%) Cybersecurity Consultant
13 4 (1.63%) Hadoop Consultant
Libraries, Frameworks & Software Standards
1 201 (81.71%) Kafka
2 195 (79.27%) JSON
3 191 (77.64%) Spring
4 182 (73.98%) Spark Streaming
5 180 (73.17%) RESTful
6 178 (72.36%) AngularJS
6 178 (72.36%) CSS
6 178 (72.36%) Express
6 178 (72.36%) Node.js
6 178 (72.36%) React
6 178 (72.36%) Redux
6 178 (72.36%) Tailwind CSS
6 178 (72.36%) Twitter Bootstrap
7 176 (71.54%) Play Framework
8 171 (69.51%) WireMock
9 131 (53.25%) RabbitMQ
10 128 (52.03%) LAMP
10 128 (52.03%) LAPP Stack
11 29 (11.79%) Web Services
12 24 (9.76%) Java EE
Miscellaneous
1 186 (75.61%) Linux Command Line
2 88 (35.77%) User Experience
3 54 (21.95%) Self-Motivation
4 16 (6.50%) Virtual Team
5 6 (2.44%) Cloud Native
5 6 (2.44%) Online Banking
6 4 (1.63%) Cyber Defence
6 4 (1.63%) Internet of Things
7 3 (1.22%) Algorithms
7 3 (1.22%) Cybercrime
7 3 (1.22%) Cyberthreat
7 3 (1.22%) Data Structures
8 2 (0.81%) Equities
8 2 (0.81%) Social Network
9 1 (0.41%) Apple TV
9 1 (0.41%) Data Centre
9 1 (0.41%) Hybrid Cloud
9 1 (0.41%) Social Media
Operating Systems
1 207 (84.15%) Unix
2 190 (77.24%) Linux
3 5 (2.03%) Windows
Processes & Methodologies
1 213 (86.59%) Agile Software Development
2 208 (84.55%) Software Engineering
3 197 (80.08%) DevOps
4 196 (79.67%) Computer Science
5 184 (74.80%) Microservices
6 180 (73.17%) Digital Banking
7 178 (72.36%) Cross-Browser Testing
7 178 (72.36%) Validation
8 146 (59.35%) Business Continuity
9 139 (56.50%) Software Testing
10 135 (54.88%) Full Stack Development
11 127 (51.63%) OO
12 126 (51.22%) Scrum
13 124 (50.41%) Clean Code
13 124 (50.41%) High Availability
13 124 (50.41%) Performance Monitoring
14 123 (50.00%) Multithreading
15 119 (48.37%) Scaled Agile Framework
16 118 (47.97%) BDD
16 118 (47.97%) Open Source
Programming Languages
1 227 (92.28%) Java
2 189 (76.83%) Shell Script
3 178 (72.36%) Groovy
4 171 (69.51%) Python
5 55 (22.36%) SQL
6 27 (10.98%) PL/SQL
7 18 (7.32%) HiveQL
8 15 (6.10%) Scala
9 5 (2.03%) Perl
9 5 (2.03%) R
10 4 (1.63%) JavaScript
11 3 (1.22%) Bash Shell
12 1 (0.41%) C#
12 1 (0.41%) C++
12 1 (0.41%) VBA
Qualifications
1 186 (75.61%) Degree
2 171 (69.51%) OCP
2 171 (69.51%) Oracle Certification
3 16 (6.50%) Computer Science Degree
4 8 (3.25%) Security Cleared
5 5 (2.03%) SC Cleared
6 3 (1.22%) DV Cleared
7 2 (0.81%) PhD
Quality Assurance & Compliance
1 3 (1.22%) SLA
2 1 (0.41%) Data Quality
System Software
1 123 (50.00%) VMware Infrastructure
2 15 (6.10%) Docker
3 10 (4.07%) Apache ZooKeeper
4 1 (0.41%) Apache Flume
Systems Management
1 184 (74.80%) Ansible
2 45 (18.29%) Opscode Chef
3 12 (4.88%) Kubernetes
4 3 (1.22%) Hadoop YARN
4 3 (1.22%) Puppet
5 1 (0.41%) Kubeflow
Vendors
1 178 (72.36%) Okta
1 178 (72.36%) VMware
2 28 (11.38%) Oracle
3 19 (7.72%) Informatica
3 19 (7.72%) Talend
4 13 (5.28%) Splunk
5 6 (2.44%) Cloudera
6 4 (1.63%) Ab Initio
6 4 (1.63%) Cisco
7 3 (1.22%) Microsoft
8 2 (0.81%) Databricks
9 1 (0.41%) Apple
9 1 (0.41%) Cognos
9 1 (0.41%) MicroStrategy
9 1 (0.41%) Teradata