Period
to 24 July 2021

The following table provides summary statistics for permanent job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited HDFS over the 6 months to 24 July 2021 with a comparison to the same period in the previous 2 years.

Hadoop Distributed File System (HDFS)
UK
6 months to
24 Jul 2021
Same period 2020 Same period 2019
Rank 762 711 917
Rank change year-on-year -51 +206 -156
Permanent jobs citing HDFS 229 89 203
As % of all permanent jobs advertised in the UK 0.20% 0.13% 0.14%
As % of the System Software category 1.09% 0.71% 0.82%
Number of salaries quoted 47 57 167
10th Percentile £49,350 £45,750 £46,250
25th Percentile £53,042 £60,500 £56,250
Median annual salary (50th Percentile) £65,000 £75,000 £65,000
Median % change year-on-year -13.33% +15.38% -7.14%
75th Percentile £79,875 £95,000 £80,000
90th Percentile £95,000 £105,000 £92,500
UK excluding London median annual salary £56,914 £75,000 £61,000
% change year-on-year -24.12% +22.95% +6.09%

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for system software skills.

All System Software Skills
UK
Permanent vacancies with a requirement for system software skills 21,000 12,592 24,627
As % of all permanent IT jobs advertised in the UK 18.56% 17.79% 17.03%
Number of salaries quoted 14,928 10,475 20,484
10th Percentile £29,750 £29,000 £25,750
25th Percentile £40,000 £38,000 £33,500
Median annual salary (50th Percentile) £55,000 £52,500 £50,000
Median % change year-on-year +4.76% +5.00% +5.26%
75th Percentile £75,000 £72,500 £70,000
90th Percentile £92,500 £87,500 £85,350
UK excluding London median annual salary £47,500 £45,000 £42,500
% change year-on-year +5.56% +5.88% -

HDFS
Job Vacancy Trend

Job postings citing HDFS as a proportion of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Salary Trend

3-month moving average salary quoted in jobs citing HDFS.

Salary trend for HDFS in the UK

HDFS
Salary Histogram

Salary distribution for jobs citing HDFS over the 6 months to 24 July 2021.

Salary histogram for HDFS in the UK

HDFS
Top 13 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing HDFS within the UK over the 6 months to 24 July 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -40 228 £65,000 -13.33% 20
UK excluding London +10 180 £56,914 -24.12% 16
South East +65 164 £65,000 -7.14% 2
London -110 46 £70,000 -11.11% 8
Work from Home - 16 £65,000 -
Midlands -91 10 £55,500 -26.00% 6
East Midlands -60 8 £55,500 -44.50% 3
North of England -60 5 £56,914 +3.48% 3
Yorkshire -56 2 - - 3
West Midlands -66 2 £58,750 +11.90% 3
North East - 2 £56,914 -
East of England -35 1 - -
North West -36 1 - -

For the 6 months to 24 July 2021, IT jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for HDFS.

1 205 (89.52%) Java
2 196 (85.59%) Apache Spark
3 187 (81.66%) HBase
4 185 (80.79%) Kafka
4 185 (80.79%) Computer Science
5 182 (79.48%) Agile Software Development
6 181 (79.04%) Degree
7 177 (77.29%) Git (software)
8 175 (76.42%) Banking
9 174 (75.98%) Finance
9 174 (75.98%) JSON
10 173 (75.55%) Unix
11 172 (75.11%) Shell Script
11 172 (75.11%) DevOps
12 170 (74.24%) Jenkins
12 170 (74.24%) Spring
12 170 (74.24%) Spark Streaming
13 169 (73.80%) Eclipse
13 169 (73.80%) IntelliJ
13 169 (73.80%) Linux
14 167 (72.93%) Ansible
15 166 (72.49%) AngularJS
15 166 (72.49%) Apache
15 166 (72.49%) Linux Command Line
16 165 (72.05%) Software Engineering
17 164 (71.62%) Microservices
18 163 (71.18%) RESTful
19 162 (70.74%) Law
19 162 (70.74%) Digital Banking
19 162 (70.74%) Inclusion and Diversity

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 196 (85.59%) Apache Spark
2 166 (72.49%) Apache
3 161 (70.31%) GemFire
4 8 (3.49%) Elasticsearch
5 5 (2.18%) SAS
6 4 (1.75%) Cloud Foundry
7 2 (0.87%) Apache Airflow
8 1 (0.44%) Apache Pig
8 1 (0.44%) BizTalk Server
Business Applications
1 1 (0.44%) Dynamics CRM
Cloud Services
1 161 (70.31%) GitHub
2 124 (54.15%) AWS
3 102 (44.54%) Amazon Kinesis
4 100 (43.67%) Azure
5 89 (38.86%) GCP
6 35 (15.28%) OpenShift
7 11 (4.80%) Power Platform
8 9 (3.93%) Amazon S3
8 9 (3.93%) Azure Synapse Analytics
9 8 (3.49%) PaaS
10 6 (2.62%) Azure Data Factory
11 4 (1.75%) AWS CloudFormation
11 4 (1.75%) Cloud Computing
11 4 (1.75%) SaaS
12 3 (1.31%) Amazon EMR
13 2 (0.87%) AWS Data Pipeline
13 2 (0.87%) IaaS
14 1 (0.44%) Amazon EC2
14 1 (0.44%) Amazon ELB
14 1 (0.44%) Snowflake
Communications & Networking
1 8 (3.49%) FTP
1 8 (3.49%) SMS
2 7 (3.06%) Internet
3 4 (1.75%) Sourcefire
Database & Business Intelligence
1 187 (81.66%) HBase
2 161 (70.31%) MariaDB
3 159 (69.43%) PostgreSQL
4 157 (68.56%) Redis
5 143 (62.45%) NoSQL
6 136 (59.39%) Relational Database
7 131 (57.21%) Big Data
8 70 (30.57%) Hadoop
9 59 (25.76%) Apache Hive
10 16 (6.99%) Impala
11 15 (6.55%) Data Lake
12 11 (4.80%) Power BI
13 10 (4.37%) DB2
13 10 (4.37%) Oracle Database
13 10 (4.37%) SQL Server
14 9 (3.93%) Metadata
14 9 (3.93%) Spark SQL
15 8 (3.49%) Azure SQL Database
15 8 (3.49%) MongoDB
15 8 (3.49%) SAP HANA
Development Applications
1 177 (77.29%) Git (software)
2 170 (74.24%) Jenkins
3 169 (73.80%) Eclipse
3 169 (73.80%) IntelliJ
4 161 (70.31%) Cucumber
4 161 (70.31%) Cypress.io
4 161 (70.31%) Gradle
4 161 (70.31%) JUnit
4 161 (70.31%) Protractor
4 161 (70.31%) Sonatype Nexus
4 161 (70.31%) Storybook
5 157 (68.56%) Mockito
6 102 (44.54%) Artifactory
7 63 (27.51%) SonarQube
8 8 (3.49%) ER Studio
8 8 (3.49%) ERwin
8 8 (3.49%) IBM UrbanCode
8 8 (3.49%) Postman
8 8 (3.49%) SoapUI
8 8 (3.49%) Subversion
General
1 175 (76.42%) Banking
2 174 (75.98%) Finance
3 162 (70.74%) Inclusion and Diversity
3 162 (70.74%) Law
4 14 (6.11%) Analytical Skills
5 9 (3.93%) Social Skills
6 6 (2.62%) Marketing
7 3 (1.31%) Advertising
7 3 (1.31%) Retail Banking
7 3 (1.31%) Telecoms
8 1 (0.44%) Documentation Skills
8 1 (0.44%) Pharmaceutical
8 1 (0.44%) Presentation Skills
8 1 (0.44%) Retail
Job Titles
1 130 (56.77%) Software Engineer
2 66 (28.82%) Principal Engineer
2 66 (28.82%) Principal Software Engineer
3 46 (20.09%) Developer
3 46 (20.09%) Lead Software Engineer
4 31 (13.54%) Lead Developer
4 31 (13.54%) Lead Software Developer
4 31 (13.54%) Software Developer
5 17 (7.42%) Architect
6 16 (6.99%) Data Engineer
7 12 (5.24%) ETL Developer
8 11 (4.80%) Senior Software Engineer
9 10 (4.37%) Solutions Architect
10 8 (3.49%) DevOps Engineer
10 8 (3.49%) Principal Architect
11 5 (2.18%) Data Architect
12 4 (1.75%) Big Data Consultant
12 4 (1.75%) Consultant
12 4 (1.75%) Data Consultant
12 4 (1.75%) Security Consultant
Libraries, Frameworks & Software Standards
1 185 (80.79%) Kafka
2 174 (75.98%) JSON
3 170 (74.24%) Spark Streaming
3 170 (74.24%) Spring
4 166 (72.49%) AngularJS
5 163 (71.18%) RESTful
6 161 (70.31%) CSS
6 161 (70.31%) Express
6 161 (70.31%) Node.js
6 161 (70.31%) React
6 161 (70.31%) Redux
6 161 (70.31%) Tailwind CSS
6 161 (70.31%) Twitter Bootstrap
7 157 (68.56%) WireMock
8 144 (62.88%) Play Framework
9 111 (48.47%) LAMP
9 111 (48.47%) LAPP Stack
10 106 (46.29%) RabbitMQ
11 28 (12.23%) Web Services
12 22 (9.61%) Java EE
Miscellaneous
1 166 (72.49%) Linux Command Line
2 77 (33.62%) User Experience
3 55 (24.02%) Self-Motivation
4 17 (7.42%) Virtual Team
5 6 (2.62%) Cloud Native
6 5 (2.18%) Social Media
7 4 (1.75%) Cyber Defence
7 4 (1.75%) Data Structures
7 4 (1.75%) Internet of Things
8 3 (1.31%) Cybercrime
8 3 (1.31%) Cyberthreat
8 3 (1.31%) Online Banking
8 3 (1.31%) Replication
9 2 (0.87%) Equities
10 1 (0.44%) Apple TV
10 1 (0.44%) Biotechnology
10 1 (0.44%) Driving Licence
10 1 (0.44%) NHS
Operating Systems
1 173 (75.55%) Unix
2 169 (73.80%) Linux
3 4 (1.75%) Windows
Processes & Methodologies
1 185 (80.79%) Computer Science
2 182 (79.48%) Agile Software Development
3 172 (75.11%) DevOps
4 165 (72.05%) Software Engineering
5 164 (71.62%) Microservices
6 162 (70.74%) Digital Banking
7 161 (70.31%) Cross-Browser Testing
7 161 (70.31%) Validation
8 121 (52.84%) Business Continuity
9 105 (45.85%) High Availability
10 102 (44.54%) Clean Code
10 102 (44.54%) OO
10 102 (44.54%) Open Source
10 102 (44.54%) Performance Monitoring
10 102 (44.54%) Software Testing
11 100 (43.67%) Multithreading
12 98 (42.79%) TDD
13 96 (41.92%) BDD
13 96 (41.92%) Scaled Agile Framework
14 92 (40.17%) Multithreaded Programming
Programming Languages
1 205 (89.52%) Java
2 172 (75.11%) Shell Script
3 161 (70.31%) Groovy
4 145 (63.32%) Python
5 54 (23.58%) SQL
6 18 (7.86%) Scala
7 13 (5.68%) PL/SQL
8 11 (4.80%) JavaScript
9 7 (3.06%) Perl
10 6 (2.62%) Bash Shell
10 6 (2.62%) HiveQL
11 3 (1.31%) PHP
12 2 (0.87%) C++
12 2 (0.87%) R
13 1 (0.44%) C#
13 1 (0.44%) Julia
13 1 (0.44%) T-SQL
Qualifications
1 181 (79.04%) Degree
2 157 (68.56%) OCP
2 157 (68.56%) Oracle Certification
3 20 (8.73%) Computer Science Degree
4 7 (3.06%) Security Cleared
5 4 (1.75%) AWS Certification
5 4 (1.75%) SC Cleared
5 4 (1.75%) TOGAF Certification
6 3 (1.31%) DV Cleared
7 2 (0.87%) PhD
Quality Assurance & Compliance
1 3 (1.31%) SLA
2 2 (0.87%) QA
3 1 (0.44%) Data Quality
3 1 (0.44%) GDPR
System Software
1 106 (46.29%) VMware Infrastructure
2 13 (5.68%) Docker
3 11 (4.80%) Apache ZooKeeper
4 4 (1.75%) Virtual Servers
5 1 (0.44%) Apache Flume
Systems Management
1 167 (72.93%) Ansible
2 45 (19.65%) Opscode Chef
3 13 (5.68%) Kubernetes
4 9 (3.93%) Hadoop YARN
5 6 (2.62%) Puppet
6 4 (1.75%) Terraform
Vendors
1 161 (70.31%) Okta
1 161 (70.31%) VMware
2 21 (9.17%) Cloudera
3 13 (5.68%) Oracle
4 12 (5.24%) Databricks
4 12 (5.24%) Splunk
5 9 (3.93%) Google
6 8 (3.49%) Salesforce.com
6 8 (3.49%) SAP
7 7 (3.06%) Talend
8 6 (2.62%) Informatica
9 5 (2.18%) Ab Initio
10 4 (1.75%) Cisco
10 4 (1.75%) Microsoft
11 3 (1.31%) Facebook
11 3 (1.31%) Hortonworks
11 3 (1.31%) Marin
12 1 (0.44%) Apple
12 1 (0.44%) Cognos
12 1 (0.44%) MicroStrategy