Hadoop Distributed File System (HDFS)
UK

The following table provides summary statistics for permanent job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited HDFS over the 6 months to 7 December 2023 with a comparison to the same period in the previous 2 years.

6 months to
7 Dec 2023
Same period 2022 Same period 2021
Rank 707 1017 873
Rank change year-on-year +310 -144 -199
Permanent jobs citing HDFS 23 55 208
As % of all permanent jobs advertised in the UK 0.043% 0.045% 0.16%
As % of the System Software category 0.26% 0.29% 0.92%
Number of salaries quoted 3 47 51
10th Percentile £57,000 £54,600 £46,250
25th Percentile £63,750 £57,046 £47,500
Median annual salary (50th Percentile) £80,000 £72,039 £57,500
Median % change year-on-year +11.05% +25.29% -11.54%
75th Percentile £87,500 £123,750 £79,375
90th Percentile £89,000 £125,000 £95,000
UK excluding London median annual salary £55,000 £72,039 £55,000
% change year-on-year -23.65% +30.98% -

All System Software Skills
UK

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for system software skills.

Permanent vacancies with a requirement for system software skills 8,872 18,743 22,601
As % of all permanent jobs advertised in the UK 16.75% 15.26% 17.64%
Number of salaries quoted 7,069 12,536 15,960
10th Percentile £29,000 £29,250 £28,250
25th Percentile £37,000 £38,250 £38,500
Median annual salary (50th Percentile) £55,000 £55,952 £55,000
Median % change year-on-year -1.70% +1.73% -
75th Percentile £72,647 £77,715 £77,500
90th Percentile £90,000 £95,000 £95,000
UK excluding London median annual salary £47,500 £49,787 £47,500
% change year-on-year -4.59% +4.82% +5.56%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a proportion of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Salary Trend

3-month moving average salary quoted in jobs citing HDFS.

Salary trend for HDFS in the UK

HDFS
Salary Histogram

Salary distribution for jobs citing HDFS over the 6 months to 7 December 2023.

Salary histogram for HDFS in the UK

HDFS
Top 6 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing HDFS within the UK over the 6 months to 7 December 2023. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
England +292 14 £80,000 +11.05% 6
UK excluding London +245 14 £55,000 -23.65% 1
London +238 8 £85,000 +11.48% 6
Scotland - 8 - -
South West +73 6 £55,000 -17.43%
Work from Home +220 3 £85,000 +32.81% 2

HDFS
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 19 (82.61%) Apache Spark
2 8 (34.78%) Apache Airflow
3 3 (13.04%) Apache
3 3 (13.04%) Apache Pig
4 1 (4.35%) Apache Solr
Cloud Services
1 14 (60.87%) AWS
2 6 (26.09%) Azure
3 4 (17.39%) Amazon EKS
3 4 (17.39%) Serverless
4 2 (8.70%) GCP
4 2 (8.70%) GitHub
4 2 (8.70%) GitHub Actions
4 2 (8.70%) Power Platform
5 1 (4.35%) Azure Cosmos DB
5 1 (4.35%) Azure Data Factory
5 1 (4.35%) Cloud Computing
Communications & Networking
1 5 (21.74%) Network Security
1 5 (21.74%) WAN
Database & Business Intelligence
1 23 (100.00%) Apache Hive
2 22 (95.65%) Hadoop
3 18 (78.26%) Big Data
4 9 (39.13%) HBase
5 8 (34.78%) Data Warehouse
6 6 (26.09%) NoSQL
7 4 (17.39%) Data Lake
8 3 (13.04%) Flink
9 2 (8.70%) Impala
9 2 (8.70%) Parquet
9 2 (8.70%) Power BI
9 2 (8.70%) Tableau
10 1 (4.35%) Apache Cassandra
10 1 (4.35%) Data Mining
10 1 (4.35%) InterSystems Cache
10 1 (4.35%) PostgreSQL
10 1 (4.35%) RDBMS
10 1 (4.35%) Redis
10 1 (4.35%) Relational Database
10 1 (4.35%) Sqoop
Development Applications
1 8 (34.78%) Jenkins
2 7 (30.43%) Git
3 4 (17.39%) Bitbucket Server
4 3 (13.04%) Checkstyle
4 3 (13.04%) JUnit
4 3 (13.04%) SonarQube
5 2 (8.70%) CircleCI
5 2 (8.70%) Maven
6 1 (4.35%) TeamCity
General
1 11 (47.83%) Finance
2 10 (43.48%) Inclusion and Diversity
3 8 (34.78%) Law
4 6 (26.09%) Social Skills
5 4 (17.39%) Legal
5 4 (17.39%) Marketing
6 3 (13.04%) Banking
6 3 (13.04%) Retail Banking
7 2 (8.70%) Analytical Skills
8 1 (4.35%) Investment Banking
8 1 (4.35%) Manufacturing
Job Titles
1 8 (34.78%) Data Engineer
1 8 (34.78%) Software Engineer
2 7 (30.43%) Developer
3 5 (21.74%) Java Engineer
4 4 (17.39%) Database Developer
4 4 (17.39%) Database Engineer
4 4 (17.39%) Developer/Software Engineer
4 4 (17.39%) Java Software Engineer
4 4 (17.39%) Python Engineer
4 4 (17.39%) Python Software Engineer
5 3 (13.04%) Lead
5 3 (13.04%) Lead Developer
5 3 (13.04%) Technical Developer
5 3 (13.04%) Technical Leader
6 2 (8.70%) Administrator
6 2 (8.70%) Big Data Specialist
6 2 (8.70%) Senior
6 2 (8.70%) Senior Data Engineer
7 1 (4.35%) Analytics Manager
7 1 (4.35%) Big Data Engineer
Libraries, Frameworks & Software Standards
1 10 (43.48%) Kafka
2 7 (30.43%) Spring
3 6 (26.09%) Apache NiFi
4 5 (21.74%) JDBC
4 5 (21.74%) ODBC
4 5 (21.74%) OLE DB
4 5 (21.74%) Regular Expression
5 4 (17.39%) JPA
5 4 (17.39%) React
5 4 (17.39%) REST
6 3 (13.04%) J2EE
6 3 (13.04%) Spring Boot
7 2 (8.70%) Apache Avro
7 2 (8.70%) Apache Beam
7 2 (8.70%) PySpark
7 2 (8.70%) Spark Streaming
8 1 (4.35%) Apache Tez
8 1 (4.35%) JSON
8 1 (4.35%) JSON Schema
Miscellaneous
1 5 (21.74%) Algorithms
1 5 (21.74%) Cyber Defence
2 4 (17.39%) Legacy Systems
2 4 (17.39%) Management Information System
3 3 (13.04%) Client/Server
3 3 (13.04%) Product Ownership
3 3 (13.04%) Self-Motivation
4 2 (8.70%) Distributed Applications
4 2 (8.70%) Distributed Systems
4 2 (8.70%) Greenfield Project
5 1 (4.35%) IoT
5 1 (4.35%) Market Risk
5 1 (4.35%) Reference Data
Operating Systems
1 9 (39.13%) Unix
2 6 (26.09%) Linux
3 5 (21.74%) Red Hat Enterprise Linux
Processes & Methodologies
1 14 (60.87%) Agile
2 13 (56.52%) CI/CD
2 13 (56.52%) Open Source
2 13 (56.52%) SDLC
3 11 (47.83%) Machine Learning
3 11 (47.83%) Microservices
4 10 (43.48%) Software Engineering
4 10 (43.48%) Visualisation
5 9 (39.13%) Analytics
6 8 (34.78%) Artificial Intelligence
6 8 (34.78%) Data Engineering
6 8 (34.78%) ETL
6 8 (34.78%) Operational Stability
6 8 (34.78%) Risk Management
7 6 (26.09%) Actionable Insight
7 6 (26.09%) Continuous Improvement
8 5 (21.74%) Data Management
8 5 (21.74%) Network Design
8 5 (21.74%) Performance Optimisation
8 5 (21.74%) Wealth Management
Programming Languages
1 19 (82.61%) Python
2 16 (69.57%) SQL
3 15 (65.22%) Java
4 5 (21.74%) Bash
5 4 (17.39%) PL/SQL
6 3 (13.04%) Scala
7 1 (4.35%) Perl
Qualifications
1 6 (26.09%) SC Cleared
1 6 (26.09%) Security Cleared
2 1 (4.35%) Azure Certification
2 1 (4.35%) CCA
2 1 (4.35%) Degree
2 1 (4.35%) Master's Degree
2 1 (4.35%) MBA
2 1 (4.35%) Microsoft Certification
Quality Assurance & Compliance
1 3 (13.04%) QA
System Software
1 1 (4.35%) Apache Flume
1 1 (4.35%) Docker
Systems Management
1 6 (26.09%) Oozie
2 4 (17.39%) Terraform
3 2 (8.70%) Apache Ranger
3 2 (8.70%) Hadoop YARN
4 1 (4.35%) Ansible
4 1 (4.35%) Kibana
Vendors
1 11 (47.83%) Cloudera
2 10 (43.48%) Databricks
3 5 (21.74%) Red Hat
3 5 (21.74%) Splunk
4 4 (17.39%) Informatica
5 1 (4.35%) Microsoft