Hadoop Distributed File System (HDFS)
UK

The following table provides summary statistics for permanent job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited HDFS over the 6 months to 27 April 2024 with a comparison to the same period in the previous 2 years.

6 months to
27 Apr 2024
Same period 2023 Same period 2022
Rank 771 927 1119
Rank change year-on-year +156 +192 -305
Permanent jobs citing HDFS 125 43 113
As % of all permanent jobs advertised in the UK 0.13% 0.041% 0.073%
As % of the System Software category 1.02% 0.27% 0.40%
Number of salaries quoted 2 33 44
10th Percentile - £50,000 £46,250
25th Percentile £29,000 £51,250 £48,750
Median annual salary (50th Percentile) £31,000 £68,823 £63,750
Median % change year-on-year -54.96% +7.96% +6.25%
75th Percentile £33,000 £100,000 £95,000
90th Percentile - £125,000 £117,750
UK excluding London median annual salary - £68,823 £57,500
% change year-on-year - +19.69% +3.60%

All System Software Skills
UK

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for system software skills.

Permanent vacancies with a requirement for system software skills 12,249 16,035 28,223
As % of all permanent jobs advertised in the UK 12.49% 15.32% 18.31%
Number of salaries quoted 9,715 10,596 15,794
10th Percentile £26,250 £28,000 £27,825
25th Percentile £32,500 £37,500 £38,750
Median annual salary (50th Percentile) £50,000 £55,000 £56,325
Median % change year-on-year -9.09% -2.35% +2.41%
75th Percentile £67,500 £77,500 £80,000
90th Percentile £86,000 £95,000 £95,000
UK excluding London median annual salary £45,000 £47,500 £50,000
% change year-on-year -5.26% -5.00% +5.26%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a proportion of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Salary Trend

3-month moving average salary quoted in jobs citing HDFS.

Salary trend for HDFS in the UK

HDFS
Top 6 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing HDFS within the UK over the 6 months to 27 April 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
UK excluding London +106 106 - -
Scotland - 102 - -
England +93 22 £31,000 -54.96%
London +121 18 £31,000 -51.87%
South East - 4 - -
Work from Home +70 2 - - 1

HDFS
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 116 (92.80%) Apache Spark
2 24 (19.20%) Apache Airflow
3 8 (6.40%) Apache
3 8 (6.40%) Apache Pig
Cloud Services
1 124 (99.20%) AWS
2 18 (14.40%) Azure
3 16 (12.80%) Amazon EKS
4 10 (8.00%) Serverless
5 8 (6.40%) GCP
5 8 (6.40%) GitHub
5 8 (6.40%) GitHub Actions
6 4 (3.20%) Amazon EMR
6 4 (3.20%) Amazon S3
7 2 (1.60%) AWS Lambda
7 2 (1.60%) Snowflake
8 1 (0.80%) Power Platform
Database & Business Intelligence
1 123 (98.40%) Apache Hive
2 102 (81.60%) HBase
3 95 (76.00%) Hadoop
4 94 (75.20%) NoSQL
5 56 (44.80%) Big Data
6 9 (7.20%) Data Warehouse
7 8 (6.40%) Data Lake
7 8 (6.40%) Flink
7 8 (6.40%) Parquet
8 4 (3.20%) Impala
9 3 (2.40%) Sqoop
10 2 (1.60%) Amazon Athena
10 2 (1.60%) Amazon Redshift
11 1 (0.80%) Power BI
11 1 (0.80%) PowerCenter
11 1 (0.80%) Tableau
Development Applications
1 94 (75.20%) Git
1 94 (75.20%) Jenkins
2 86 (68.80%) Bitbucket Server
3 8 (6.40%) Checkstyle
3 8 (6.40%) CircleCI
3 8 (6.40%) JUnit
3 8 (6.40%) SonarQube
4 1 (0.80%) ERwin
General
1 110 (88.00%) Finance
1 110 (88.00%) Inclusion and Diversity
2 102 (81.60%) Law
3 83 (66.40%) Legal
4 19 (15.20%) Marketing
5 11 (8.80%) Banking
6 8 (6.40%) Retail Banking
7 5 (4.00%) Analytical Skills
8 4 (3.20%) Social Skills
9 2 (1.60%) Telecoms
Job Titles
1 102 (81.60%) Software Engineer
2 69 (55.20%) AWS Engineer
3 67 (53.60%) AWS Software Engineer
4 66 (52.80%) Developer
5 56 (44.80%) Database Developer
5 56 (44.80%) Database Engineer
5 56 (44.80%) Developer/Software Engineer
6 40 (32.00%) Data Engineer
7 37 (29.60%) AWS Database Engineer
7 37 (29.60%) AWS Developer
8 30 (24.00%) Big Data Engineer
8 30 (24.00%) Big Data Software Engineer
9 17 (13.60%) Java Engineer
10 16 (12.80%) Java Software Engineer
11 14 (11.20%) Python Engineer
12 13 (10.40%) Python Software Engineer
13 10 (8.00%) Lead
14 8 (6.40%) Senior
14 8 (6.40%) Senior Data Engineer
14 8 (6.40%) Technical Developer
Libraries, Frameworks & Software Standards
1 24 (19.20%) Spring
2 16 (12.80%) JPA
2 16 (12.80%) React
2 16 (12.80%) REST
3 13 (10.40%) Kafka
4 8 (6.40%) Apache Avro
4 8 (6.40%) Apache Beam
4 8 (6.40%) J2EE
4 8 (6.40%) PySpark
4 8 (6.40%) Spark Streaming
4 8 (6.40%) Spring Boot
5 2 (1.60%) Apache NiFi
5 2 (1.60%) Elastic Stack
Miscellaneous
1 87 (69.60%) Algorithms
2 8 (6.40%) Client/Server
2 8 (6.40%) Distributed Applications
2 8 (6.40%) Distributed Systems
2 8 (6.40%) Greenfield Project
2 8 (6.40%) Product Ownership
2 8 (6.40%) Self-Motivation
3 7 (5.60%) Legacy Systems
3 7 (5.60%) Management Information System
Operating Systems
1 102 (81.60%) Unix
2 2 (1.60%) Linux
Processes & Methodologies
1 120 (96.00%) SDLC
2 118 (94.40%) Agile
2 118 (94.40%) CI/CD
3 112 (89.60%) Machine Learning
4 110 (88.00%) Software Engineering
5 103 (82.40%) Artificial Intelligence
6 102 (81.60%) Operational Stability
7 94 (75.20%) Continuous Improvement
7 94 (75.20%) Open Source
8 90 (72.00%) Risk Management
9 87 (69.60%) Visualisation
10 86 (68.80%) Batch Processing
10 86 (68.80%) Community of Practice
10 86 (68.80%) TDD
10 86 (68.80%) Test Automation
11 80 (64.00%) Microservices
12 36 (28.80%) Wealth Management
13 17 (13.60%) Decision-Making
14 14 (11.20%) ETL
15 12 (9.60%) Data Modelling
Programming Languages
1 121 (96.80%) SQL
2 116 (92.80%) Python
3 93 (74.40%) Java
4 86 (68.80%) PL/SQL
5 11 (8.80%) Scala
6 2 (1.60%) C#
Qualifications
1 5 (4.00%) Degree
2 3 (2.40%) Master's Degree
3 2 (1.60%) Computer Science Degree
4 1 (0.80%) AWS Certification
4 1 (0.80%) AWS Certified Solutions Architect
4 1 (0.80%) MBA
Quality Assurance & Compliance
1 8 (6.40%) QA
2 2 (1.60%) Data Quality
System Software
1 2 (1.60%) Docker
Systems Management
1 86 (68.80%) Terraform
Vendors
1 110 (88.00%) Databricks
2 90 (72.00%) Informatica
3 84 (67.20%) Cloudera
4 2 (1.60%) Apple
4 2 (1.60%) Facebook
4 2 (1.60%) Google
4 2 (1.60%) Intel
4 2 (1.60%) Microsoft