Period
to 7 July 2020

The following table provides summary statistics for permanent job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited HDFS over the 6 months to 7 July 2020 with a comparison to the same period in the previous 2 years.

Hadoop Distributed File System (HDFS)
UK
6 months to
7 Jul 2020
Same period 2019 Same period 2018
Rank 752 892 760
Rank change year-on-year +140 -132 +199
Permanent jobs citing HDFS 94 217 474
As % of all permanent jobs advertised in the UK 0.12% 0.15% 0.27%
As % of the System Software category 0.68% 0.87% 1.85%
Number of salaries quoted 65 178 354
Median annual salary £75,000 £65,000 £67,500
Median salary % change year-on-year +15.38% -3.70% -6.90%
10th Percentile £46,250 £46,250 £37,500
90th Percentile £105,000 £92,500 £96,250
UK excluding London median annual salary £75,000 £62,500 £60,000
% change year-on-year +20.00% +4.17% +4.35%

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for system software skills.

All System Software Skills
UK
Permanent vacancies with a requirement for system software skills 13,897 24,874 25,637
As % of all permanent IT jobs advertised in the UK 17.67% 16.92% 14.36%
Number of salaries quoted 11,346 20,436 20,798
Median annual salary £52,500 £50,000 £47,500
Median salary % change year-on-year +5.00% +5.26% +11.76%
10th Percentile £28,000 £25,750 £25,000
90th Percentile £87,500 £85,000 £80,000
UK excluding London median annual salary £45,000 £42,500 £41,500
% change year-on-year +5.88% +2.41% +10.67%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a proportion of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Salary Trend

3-month moving average salary quoted in jobs citing HDFS.

Salary trend for HDFS in the UK

HDFS
Salary Histogram

Salary distribution for jobs citing HDFS over the 6 months to 7 July 2020.

Salary histogram for HDFS in the UK

HDFS
Top 12 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing HDFS within the UK over the 6 months to 7 July 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +154 94 £75,000 +15.38% 7
UK excluding London +205 54 £75,000 +20.00% 5
London +51 40 £77,500 +12.73% 2
Midlands +135 31 £75,000 -3.23% 4
East Midlands +82 21 £75,000 -3.23% 4
South East +154 14 £70,000 +40.00% 1
West Midlands +104 10 £52,500 -
North of England +117 4 £55,000 +11.48%
East of England +89 4 £78,750 +12.50%
Yorkshire +102 2 £55,000 -0.01%
North West +101 2 - -
South West +82 1 - -

For the 6 months to 7 July 2020, IT jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for HDFS.

1 94 (100.00%) Hadoop
2 89 (94.68%) Big Data
3 69 (73.40%) Apache Hive
4 65 (69.15%) Java
5 58 (61.70%) Apache Spark
6 56 (59.57%) HBase
6 56 (59.57%) Agile Software Development
7 46 (48.94%) Python
7 46 (48.94%) Analytics
8 45 (47.87%) Impala
9 43 (45.74%) Kafka
9 43 (45.74%) AWS
10 41 (43.62%) Scala
10 41 (43.62%) Software Engineering
11 39 (41.49%) SQL
12 36 (38.30%) Cloudera
13 34 (36.17%) Scrum
14 27 (28.72%) ETL
15 26 (27.66%) Degree
15 26 (27.66%) Data Analytics
15 26 (27.66%) Azure
16 24 (25.53%) Unix
17 23 (24.47%) Hadoop YARN
18 22 (23.40%) Software Delivery
18 22 (23.40%) Oracle
18 22 (23.40%) Finance
18 22 (23.40%) Shell Script
19 21 (22.34%) Data Analysis
19 21 (22.34%) Requirements Gathering
20 20 (21.28%) Computer Science

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 58 (61.70%) Apache Spark
2 13 (13.83%) SAS
3 12 (12.77%) CMS
4 8 (8.51%) Apache
5 4 (4.26%) Apache Pig
5 4 (4.26%) Jupyter
6 2 (2.13%) Elasticsearch
Applications
1 1 (1.06%) Microsoft Excel
1 1 (1.06%) SPSS
Cloud Services
1 43 (45.74%) AWS
2 26 (27.66%) Azure
3 13 (13.83%) Power Platform
4 6 (6.38%) Amazon EMR
5 5 (5.32%) Amazon Kinesis
6 3 (3.19%) Adobe Creative Cloud
7 2 (2.13%) Amazon S3
7 2 (2.13%) Azure Data Factory
7 2 (2.13%) GCP
7 2 (2.13%) Serverless
8 1 (1.06%) Amazon EC2
8 1 (1.06%) GitHub
Communications & Networking
1 12 (12.77%) FTP
2 5 (5.32%) SSL
Database & Business Intelligence
1 94 (100.00%) Hadoop
2 89 (94.68%) Big Data
3 69 (73.40%) Apache Hive
4 56 (59.57%) HBase
5 45 (47.87%) Impala
6 18 (19.15%) Sqoop
7 17 (18.09%) Pentaho
8 15 (15.96%) NoSQL
8 15 (15.96%) Relational Database
9 14 (14.89%) RDBMS
10 13 (13.83%) BusinessObjects
10 13 (13.83%) Power BI
11 12 (12.77%) Data Warehouse
11 12 (12.77%) Oracle Database
12 10 (10.64%) Data Lake
13 8 (8.51%) SQL Server
14 7 (7.45%) Apache Cassandra
15 6 (6.38%) MongoDB
16 5 (5.32%) Amazon Redshift
17 3 (3.19%) Apache Kudu
Development Applications
1 15 (15.96%) Git (software)
2 11 (11.70%) Jenkins
3 7 (7.45%) JUnit
4 4 (4.26%) Artifactory
4 4 (4.26%) GitLab
5 3 (3.19%) JIRA
5 3 (3.19%) Octopus Deploy
5 3 (3.19%) TeamCity
5 3 (3.19%) Visual Studio Team System
6 2 (2.13%) Sonatype Nexus
7 1 (1.06%) Gradle
7 1 (1.06%) IBM UrbanCode
7 1 (1.06%) Maven
General
1 22 (23.40%) Finance
2 11 (11.70%) Retail
3 5 (5.32%) Banking
4 4 (4.26%) Marketing
5 3 (3.19%) Manufacturing
6 2 (2.13%) Public Sector
6 2 (2.13%) Telecoms
Job Titles
1 41 (43.62%) Data Engineer
2 22 (23.40%) Architect
3 21 (22.34%) Data Architect
4 20 (21.28%) Big Data Engineer
5 18 (19.15%) Developer
6 14 (14.89%) Big Data Architect
7 12 (12.77%) Big Data Developer
8 9 (9.57%) Java Developer
9 8 (8.51%) Lead Developer
9 8 (8.51%) Senior Big Data Developer
9 8 (8.51%) Senior Developer
10 7 (7.45%) Analytics Developer
10 7 (7.45%) Lead Java Developer
10 7 (7.45%) Senior Java Developer
11 6 (6.38%) AWS Engineer
12 5 (5.32%) Big Data Software Engineer
12 5 (5.32%) Software Engineer
13 4 (4.26%) Analyst
13 4 (4.26%) AWS Developer
13 4 (4.26%) Consultant
Libraries, Frameworks & Software Standards
1 43 (45.74%) Kafka
2 13 (13.83%) J2EE
3 10 (10.64%) PySpark
4 6 (6.38%) Elastic Stack
5 5 (5.32%) HTML
6 4 (4.26%) ADO
6 4 (4.26%) Spark Streaming
7 3 (3.19%) Web Services
8 2 (2.13%) .NET
8 2 (2.13%) Lambda Expressions
8 2 (2.13%) Pandas
9 1 (1.06%) AngularJS
9 1 (1.06%) Flask
9 1 (1.06%) Keras
9 1 (1.06%) NumPy
9 1 (1.06%) PyTorch
9 1 (1.06%) REST
9 1 (1.06%) scikit-learn
9 1 (1.06%) TensorFlow
9 1 (1.06%) XML
Miscellaneous
1 11 (11.70%) Enterprise Software
2 9 (9.57%) Analytical Skills
3 8 (8.51%) Algorithms
4 7 (7.45%) Public Cloud
5 4 (4.26%) PMI
6 3 (3.19%) Mobile App
7 2 (2.13%) Credit Risk
8 1 (1.06%) FMCG
8 1 (1.06%) Greenfield Project
8 1 (1.06%) Management Information System
Operating Systems
1 24 (25.53%) Unix
2 14 (14.89%) Linux
3 5 (5.32%) Red Hat Enterprise Linux
4 3 (3.19%) Android
4 3 (3.19%) Apple iOS
Processes & Methodologies
1 56 (59.57%) Agile Software Development
2 46 (48.94%) Analytics
3 41 (43.62%) Software Engineering
4 34 (36.17%) Scrum
5 27 (28.72%) ETL
6 26 (27.66%) Data Analytics
7 22 (23.40%) Software Delivery
8 21 (22.34%) Data Analysis
8 21 (22.34%) Requirements Gathering
9 20 (21.28%) Computer Science
9 20 (21.28%) MapReduce
10 19 (20.21%) DevOps
10 19 (20.21%) Problem-Solving
11 18 (19.15%) Mentoring
12 16 (17.02%) Data Development
12 16 (17.02%) Technical Leadership
13 15 (15.96%) Coaching
13 15 (15.96%) Machine Learning
14 14 (14.89%) Process Improvement
15 13 (13.83%) Data Modelling
Programming Languages
1 65 (69.15%) Java
2 46 (48.94%) Python
3 41 (43.62%) Scala
4 39 (41.49%) SQL
5 22 (23.40%) Shell Script
6 17 (18.09%) HiveQL
7 14 (14.89%) JavaScript
7 14 (14.89%) PL/SQL
8 6 (6.38%) Java 8
9 5 (5.32%) R
10 3 (3.19%) Bash Shell
11 1 (1.06%) C
11 1 (1.06%) Groovy
11 1 (1.06%) Korn
Qualifications
1 26 (27.66%) Degree
2 13 (13.83%) Computer Science Degree
3 12 (12.77%) Master's Degree
4 9 (9.57%) BPSS Clearance
5 7 (7.45%) AWS Certification
6 4 (4.26%) PMI Certification
Quality Assurance & Compliance
1 9 (9.57%) Data Quality
2 2 (2.13%) Sarbanes-Oxley
3 1 (1.06%) QA
System Software
1 8 (8.51%) Apache Flume
2 4 (4.26%) Docker
3 2 (2.13%) Apache ZooKeeper
Systems Management
1 23 (24.47%) Hadoop YARN
2 15 (15.96%) Kibana
3 10 (10.64%) Terraform
4 8 (8.51%) Kubernetes
5 7 (7.45%) Ansible
5 7 (7.45%) Oozie
6 3 (3.19%) logstash
6 3 (3.19%) Mesos
6 3 (3.19%) Opscode Chef
6 3 (3.19%) Puppet
7 2 (2.13%) Graphite
7 2 (2.13%) Nagios
7 2 (2.13%) Opsview
Vendors
1 36 (38.30%) Cloudera
2 22 (23.40%) Oracle
3 17 (18.09%) Talend
4 14 (14.89%) Informatica
5 12 (12.77%) IBM
6 11 (11.70%) Hortonworks
7 8 (8.51%) MapR
8 5 (5.32%) Splunk
9 3 (3.19%) Adobe
9 3 (3.19%) Databricks
9 3 (3.19%) Tableau
10 2 (2.13%) Ab Initio
10 2 (2.13%) SolarWinds
10 2 (2.13%) Teradata
11 1 (1.06%) Google
11 1 (1.06%) Microsoft