Period
to 22 September 2020

The following table provides summary statistics for permanent job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited HDFS over the 6 months to 22 September 2020 with a comparison to the same period in the previous 2 years.

Hadoop Distributed File System (HDFS)
UK
6 months to
22 Sep 2020
Same period 2019 Same period 2018
Rank 616 857 764
Rank change year-on-year +241 -93 +218
Permanent jobs citing HDFS 91 225 427
As % of all permanent jobs advertised in the UK 0.18% 0.16% 0.26%
As % of the System Software category 0.98% 0.93% 1.74%
Number of salaries quoted 44 197 342
Median annual salary £83,750 £70,000 £70,000
Median salary % change year-on-year +19.64% - -
10th Percentile £45,000 £46,250 £41,215
90th Percentile £109,375 £92,500 £97,500
UK excluding London median annual salary £70,000 £57,275 £58,000
% change year-on-year +22.22% -1.25% +3.11%

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for system software skills.

All System Software Skills
UK
Permanent vacancies with a requirement for system software skills 9,300 24,106 24,514
As % of all permanent IT jobs advertised in the UK 18.67% 17.33% 14.72%
Number of salaries quoted 7,790 19,719 20,029
Median annual salary £55,000 £50,000 £49,039
Median salary % change year-on-year +10.00% +1.96% +8.97%
10th Percentile £30,750 £25,500 £24,750
90th Percentile £87,500 £85,000 £81,250
UK excluding London median annual salary £46,660 £42,500 £42,500
% change year-on-year +9.79% - +10.39%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a proportion of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Salary Trend

3-month moving average salary quoted in jobs citing HDFS.

Salary trend for HDFS in the UK

HDFS
Salary Histogram

Salary distribution for jobs citing HDFS over the 6 months to 22 September 2020.

Salary histogram for HDFS in the UK

HDFS
Top 8 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing HDFS within the UK over the 6 months to 22 September 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +267 90 £83,750 +19.64% 20
London +151 55 £90,000 +20.00% 10
UK excluding London +296 35 £70,000 +22.22% 13
Midlands +171 23 £100,000 +53.85% 8
East Midlands +109 15 £100,000 +33.33% 5
South East +211 11 £70,000 +47.37% 2
West Midlands +110 8 £50,000 -11.11% 3
East of England +120 1 £90,000 +28.57%

For the 6 months to 22 September 2020, IT jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for HDFS.

1 90 (98.90%) Hadoop
2 84 (92.31%) Big Data
3 77 (84.62%) Java
4 71 (78.02%) Apache Hive
5 67 (73.63%) Agile Software Development
6 60 (65.93%) Cloudera
7 58 (63.74%) SQL
8 57 (62.64%) Python
8 57 (62.64%) Analytics
9 55 (60.44%) Impala
9 55 (60.44%) ETL
10 54 (59.34%) Scrum
10 54 (59.34%) Software Engineering
11 52 (57.14%) AWS
12 46 (50.55%) Requirements Gathering
13 45 (49.45%) Data Analytics
13 45 (49.45%) Oracle
13 45 (49.45%) Unix
14 42 (46.15%) Talend
15 41 (45.05%) Shell Script
15 41 (45.05%) Azure
16 40 (43.96%) HBase
16 40 (43.96%) Software Delivery
17 39 (42.86%) HiveQL
17 39 (42.86%) Pentaho
18 38 (41.76%) Mentoring
18 38 (41.76%) Coaching
18 38 (41.76%) Apache Spark
19 37 (40.66%) Process Improvement
19 37 (40.66%) Oracle Database

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 38 (41.76%) Apache Spark
2 35 (38.46%) SAS
3 11 (12.09%) CMS
4 5 (5.49%) Apache
5 4 (4.40%) Jupyter
6 1 (1.10%) Apache Pig
6 1 (1.10%) Confluence
6 1 (1.10%) Elasticsearch
Cloud Services
1 52 (57.14%) AWS
2 41 (45.05%) Azure
3 35 (38.46%) Power Platform
4 5 (5.49%) IaaS
4 5 (5.49%) PaaS
5 3 (3.30%) SaaS
6 2 (2.20%) Amazon EMR
7 1 (1.10%) Amazon Cognito
7 1 (1.10%) Amazon EC2
7 1 (1.10%) Amazon Kinesis
7 1 (1.10%) Amazon SQS
7 1 (1.10%) AWS CloudFormation
7 1 (1.10%) AWS Lambda
7 1 (1.10%) Azure Data Factory
7 1 (1.10%) GCP
7 1 (1.10%) GitHub
7 1 (1.10%) Serverless
Communications & Networking
1 11 (12.09%) FTP
2 6 (6.59%) SSL
3 2 (2.20%) Firewall
3 2 (2.20%) HTTP
3 2 (2.20%) HTTPS
3 2 (2.20%) Kerberos
4 1 (1.10%) Network Security
Database & Business Intelligence
1 90 (98.90%) Hadoop
2 84 (92.31%) Big Data
3 71 (78.02%) Apache Hive
4 55 (60.44%) Impala
5 40 (43.96%) HBase
6 39 (42.86%) Pentaho
7 37 (40.66%) Oracle Database
8 35 (38.46%) BusinessObjects
8 35 (38.46%) Power BI
9 17 (18.68%) Sqoop
10 14 (15.38%) Data Lake
11 13 (14.29%) RDBMS
11 13 (14.29%) Relational Database
12 10 (10.99%) NoSQL
13 8 (8.79%) SQL Server
14 6 (6.59%) Spark SQL
15 4 (4.40%) Apache Cassandra
15 4 (4.40%) Flink
15 4 (4.40%) MongoDB
15 4 (4.40%) MySQL
Development Applications
1 11 (12.09%) Git (software)
2 5 (5.49%) Jenkins
3 4 (4.40%) Artifactory
3 4 (4.40%) GitLab
3 4 (4.40%) JUnit
4 1 (1.10%) Gradle
4 1 (1.10%) JIRA
4 1 (1.10%) Sonatype Nexus
General
1 14 (15.38%) Finance
2 5 (5.49%) Banking
3 1 (1.10%) Front Office
3 1 (1.10%) Games
3 1 (1.10%) Investment Banking
3 1 (1.10%) Manufacturing
3 1 (1.10%) Pharmaceutical
3 1 (1.10%) Retail
3 1 (1.10%) Telecoms
Job Titles
1 51 (56.04%) Data Engineer
2 21 (23.08%) Architect
2 21 (23.08%) Data Architect
3 17 (18.68%) Developer
4 14 (15.38%) Big Data Engineer
5 11 (12.09%) Big Data Architect
6 8 (8.79%) Big Data Developer
7 6 (6.59%) Software Engineer
8 5 (5.49%) Apache Spark Developer
8 5 (5.49%) Kubernetes Engineer
9 4 (4.40%) AWS Developer
9 4 (4.40%) AWS Engineer
9 4 (4.40%) Big Data Software Engineer
9 4 (4.40%) Cloud Engineer
9 4 (4.40%) ETL Developer
9 4 (4.40%) Scala Developer
9 4 (4.40%) Senior Developer
10 3 (3.30%) Consultant
10 3 (3.30%) Senior Architect
10 3 (3.30%) Senior Data Architect
Libraries, Frameworks & Software Standards
1 31 (34.07%) Kafka
2 9 (9.89%) Spark Streaming
3 8 (8.79%) J2EE
4 6 (6.59%) PySpark
5 5 (5.49%) Spring
6 4 (4.40%) ADO
6 4 (4.40%) Elastic Stack
6 4 (4.40%) HTML
7 2 (2.20%) LDAP
7 2 (2.20%) OAuth
7 2 (2.20%) SAML
7 2 (2.20%) TensorFlow
8 1 (1.10%) .NET
8 1 (1.10%) AngularJS
8 1 (1.10%) Django
8 1 (1.10%) Keras
8 1 (1.10%) Pandas
8 1 (1.10%) RESTful
8 1 (1.10%) scikit-learn
8 1 (1.10%) Spring Boot
Miscellaneous
1 10 (10.99%) Analytical Skills
2 4 (4.40%) Credit Risk
2 4 (4.40%) Enterprise Software
2 4 (4.40%) PMI
3 3 (3.30%) Social Media
4 1 (1.10%) Algorithms
4 1 (1.10%) FMCG
4 1 (1.10%) Hedge funds
4 1 (1.10%) Self-Motivation
Operating Systems
1 45 (49.45%) Unix
2 9 (9.89%) Linux
3 3 (3.30%) Red Hat Enterprise Linux
Processes & Methodologies
1 67 (73.63%) Agile Software Development
2 57 (62.64%) Analytics
3 55 (60.44%) ETL
4 54 (59.34%) Scrum
4 54 (59.34%) Software Engineering
5 46 (50.55%) Requirements Gathering
6 45 (49.45%) Data Analytics
7 40 (43.96%) Software Delivery
8 38 (41.76%) Coaching
8 38 (41.76%) Mentoring
9 37 (40.66%) Process Improvement
10 22 (24.18%) Computer Science
11 21 (23.08%) DevOps
12 20 (21.98%) Problem-Solving
13 15 (16.48%) Business Intelligence
14 14 (15.38%) Data Governance
15 13 (14.29%) Data Modelling
16 12 (13.19%) Data Analysis
17 11 (12.09%) Content Management
17 11 (12.09%) Data Development
Programming Languages
1 77 (84.62%) Java
2 58 (63.74%) SQL
3 57 (62.64%) Python
4 41 (45.05%) Shell Script
5 39 (42.86%) HiveQL
6 37 (40.66%) PL/SQL
7 36 (39.56%) JavaScript
8 26 (28.57%) Scala
9 5 (5.49%) Java 8
10 3 (3.30%) Bash Shell
11 1 (1.10%) C#
11 1 (1.10%) C++
11 1 (1.10%) Groovy
11 1 (1.10%) Korn
Qualifications
1 26 (28.57%) BPSS Clearance
2 23 (25.27%) Degree
3 11 (12.09%) Computer Science Degree
3 11 (12.09%) Master's Degree
4 4 (4.40%) PMI Certification
5 3 (3.30%) AWS Certification
6 1 (1.10%) HND
Quality Assurance & Compliance
1 13 (14.29%) Data Quality
2 5 (5.49%) Sarbanes-Oxley
System Software
1 6 (6.59%) Apache ZooKeeper
2 4 (4.40%) Docker
3 2 (2.20%) Apache Flume
4 1 (1.10%) Tivoli Workload Scheduler
Systems Management
1 19 (20.88%) Hadoop YARN
2 14 (15.38%) Kibana
3 9 (9.89%) Kubernetes
4 6 (6.59%) Oozie
4 6 (6.59%) Terraform
5 3 (3.30%) Ansible
5 3 (3.30%) logstash
6 2 (2.20%) Ambari
6 2 (2.20%) Single Sign-On
7 1 (1.10%) Mesos
7 1 (1.10%) Prometheus
7 1 (1.10%) Tivoli
Vendors
1 60 (65.93%) Cloudera
2 45 (49.45%) Oracle
3 42 (46.15%) Talend
4 37 (40.66%) Informatica
5 11 (12.09%) IBM
6 9 (9.89%) Hortonworks
7 7 (7.69%) Ab Initio
7 7 (7.69%) Teradata
8 5 (5.49%) MapR
9 4 (4.40%) Splunk
10 2 (2.20%) Databricks
11 1 (1.10%) Microsoft