Period
to 24 November 2020

The following table provides summary statistics for permanent job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited HDFS over the 6 months to 24 November 2020 with a comparison to the same period in the previous 2 years.

Hadoop Distributed File System (HDFS)
UK
6 months to
24 Nov 2020
Same period 2019 Same period 2018
Rank 621 850 826
Rank change year-on-year +229 -24 +160
Permanent jobs citing HDFS 88 209 335
As % of all permanent jobs advertised in the UK 0.16% 0.16% 0.21%
As % of the System Software category 0.86% 0.91% 1.37%
Number of salaries quoted 48 201 267
Median annual salary £67,500 £75,000 £70,000
Median salary % change year-on-year -10.00% +7.14% -
10th Percentile £46,950 £49,337 £48,350
90th Percentile £100,500 £91,250 £97,500
UK excluding London median annual salary £55,500 £59,050 £58,000
% change year-on-year -6.01% +1.81% +0.87%

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for system software skills.

All System Software Skills
UK
Permanent vacancies with a requirement for system software skills 10,195 23,023 24,496
As % of all permanent IT jobs advertised in the UK 18.65% 17.18% 15.19%
Number of salaries quoted 8,719 18,939 19,991
Median annual salary £55,000 £50,000 £50,000
Median salary % change year-on-year +10.00% - +11.11%
10th Percentile £29,750 £25,625 £25,000
90th Percentile £86,250 £85,000 £81,250
UK excluding London median annual salary £45,500 £42,500 £42,500
% change year-on-year +7.06% - +6.25%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a proportion of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Salary Trend

3-month moving average salary quoted in jobs citing HDFS.

Salary trend for HDFS in the UK

HDFS
Salary Histogram

Salary distribution for jobs citing HDFS over the 6 months to 24 November 2020.

Salary histogram for HDFS in the UK

HDFS
Top 13 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing HDFS within the UK over the 6 months to 24 November 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +234 85 £67,500 -10.00% 17
London +152 58 £90,000 +17.65% 5
UK excluding London +232 29 £55,500 -6.01% 12
Midlands +122 16 £53,000 -15.20% 1
East Midlands +92 15 £55,500 -17.78%
South East +186 7 £65,000 +36.84% 9
Work from Home -20 4 £65,000 -13.33% 1
North of England +119 3 £55,000 -2.03% 1
Yorkshire +94 2 £55,000 -2.03% 1
Scotland +72 2 £53,000 +4.33%
East of England +114 1 £90,000 +20.00% 1
West Midlands +75 1 £50,000 -11.11% 1
North West - 1 £52,500 -

For the 6 months to 24 November 2020, IT jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for HDFS.

1 87 (98.86%) Hadoop
2 78 (88.64%) Apache Hive
2 78 (88.64%) Big Data
3 71 (80.68%) ETL
4 65 (73.86%) SQL
5 63 (71.59%) Java
6 62 (70.45%) Cloudera
7 59 (67.05%) Python
8 58 (65.91%) Impala
9 57 (64.77%) Analytics
9 57 (64.77%) AWS
10 52 (59.09%) Oracle
11 50 (56.82%) Agile Software Development
11 50 (56.82%) Data Analytics
12 47 (53.41%) Azure
12 47 (53.41%) Unix
13 46 (52.27%) Talend
14 45 (51.14%) Requirements Gathering
15 44 (50.00%) Shell Script
16 43 (48.86%) Informatica
17 42 (47.73%) Scrum
17 42 (47.73%) Software Engineering
17 42 (47.73%) Software Delivery
17 42 (47.73%) Power BI
17 42 (47.73%) Power Platform
18 41 (46.59%) Oracle Database
19 40 (45.45%) Mentoring
19 40 (45.45%) Coaching
20 39 (44.32%) HiveQL
20 39 (44.32%) BusinessObjects

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 38 (43.18%) SAS
2 25 (28.41%) Apache Spark
3 5 (5.68%) Apache
4 3 (3.41%) Apache Pig
5 2 (2.27%) Confluence
5 2 (2.27%) Elasticsearch
6 1 (1.14%) BizTalk Server
6 1 (1.14%) CMS
6 1 (1.14%) Mule
6 1 (1.14%) WebLogic
Business Applications
1 1 (1.14%) Dynamics CRM
1 1 (1.14%) SAP FI-CA
1 1 (1.14%) SAP PSCD
Cloud Services
1 57 (64.77%) AWS
2 47 (53.41%) Azure
3 42 (47.73%) Power Platform
4 5 (5.68%) IaaS
4 5 (5.68%) PaaS
5 3 (3.41%) Amazon EMR
5 3 (3.41%) GCP
5 3 (3.41%) SaaS
6 2 (2.27%) Amazon S3
6 2 (2.27%) Azure Data Factory
7 1 (1.14%) Amazon Cognito
7 1 (1.14%) Amazon EC2
7 1 (1.14%) Amazon Kinesis
7 1 (1.14%) Amazon SQS
7 1 (1.14%) AWS CloudFormation
7 1 (1.14%) AWS Lambda
7 1 (1.14%) GitHub
7 1 (1.14%) Serverless
Communications & Networking
1 2 (2.27%) Firewall
1 2 (2.27%) HTTP
1 2 (2.27%) HTTPS
1 2 (2.27%) Kerberos
1 2 (2.27%) SSL
2 1 (1.14%) FTP
2 1 (1.14%) Network Security
Database & Business Intelligence
1 87 (98.86%) Hadoop
2 78 (88.64%) Apache Hive
2 78 (88.64%) Big Data
3 58 (65.91%) Impala
4 42 (47.73%) Power BI
5 41 (46.59%) Oracle Database
6 39 (44.32%) BusinessObjects
7 38 (43.18%) Pentaho
8 34 (38.64%) HBase
9 26 (29.55%) Data Lake
10 16 (18.18%) Spark SQL
11 15 (17.05%) RDBMS
11 15 (17.05%) Relational Database
12 14 (15.91%) Sqoop
13 12 (13.64%) SQL Server
14 10 (11.36%) NoSQL
15 8 (9.09%) MySQL
16 6 (6.82%) Snowflake Schema
17 4 (4.55%) MongoDB
17 4 (4.55%) Tableau
Development Applications
1 8 (9.09%) Git (software)
2 6 (6.82%) Jenkins
3 2 (2.27%) JIRA
3 2 (2.27%) JUnit
4 1 (1.14%) Cucumber
4 1 (1.14%) Gradle
4 1 (1.14%) Oracle Forms
4 1 (1.14%) Postman
4 1 (1.14%) Selenium
4 1 (1.14%) SoapUI
4 1 (1.14%) Sonatype Nexus
4 1 (1.14%) Toad
General
1 19 (21.59%) Finance
2 5 (5.68%) Banking
3 2 (2.27%) Retail
4 1 (1.14%) Financial Institution
4 1 (1.14%) Front Office
4 1 (1.14%) Games
4 1 (1.14%) Investment Banking
4 1 (1.14%) Manufacturing
4 1 (1.14%) Pharmaceutical
4 1 (1.14%) Public Sector
4 1 (1.14%) Telecoms
Job Titles
1 48 (54.55%) Data Engineer
2 22 (25.00%) Developer
3 14 (15.91%) Architect
4 12 (13.64%) Data Architect
5 10 (11.36%) ETL Developer
6 9 (10.23%) Big Data Developer
6 9 (10.23%) Senior Developer
7 5 (5.68%) Ab Initio Developer
7 5 (5.68%) Senior ETL Developer
8 4 (4.55%) AWS Engineer
8 4 (4.55%) Big Data Engineer
8 4 (4.55%) DevOps Engineer
8 4 (4.55%) Senior Big Data Developer
9 3 (3.41%) Apache Spark Developer
9 3 (3.41%) AWS Architect
9 3 (3.41%) Kubernetes Engineer
10 2 (2.27%) AWS Developer
10 2 (2.27%) Azure Engineer
10 2 (2.27%) Hadoop Architect
10 2 (2.27%) Senior Hadoop Developer
Libraries, Frameworks & Software Standards
1 14 (15.91%) Kafka
2 8 (9.09%) J2EE
3 7 (7.95%) Spring
4 5 (5.68%) Spark Streaming
5 2 (2.27%) .NET
5 2 (2.27%) Elastic Stack
5 2 (2.27%) LDAP
5 2 (2.27%) OAuth
5 2 (2.27%) PySpark
5 2 (2.27%) RabbitMQ
5 2 (2.27%) SAML
5 2 (2.27%) TensorFlow
5 2 (2.27%) Web Services
6 1 (1.14%) AngularJS
6 1 (1.14%) ASP.NET MVC
6 1 (1.14%) NumPy
6 1 (1.14%) Pandas
6 1 (1.14%) REST
6 1 (1.14%) RESTful
6 1 (1.14%) scikit-learn
Miscellaneous
1 11 (12.50%) Analytical Skills
2 4 (4.55%) Credit Risk
3 3 (3.41%) Social Media
4 2 (2.27%) Cloud Native
5 1 (1.14%) Algorithms
5 1 (1.14%) Data Structures
5 1 (1.14%) Driving Licence
5 1 (1.14%) FMCG
5 1 (1.14%) Hedge funds
5 1 (1.14%) Reference Data
5 1 (1.14%) Self-Motivation
Operating Systems
1 47 (53.41%) Unix
2 8 (9.09%) Linux
3 3 (3.41%) Red Hat Enterprise Linux
4 1 (1.14%) Windows
Processes & Methodologies
1 71 (80.68%) ETL
2 57 (64.77%) Analytics
3 50 (56.82%) Agile Software Development
3 50 (56.82%) Data Analytics
4 45 (51.14%) Requirements Gathering
5 42 (47.73%) Scrum
5 42 (47.73%) Software Delivery
5 42 (47.73%) Software Engineering
6 40 (45.45%) Coaching
6 40 (45.45%) Mentoring
7 38 (43.18%) Process Improvement
8 19 (21.59%) Business Intelligence
9 18 (20.45%) Data Modelling
10 15 (17.05%) Data Governance
11 13 (14.77%) Computer Science
11 13 (14.77%) Data Engineering
11 13 (14.77%) DevOps
12 10 (11.36%) Continuous Integration
12 10 (11.36%) ETL Development
13 9 (10.23%) Test Automation
Programming Languages
1 65 (73.86%) SQL
2 63 (71.59%) Java
3 59 (67.05%) Python
4 44 (50.00%) Shell Script
5 39 (44.32%) HiveQL
5 39 (44.32%) JavaScript
5 39 (44.32%) PL/SQL
6 11 (12.50%) Scala
7 5 (5.68%) Bash Shell
8 2 (2.27%) Java 8
8 2 (2.27%) R
9 1 (1.14%) C#
9 1 (1.14%) C++
9 1 (1.14%) Groovy
9 1 (1.14%) Korn
Qualifications
1 30 (34.09%) BPSS Clearance
2 13 (14.77%) Degree
3 3 (3.41%) AWS Certification
4 2 (2.27%) SC Cleared
4 2 (2.27%) Security Cleared
5 1 (1.14%) Computer Science Degree
5 1 (1.14%) HND
5 1 (1.14%) ISEB
5 1 (1.14%) ISTQB
5 1 (1.14%) ISTQB Foundation Certification
5 1 (1.14%) Master's Degree
5 1 (1.14%) Sun Certification
Quality Assurance & Compliance
1 15 (17.05%) Data Quality
2 5 (5.68%) Sarbanes-Oxley
3 1 (1.14%) QA
System Software
1 16 (18.18%) Apache ZooKeeper
2 6 (6.82%) Docker
3 1 (1.14%) Apache Flume
3 1 (1.14%) Tivoli Workload Scheduler
Systems Management
1 27 (30.68%) Hadoop YARN
2 9 (10.23%) Kubernetes
3 6 (6.82%) Terraform
4 5 (5.68%) Ansible
5 2 (2.27%) Ambari
5 2 (2.27%) Graphite
5 2 (2.27%) Kibana
5 2 (2.27%) Nagios
5 2 (2.27%) Oozie
5 2 (2.27%) Opscode Chef
5 2 (2.27%) Opsview
5 2 (2.27%) Puppet
5 2 (2.27%) Single Sign-On
6 1 (1.14%) logstash
6 1 (1.14%) Mesos
6 1 (1.14%) Prometheus
6 1 (1.14%) Tivoli
Vendors
1 62 (70.45%) Cloudera
2 52 (59.09%) Oracle
3 46 (52.27%) Talend
4 43 (48.86%) Informatica
5 13 (14.77%) Ab Initio
6 11 (12.50%) Teradata
7 6 (6.82%) Hortonworks
8 5 (5.68%) Databricks
9 4 (4.55%) Microsoft
10 3 (3.41%) MapR
11 2 (2.27%) Cognos
12 1 (1.14%) Google
12 1 (1.14%) IBM
12 1 (1.14%) MicroStrategy
12 1 (1.14%) Netezza
12 1 (1.14%) SAP
12 1 (1.14%) TIBCO