Period
to 25 January 2020

The following table provides summary statistics for permanent job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited HDFS over the 6 months to 25 January 2020 with a comparison to the same period in the previous 2 years.

Hadoop Distributed File System (HDFS)
UK
6 months to
25 Jan 2020
Same period 2019 Same period 2018
Rank 842 806 924
Rank change year-on-year -36 +118 +4
Permanent jobs citing HDFS 184 311 270
As % of all permanent IT jobs advertised in the UK 0.15% 0.20% 0.16%
As % of the System Software category 0.86% 1.32% 1.18%
Number of salaries quoted 170 246 238
UK median annual salary £75,000 £70,000 £70,000
Median salary % change year-on-year +7.14% - -3.45%
10th Percentile £51,000 £49,337 £45,000
90th Percentile £90,000 £97,500 £102,875
UK excluding London median annual salary £65,000 £55,000 £58,750
% change year-on-year +18.18% -6.38% -2.08%

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for system software skills.

All System Software Skills
UK
Permanent vacancies with a requirement for system software skills 21,326 23,515 22,848
As % of all permanent IT jobs advertised in the UK 17.07% 15.50% 13.12%
Number of salaries quoted 17,395 19,177 18,992
UK median annual salary £50,000 £50,000 £45,000
Median salary % change year-on-year - +11.11% +5.88%
10th Percentile £25,500 £25,500 £24,000
90th Percentile £85,000 £82,500 £75,000
UK excluding London median annual salary £42,500 £42,500 £40,000
% change year-on-year - +6.25% +6.67%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a percentage of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing HDFS.

Salary trend for HDFS in the UK

HDFS
Salary Histogram

The salary distribution of IT jobs citing HDFS over the 6 months to 25 January 2020.

Salary histogram for HDFS in the UK

HDFS
Top 13 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing HDFS within the UK over the 6 months to 25 January 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -17 174 £75,000 +7.14% 26
London -3 112 £77,000 +2.67% 10
UK excluding London +40 72 £65,000 +18.18% 21
North of England +37 22 £55,000 +11.48% 1
East of England +68 18 £75,000 +3.45% 2
Yorkshire +38 15 £55,000 +10.73% 1
Midlands +55 14 £75,000 - 3
East Midlands +30 13 £75,000 - 3
Scotland +43 10 £50,800 -24.74%
North East +11 7 £54,671 +10.81%
South East +43 4 £62,500 +25.00% 8
South West +10 4 £75,000 +50.00% 2
West Midlands +38 1 £65,000 -

For the 6 months to 25 January 2020, IT jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for HDFS.

1 175 (95.11%) Hadoop
2 159 (86.41%) Big Data
3 156 (84.78%) Apache Hive
4 132 (71.74%) Apache Spark
5 114 (61.96%) Python
6 97 (52.72%) Azure
7 96 (52.17%) Agile Software Development
8 93 (50.54%) AWS
9 90 (48.91%) SQL
10 80 (43.48%) Data Modelling
11 79 (42.93%) Impala
12 75 (40.76%) Data Engineering
13 74 (40.22%) Scala
14 71 (38.59%) Linux
15 68 (36.96%) HBase
16 67 (36.41%) NoSQL
17 66 (35.87%) Java
17 66 (35.87%) Business Intelligence
18 65 (35.33%) Amazon S3
19 60 (32.61%) Data Science
20 56 (30.43%) Amazon EMR
21 52 (28.26%) Hadoop YARN
22 51 (27.72%) Apache
23 49 (26.63%) Kafka
24 48 (26.09%) SQL Server
25 46 (25.00%) Analytics
26 44 (23.91%) Data Warehouse
26 44 (23.91%) ETL
27 39 (21.20%) Software Engineering
27 39 (21.20%) HDInsight

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 132 (71.74%) Apache Spark
2 51 (27.72%) Apache
3 20 (10.87%) Apache Airflow
4 10 (5.43%) Elasticsearch
5 8 (4.35%) SAS
6 7 (3.80%) Tomcat
6 7 (3.80%) WebLogic
7 5 (2.72%) Apache Pig
8 1 (0.54%) Apache Solr
8 1 (0.54%) OpenStack
8 1 (0.54%) OpenWhisk
Applications
1 6 (3.26%) Chatbot
2 1 (0.54%) Microsoft Project
Business Applications
1 1 (0.54%) Adobe Campaign
1 1 (0.54%) Oracle Applications
Cloud Services
1 97 (52.72%) Azure
2 93 (50.54%) AWS
3 65 (35.33%) Amazon S3
4 56 (30.43%) Amazon EMR
5 29 (15.76%) GCP
6 20 (10.87%) Amazon Kinesis
6 20 (10.87%) IaaS
6 20 (10.87%) PaaS
7 17 (9.24%) Azure Data Factory
8 8 (4.35%) Cloud Computing
8 8 (4.35%) OpenShift
8 8 (4.35%) SaaS
9 6 (3.26%) AWS CloudFormation
10 5 (2.72%) Serverless
11 2 (1.09%) Adobe Creative Cloud
11 2 (1.09%) Power Platform
12 1 (0.54%) Amazon EC2
12 1 (0.54%) AWS Lambda
Communications & Networking
1 17 (9.24%) Firewall
2 13 (7.07%) Kerberos
3 12 (6.52%) VPN
4 7 (3.80%) HTTP
4 7 (3.80%) HTTPS
4 7 (3.80%) SSL
5 6 (3.26%) DNS
6 1 (0.54%) Broadband
6 1 (0.54%) IPFS
Database & Business Intelligence
1 175 (95.11%) Hadoop
2 159 (86.41%) Big Data
3 156 (84.78%) Apache Hive
4 79 (42.93%) Impala
5 68 (36.96%) HBase
6 67 (36.41%) NoSQL
7 48 (26.09%) SQL Server
8 44 (23.91%) Data Warehouse
9 39 (21.20%) HDInsight
10 29 (15.76%) Sqoop
11 23 (12.50%) Data Lake
12 21 (11.41%) Neo4j
12 21 (11.41%) PostgreSQL
12 21 (11.41%) Presto
13 20 (10.87%) Apache Kudu
14 19 (10.33%) Amazon Redshift
15 18 (9.78%) Data Mining
16 17 (9.24%) Amazon DynamoDB
17 16 (8.70%) BigQuery
18 15 (8.15%) SQL Server Analysis Services
Development Applications
1 14 (7.61%) ProC
2 10 (5.43%) Git (software)
3 9 (4.89%) Jenkins
4 2 (1.09%) JIRA
5 1 (0.54%) Bitbucket
5 1 (0.54%) Eclipse
5 1 (0.54%) IBM UrbanCode
5 1 (0.54%) IntelliJ
5 1 (0.54%) JUnit
5 1 (0.54%) Maven
5 1 (0.54%) NetBeans
5 1 (0.54%) Octopus Deploy
5 1 (0.54%) Selenium
5 1 (0.54%) Sonatype Nexus
5 1 (0.54%) Team Foundation Server
5 1 (0.54%) TeamCity
5 1 (0.54%) Visual Studio Team System
General
1 23 (12.50%) Finance
2 12 (6.52%) Games
2 12 (6.52%) Public Sector
3 11 (5.98%) Legal
4 8 (4.35%) Retail
5 4 (2.17%) Banking
6 3 (1.63%) Marketing
Job Titles
1 96 (52.17%) Data Engineer
2 43 (23.37%) Architect
3 37 (20.11%) Big Data Engineer
4 33 (17.93%) Senior Data Engineer
5 32 (17.39%) Data Architect
6 28 (15.22%) Cloud Architect
7 17 (9.24%) Data Scientist
8 16 (8.70%) Developer
9 15 (8.15%) Senior Data Scientist
10 13 (7.07%) DevOps Engineer
10 13 (7.07%) Java Developer
11 11 (5.98%) Big Data Developer
11 11 (5.98%) Hadoop Architect
12 10 (5.43%) Senior Developer
12 10 (5.43%) Senior Java Developer
12 10 (5.43%) Technical Architect
13 9 (4.89%) Cloud Engineer
13 9 (4.89%) Senior Big Data Developer
14 8 (4.35%) Analytics Developer
14 8 (4.35%) Senior DevOps
Libraries, Frameworks & Software Standards
1 49 (26.63%) Kafka
2 21 (11.41%) LAPP Stack
3 17 (9.24%) Lambda Expressions
4 13 (7.07%) D3.js
4 13 (7.07%) ggplot2
4 13 (7.07%) OAuth
5 12 (6.52%) Elastic Stack
5 12 (6.52%) SAML
6 11 (5.98%) LDAP
7 9 (4.89%) J2EE
8 7 (3.80%) Web Services
9 6 (3.26%) Middleware
10 5 (2.72%) .NET
11 3 (1.63%) Akka
11 3 (1.63%) Spring
12 2 (1.09%) PySpark
12 2 (1.09%) RabbitMQ
13 1 (0.54%) Natural Language Processing
13 1 (0.54%) Node.js
13 1 (0.54%) REST
Miscellaneous
1 31 (16.85%) Analytical Skills
2 26 (14.13%) Algorithms
3 17 (9.24%) Public Cloud
4 12 (6.52%) Data Centre
5 9 (4.89%) Cloud Native
5 9 (4.89%) Management Information System
6 8 (4.35%) Enterprise Software
7 6 (3.26%) Self-Motivation
8 4 (2.17%) Greenfield Project
9 3 (1.63%) Derivative
10 2 (1.09%) Equities
10 2 (1.09%) Fixed Income
10 2 (1.09%) Hybrid Cloud
10 2 (1.09%) Tandem
11 1 (0.54%) Embedded Systems
11 1 (0.54%) Linux Command Line
11 1 (0.54%) Replication
11 1 (0.54%) Social Network
11 1 (0.54%) Virtual Team
Operating Systems
1 71 (38.59%) Linux
2 22 (11.96%) Unix
3 1 (0.54%) Red Hat Enterprise Linux
Processes & Methodologies
1 96 (52.17%) Agile Software Development
2 80 (43.48%) Data Modelling
3 75 (40.76%) Data Engineering
4 66 (35.87%) Business Intelligence
5 60 (32.61%) Data Science
6 46 (25.00%) Analytics
7 44 (23.91%) ETL
8 39 (21.20%) Software Engineering
9 37 (20.11%) Machine Learning
10 23 (12.50%) DevOps
10 23 (12.50%) Test Automation
11 22 (11.96%) Data Management
11 22 (11.96%) Technical Leadership
12 21 (11.41%) Data Integration
12 21 (11.41%) MapReduce
12 21 (11.41%) Microservices
13 20 (10.87%) Data Development
14 19 (10.33%) Data Ingestion
14 19 (10.33%) Metadata Management
15 17 (9.24%) Data Analysis
Programming Languages
1 114 (61.96%) Python
2 90 (48.91%) SQL
3 74 (40.22%) Scala
4 66 (35.87%) Java
5 35 (19.02%) Shell Script
6 17 (9.24%) R
6 17 (9.24%) Ruby
7 15 (8.15%) Bash Shell
7 15 (8.15%) C
7 15 (8.15%) PL/SQL
8 9 (4.89%) JavaScript
9 4 (2.17%) Groovy
9 4 (2.17%) Java 8
10 2 (1.09%) Haskell
10 2 (1.09%) Perl
10 2 (1.09%) PHP
10 2 (1.09%) T-SQL
11 1 (0.54%) Go
11 1 (0.54%) HiveQL
11 1 (0.54%) Pig Latin
Qualifications
1 18 (9.78%) Degree
2 8 (4.35%) AWS Certification
3 2 (1.09%) Computer Science Degree
3 2 (1.09%) MCSA
3 2 (1.09%) Microsoft Certification
4 1 (0.54%) SC Cleared
4 1 (0.54%) Security Cleared
Quality Assurance & Compliance
1 31 (16.85%) Data Quality
2 26 (14.13%) QA
3 12 (6.52%) GDPR
4 11 (5.98%) ISO/IEC 27001
System Software
1 18 (9.78%) Docker
2 16 (8.70%) Apache Flume
3 7 (3.80%) Apache ZooKeeper
4 1 (0.54%) Alluxio
4 1 (0.54%) Ceph
4 1 (0.54%) etcd
4 1 (0.54%) Gluster
Systems Management
1 52 (28.26%) Hadoop YARN
2 34 (18.48%) Oozie
3 13 (7.07%) Kubernetes
4 12 (6.52%) Ambari
4 12 (6.52%) Single Sign-On
4 12 (6.52%) Terraform
5 6 (3.26%) Ansible
6 5 (2.72%) Puppet
7 2 (1.09%) Mesos
8 1 (0.54%) Grafana
8 1 (0.54%) Graphite
8 1 (0.54%) Istio
8 1 (0.54%) Knative
8 1 (0.54%) Nagios
8 1 (0.54%) Opscode Chef
8 1 (0.54%) Opsview
8 1 (0.54%) Prometheus
Vendors
1 37 (20.11%) Cloudera
2 19 (10.33%) Google
2 19 (10.33%) Oracle
3 17 (9.24%) Databricks
4 16 (8.70%) Hortonworks
4 16 (8.70%) Teradata
5 9 (4.89%) Informatica
6 6 (3.26%) Tableau
7 4 (2.17%) MapR
7 4 (2.17%) Microsoft
7 4 (2.17%) SolarWinds
8 3 (1.63%) Adobe
9 2 (1.09%) Blue Prism
9 2 (1.09%) Talend
9 2 (1.09%) UiPath
10 1 (0.54%) Alteryx
10 1 (0.54%) Confluent
10 1 (0.54%) IBM
10 1 (0.54%) Salesforce Marketing Cloud
10 1 (0.54%) Salesforce.com