Period
to 15 June 2019

The following table provides summary statistics for permanent job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited HDFS over the 6 months to 15 June 2019 with a comparison to the same period in the previous 2 years.

Hadoop Distributed File System (HDFS)
UK
6 months to
15 Jun 2019
Same period 2018 Same period 2017
Rank 860 756 954
Rank change year-on-year -104 +198 -25
Permanent jobs citing HDFS 237 465 247
As % of all permanent IT jobs advertised in the UK 0.16% 0.27% 0.14%
As % of the System Software category 0.99% 1.90% 1.15%
Number of salaries quoted 189 323 196
UK median annual salary £65,000 £67,500 £71,250
Median salary % change year-on-year -3.70% -5.26% +1.79%
10th Percentile £46,250 £37,500 £48,750
90th Percentile £92,500 £96,250 £115,000
UK excluding London median annual salary £52,500 £60,000 £55,000
% change year-on-year -12.50% +9.09% -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for system software skills.

All System Software Skills
UK
Permanent vacancies with a requirement for system software skills 23,990 24,481 21,415
As % of all permanent IT jobs advertised in the UK 16.70% 14.27% 12.48%
Number of salaries quoted 19,632 19,851 18,018
UK median annual salary £50,000 £47,500 £42,500
Median salary % change year-on-year +5.26% +11.76% +2.41%
10th Percentile £25,750 £25,000 £23,500
90th Percentile £85,000 £80,000 £75,000
UK excluding London median annual salary £42,500 £41,500 £37,500
% change year-on-year +2.41% +10.67% -

HDFS
Job Vacancy Trend

Job postings citing HDFS as a percentage of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing HDFS.

Salary trend for HDFS in the UK

HDFS
Salary Histogram

The salary distribution of IT jobs citing HDFS over the 6 months to 15 June 2019.

Salary histogram for HDFS in the UK

HDFS
Top 14 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing HDFS within the UK over the 6 months to 15 June 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -115 228 £65,000 -7.14% 5
London -3 150 £67,500 -10.00% 3
UK excluding London -69 84 £52,500 -12.50% 2
North of England +24 29 £49,337 +12.77%
Yorkshire +14 20 £49,337 +3.87%
East of England +49 18 £70,000 -3.45% 1
South West +27 18 £50,000 -14.89%
South East -52 8 £50,000 -16.67%
North East +24 6 £49,337 -26.91%
Scotland -2 6 £67,500 +35.00%
Midlands +34 5 £77,500 +29.17% 1
East Midlands - 4 £77,500 - 1
North West +17 3 £70,000 +75.00%
West Midlands +25 1 - -

For the 6 months to 15 June 2019, IT jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for HDFS.

1 202 (85.23%) Hadoop
2 190 (80.17%) Big Data
3 182 (76.79%) Apache Hive
4 164 (69.20%) Apache Spark
5 137 (57.81%) Python
6 133 (56.12%) Agile Software Development
7 131 (55.27%) HBase
8 122 (51.48%) Java
9 108 (45.57%) SQL
10 103 (43.46%) Analytics
11 102 (43.04%) ETL
12 101 (42.62%) Scala
13 94 (39.66%) Kafka
14 79 (33.33%) NoSQL
15 74 (31.22%) Data Science
16 73 (30.80%) Sqoop
16 73 (30.80%) Impala
17 72 (30.38%) Linux
18 69 (29.11%) Hadoop YARN
19 66 (27.85%) Oracle
19 66 (27.85%) Amazon AWS
20 61 (25.74%) Apache Flume
21 60 (25.32%) Data Warehouse
22 59 (24.89%) Machine Learning
22 59 (24.89%) Data Engineering
23 57 (24.05%) Data Integration
24 53 (22.36%) Cloudera
25 52 (21.94%) MongoDB
25 52 (21.94%) Unix
26 51 (21.52%) Apache Cassandra

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 164 (69.20%) Apache Spark
2 34 (14.35%) Apache Pig
3 30 (12.66%) Apache
4 27 (11.39%) Elasticsearch
5 25 (10.55%) Apache Solr
6 9 (3.80%) SAS
7 7 (2.95%) WildFly
8 6 (2.53%) Apache Airflow
8 6 (2.53%) JBoss
8 6 (2.53%) WebSphere
9 5 (2.11%) Apache Geronimo
9 5 (2.11%) Confluence
9 5 (2.11%) GlassFish
9 5 (2.11%) Tomcat
10 4 (1.69%) Cloud Foundry
11 1 (0.42%) Jupyter
11 1 (0.42%) OpenStack
11 1 (0.42%) WebLogic
Applications
1 5 (2.11%) Weka
Business Applications
1 3 (1.27%) Dynamics CRM
2 1 (0.42%) Adobe Campaign
Cloud Services
1 66 (27.85%) Amazon AWS
2 45 (18.99%) Microsoft Azure
3 31 (13.08%) GitHub
4 18 (7.59%) Amazon EMR
4 18 (7.59%) Amazon S3
4 18 (7.59%) Google Cloud Platform
5 13 (5.49%) Cloud Computing
6 8 (3.38%) Serverless
7 7 (2.95%) Amazon Kinesis
8 5 (2.11%) Amazon EC2
8 5 (2.11%) Azure Data Factory
9 4 (1.69%) AWS Lambda
9 4 (1.69%) Power Platform
10 3 (1.27%) IaaS
10 3 (1.27%) PaaS
10 3 (1.27%) Virtual Private Cloud
11 2 (0.84%) IBM Cloud
11 2 (0.84%) SaaS
12 1 (0.42%) Data Lake Analytics
12 1 (0.42%) Google Analytics
Communications & Networking
1 5 (2.11%) Kerberos
2 4 (1.69%) Internet
3 1 (0.42%) Firewall
3 1 (0.42%) HTTP
3 1 (0.42%) HTTPS
3 1 (0.42%) SSL
3 1 (0.42%) VPN
Database & Business Intelligence
1 202 (85.23%) Hadoop
2 190 (80.17%) Big Data
3 182 (76.79%) Apache Hive
4 131 (55.27%) HBase
5 79 (33.33%) NoSQL
6 73 (30.80%) Impala
6 73 (30.80%) Sqoop
7 60 (25.32%) Data Warehouse
8 52 (21.94%) MongoDB
9 51 (21.52%) Apache Cassandra
10 37 (15.61%) SQL Server
11 25 (10.55%) Data Lake
12 24 (10.13%) Relational Database
12 24 (10.13%) Spark SQL
13 22 (9.28%) PostgreSQL
14 19 (8.02%) Redis
15 17 (7.17%) MySQL
16 16 (6.75%) RDBMS
17 15 (6.33%) Neo4j
17 15 (6.33%) Oracle Database
Development Applications
1 29 (12.24%) Git (software)
2 17 (7.17%) IntelliJ
3 16 (6.75%) Jenkins
4 11 (4.64%) Eclipse
4 11 (4.64%) NetBeans
5 10 (4.22%) JIRA
6 9 (3.80%) Maven
7 6 (2.53%) Gradle
8 5 (2.11%) Ant
9 4 (1.69%) JUnit
10 3 (1.27%) Sonatype Nexus
10 3 (1.27%) Visual Studio Team System
11 2 (0.84%) GitLab
12 1 (0.42%) Artifactory
12 1 (0.42%) Hudson
12 1 (0.42%) Oracle APEX
12 1 (0.42%) Selenium
12 1 (0.42%) Travis CI
General
1 34 (14.35%) Finance
2 16 (6.75%) Retail
3 7 (2.95%) Banking
3 7 (2.95%) Telecoms
4 6 (2.53%) Marketing
5 5 (2.11%) Legal
6 2 (0.84%) Front Office
6 2 (0.84%) Investment Banking
6 2 (0.84%) Manufacturing
7 1 (0.42%) Retail Banking
Job Titles
1 112 (47.26%) Data Engineer
2 53 (22.36%) Big Data Engineer
3 52 (21.94%) Developer
4 40 (16.88%) Senior Data Engineer
5 32 (13.50%) Java Developer
6 24 (10.13%) Architect
7 20 (8.44%) Python Engineer
8 18 (7.59%) Data Architect
9 17 (7.17%) Hadoop Engineer
10 16 (6.75%) Senior Developer
11 15 (6.33%) Senior Java Developer
12 14 (5.91%) Big Data Developer
13 12 (5.06%) Data Scientist
13 12 (5.06%) Data Warehouse Developer
13 12 (5.06%) Oracle Data Warehouse Developer
13 12 (5.06%) Oracle Developer
13 12 (5.06%) Senior Data Scientist
14 11 (4.64%) Big Data Architect
15 9 (3.80%) QA Engineer
16 8 (3.38%) Lead Java Developer
Libraries, Frameworks & Software Standards
1 94 (39.66%) Kafka
2 27 (11.39%) Spark Streaming
3 25 (10.55%) Spring
3 25 (10.55%) Web Services
4 19 (8.02%) RabbitMQ
5 17 (7.17%) Hibernate
5 17 (7.17%) Spring Boot
6 16 (6.75%) JPA
6 16 (6.75%) SOAP
7 13 (5.49%) D3.js
8 12 (5.06%) ActiveMQ
8 12 (5.06%) ggplot2
8 12 (5.06%) REST
9 11 (4.64%) Node.js
9 11 (4.64%) Spring Cloud
10 9 (3.80%) Apache NiFi
11 8 (3.38%) HTML
11 8 (3.38%) JSP
12 7 (2.95%) LAMP
13 6 (2.53%) J2EE
Miscellaneous
1 22 (9.28%) Data Structures
2 18 (7.59%) Analytical Skills
3 16 (6.75%) Algorithms
4 9 (3.80%) Greenfield Project
5 8 (3.38%) Public Cloud
6 6 (2.53%) Virtual Team
7 5 (2.11%) Data Centre
7 5 (2.11%) User Experience
8 4 (1.69%) Enterprise Software
8 4 (1.69%) Hybrid Cloud
9 3 (1.27%) Replication
9 3 (1.27%) Self-Motivation
10 2 (0.84%) FMCG
10 2 (0.84%) Management Information System
11 1 (0.42%) Cloud Native
11 1 (0.42%) Data Protection Act
11 1 (0.42%) Fixed Income
11 1 (0.42%) Linux Command Line
11 1 (0.42%) Market Risk
Operating Systems
1 72 (30.38%) Linux
2 52 (21.94%) Unix
3 11 (4.64%) Windows
4 6 (2.53%) Ubuntu
5 2 (0.84%) Windows Server
6 1 (0.42%) VMS
Processes & Methodologies
1 133 (56.12%) Agile Software Development
2 103 (43.46%) Analytics
3 102 (43.04%) ETL
4 74 (31.22%) Data Science
5 59 (24.89%) Data Engineering
5 59 (24.89%) Machine Learning
6 57 (24.05%) Data Integration
7 49 (20.68%) MapReduce
8 46 (19.41%) Data Analysis
9 43 (18.14%) Data Modelling
9 43 (18.14%) Mentoring
10 40 (16.88%) Problem-Solving
11 39 (16.46%) DevOps
12 31 (13.08%) Continuous Integration
12 31 (13.08%) Use Case
13 28 (11.81%) Coaching
14 25 (10.55%) Data Reconciliation
14 25 (10.55%) Data Transformation
14 25 (10.55%) Performance Tuning
15 24 (10.13%) Data Ingestion
Programming Languages
1 137 (57.81%) Python
2 122 (51.48%) Java
3 108 (45.57%) SQL
4 101 (42.62%) Scala
5 27 (11.39%) JavaScript
6 24 (10.13%) R
7 22 (9.28%) PL/SQL
8 16 (6.75%) T-SQL
9 14 (5.91%) Shell Script
10 8 (3.38%) Java 8
10 8 (3.38%) Ruby
11 6 (2.53%) Groovy
12 5 (2.11%) Bash Shell
13 4 (1.69%) C#
14 3 (1.27%) C
15 2 (0.84%) PowerShell
16 1 (0.42%) HiveQL
16 1 (0.42%) MATLAB
16 1 (0.42%) Perl
Qualifications
1 23 (9.70%) Degree
2 12 (5.06%) Computer Science Degree
3 6 (2.53%) Security Cleared
4 4 (1.69%) AWS Certification
5 3 (1.27%) SC Cleared
6 2 (0.84%) DV Cleared
7 1 (0.42%) TOGAF Certification
Quality Assurance & Compliance
1 22 (9.28%) Data Quality
2 21 (8.86%) QA
3 13 (5.49%) GDPR
4 5 (2.11%) ISO/IEC 27001
5 4 (1.69%) PCI DSS
System Software
1 61 (25.74%) Apache Flume
2 30 (12.66%) Docker
3 15 (6.33%) Apache ZooKeeper
4 2 (0.84%) Virtual Machines
4 2 (0.84%) VMware Infrastructure
4 2 (0.84%) vSphere
5 1 (0.42%) Cronacle
Systems Management
1 69 (29.11%) Hadoop YARN
2 51 (21.52%) Oozie
3 20 (8.44%) Kubernetes
4 7 (2.95%) Ansible
5 6 (2.53%) Terraform
6 4 (1.69%) Mesos
6 4 (1.69%) Puppet
7 3 (1.27%) Kibana
8 2 (0.84%) Check_MK
8 2 (0.84%) Nagios
8 2 (0.84%) Rsyslog
9 1 (0.42%) Ambari
9 1 (0.42%) Control-M
9 1 (0.42%) Single Sign-On
Vendors
1 66 (27.85%) Oracle
2 53 (22.36%) Cloudera
3 43 (18.14%) Hortonworks
4 17 (7.17%) Informatica
5 15 (6.33%) Talend
6 12 (5.06%) Google
6 12 (5.06%) Microsoft
6 12 (5.06%) Teradata
7 10 (4.22%) Tableau
8 9 (3.80%) MapR
9 7 (2.95%) Databricks
10 5 (2.11%) Alteryx
11 4 (1.69%) Ab Initio
11 4 (1.69%) MicroStrategy
11 4 (1.69%) Salesforce.com
12 3 (1.27%) Cognos
12 3 (1.27%) IBM
12 3 (1.27%) Jaspersoft
12 3 (1.27%) Syncsort
12 3 (1.27%) TIBCO