Period
to 22 October 2017

The following table provides summary statistics for permanent job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited HDFS over the 6 months to 22 October 2017 with a comparison to the same period in the previous 2 years.

Hadoop Distributed File System (HDFS)
UK
6 months to
22 Oct 2017
Same period 2016 Same period 2015
Rank 950 965 1105
Rank change year-on-year +15 +140 +37
Permanent jobs citing HDFS 235 282 198
As % of all permanent IT jobs advertised in the UK 0.13% 0.15% 0.091%
As % of the System Software category 1.03% 1.20% 0.79%
Number of salaries quoted 189 210 142
UK median annual salary £70,000 £80,000 £70,000
Median salary % change year-on-year -12.50% +14.29% +6.06%
10th Percentile £44,750 £47,500 £46,375
90th Percentile £108,000 £105,500 £102,375
UK excluding London median annual salary £57,500 £55,000 £60,000
% change year-on-year +4.55% -8.33% -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for system software skills.

All System Software Skills
UK
Permanent vacancies with a requirement for system software skills 22776 23581 24937
As % of all permanent IT jobs advertised in the UK 13.07% 12.43% 11.41%
Number of salaries quoted 19173 19842 20791
UK median annual salary £45,000 £42,500 £40,000
Median salary % change year-on-year +5.88% +6.25% -
10th Percentile £23,750 £23,500 £22,696
90th Percentile £75,000 £71,250 £67,500
UK excluding London median annual salary £39,000 £37,500 £37,500
% change year-on-year +4.00% - +7.14%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a percentage of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing HDFS.

Salary trend for HDFS in the UK

HDFS
Salary Histogram

The salary distribution of IT jobs citing HDFS over the 6 months to 22 October 2017.

Salary histogram for HDFS in the UK

HDFS
Top 13 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing HDFS within the UK over the 6 months to 22 October 2017. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +18 233 £70,000 -12.50% 20
London -7 167 £70,000 -17.65% 18
UK excluding London +30 68 £57,500 +4.55% 2
South East +46 32 £57,500 +9.00% 2
North of England +18 13 £70,000 +27.27%
North West +4 12 £70,000 +89.19%
East of England - 12 £105,000 -
South West - 5 £50,000 -
Midlands +20 4 £46,125 -16.14%
West Midlands -4 3 £42,250 -23.18%
Scotland -18 2 £57,500 -
Yorkshire +10 1 - -
East Midlands - 1 £50,000 -

For the 6 months to 22 October 2017, IT jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all job ads with a requirement for HDFS.

1 196 (83.40%) Big Data
2 193 (82.13%) Hadoop
3 177 (75.32%) Apache Spark
4 159 (67.66%) Java
5 142 (60.43%) Apache Hive
6 113 (48.09%) HBase
7 108 (45.96%) MapReduce
8 101 (42.98%) Agile Software Development
9 97 (41.28%) Kafka
10 94 (40.00%) Python
11 89 (37.87%) Scala
12 88 (37.45%) Analytics
13 77 (32.77%) SQL
14 74 (31.49%) Finance
15 69 (29.36%) NoSQL
16 58 (24.68%) Apache
17 57 (24.26%) ETL
18 55 (23.40%) Apache Pig
19 54 (22.98%) Linux
20 50 (21.28%) Cloudera
21 48 (20.43%) Analytical Skills
21 48 (20.43%) Amazon AWS
22 44 (18.72%) Degree
22 44 (18.72%) JavaScript
22 44 (18.72%) Apache Flume
23 42 (17.87%) Business Intelligence
23 42 (17.87%) TDD
24 41 (17.45%) SQL Server
25 40 (17.02%) Continuous Integration
25 40 (17.02%) Relational Database

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 177 (75.32%) Apache Spark
2 58 (24.68%) Apache
3 55 (23.40%) Apache Pig
4 30 (12.77%) Apache Solr
5 13 (5.53%) Elasticsearch
6 9 (3.83%) Tomcat
7 6 (2.55%) Confluence
7 6 (2.55%) SAS
8 3 (1.28%) JBoss
9 2 (0.85%) OpenStack
10 1 (0.43%) Oracle Coherence
10 1 (0.43%) Oracle SOA Suite
10 1 (0.43%) WebLogic
10 1 (0.43%) WebSphere
Applications
1 2 (0.85%) Microsoft Excel
Cloud Services
1 48 (20.43%) Amazon AWS
2 15 (6.38%) Cloud Computing
3 12 (5.11%) Microsoft Azure
4 9 (3.83%) Google Cloud Platform
5 6 (2.55%) Amazon EMR
6 5 (2.13%) IaaS
7 4 (1.70%) Amazon S3
8 3 (1.28%) PaaS
8 3 (1.28%) SaaS
9 2 (0.85%) AWS Lambda
9 2 (0.85%) Cortana Intelligence Suite
9 2 (0.85%) Loggly
9 2 (0.85%) Serverless
9 2 (0.85%) Sumo Logic
10 1 (0.43%) Amazon Kinesis
Communications & Networking
1 16 (6.81%) Internet
2 10 (4.26%) Kerberos
3 5 (2.13%) HTTP
4 4 (1.70%) DNS
5 3 (1.28%) LDAP
6 2 (0.85%) Broadband
6 2 (0.85%) TCP/IP
7 1 (0.43%) Network Security
7 1 (0.43%) SSL
Database & Business Intelligence
1 196 (83.40%) Big Data
2 193 (82.13%) Hadoop
3 142 (60.43%) Apache Hive
4 113 (48.09%) HBase
5 69 (29.36%) NoSQL
6 41 (17.45%) SQL Server
7 40 (17.02%) Relational Database
8 36 (15.32%) Apache Cassandra
9 33 (14.04%) Impala
10 32 (13.62%) MongoDB
11 31 (13.19%) Sqoop
12 24 (10.21%) SQL Server Integration Services
13 19 (8.09%) Data Warehouse
14 18 (7.66%) MySQL
15 17 (7.23%) Redis
16 14 (5.96%) PostgreSQL
17 12 (5.11%) RDBMS
18 10 (4.26%) SQL Server Analysis Services
19 9 (3.83%) Data Mining
19 9 (3.83%) Spark SQL
Development Applications
1 32 (13.62%) Git (software)
2 24 (10.21%) Jenkins
3 19 (8.09%) Maven
3 19 (8.09%) Subversion
4 18 (7.66%) RStudio
4 18 (7.66%) SQL Server Management Studio (SSMS)
5 17 (7.23%) JIRA
6 12 (5.11%) Eclipse
7 10 (4.26%) Grunt
8 8 (3.40%) Gerrit
8 8 (3.40%) Jasmine
8 8 (3.40%) Selenium
8 8 (3.40%) Talend Open Studio
9 6 (2.55%) Bitbucket
9 6 (2.55%) Cucumber
9 6 (2.55%) IntelliJ
10 5 (2.13%) NetBeans
11 4 (1.70%) JIRA Agile
12 3 (1.28%) Gradle
13 2 (0.85%) Vagrant
General
1 74 (31.49%) Finance
2 25 (10.64%) Banking
3 24 (10.21%) Retail
4 9 (3.83%) Front Office
4 9 (3.83%) Telecoms
5 7 (2.98%) Legal
5 7 (2.98%) Marketing
6 6 (2.55%) Investment Banking
7 5 (2.13%) Games
8 3 (1.28%) Electronics
8 3 (1.28%) Manufacturing
9 2 (0.85%) Advertising
9 2 (0.85%) Financial Institution
10 1 (0.43%) iGaming
10 1 (0.43%) Retail Banking
Job Titles
1 85 (36.17%) Developer
2 42 (17.87%) Java Developer
3 39 (16.60%) Data Engineer
4 34 (14.47%) Big Data Engineer
5 31 (13.19%) Big Data Developer
6 29 (12.34%) Hadoop Engineer
7 26 (11.06%) Architect
8 25 (10.64%) Senior Developer
9 23 (9.79%) Data Manager
9 23 (9.79%) Senior Java Developer
10 22 (9.36%) Data Architect
11 18 (7.66%) Big Data Architect
12 15 (6.38%) Scala Developer
13 13 (5.53%) Data Scientist
13 13 (5.53%) Software Engineer
14 10 (4.26%) Hadoop Developer
15 9 (3.83%) Big Data Software Engineer
16 8 (3.40%) Angular Developer
17 7 (2.98%) Java Architect
17 7 (2.98%) Java Scala Developer
Libraries, Frameworks & Software Standards
1 97 (41.28%) Kafka
2 32 (13.62%) Spring
3 31 (13.19%) REST
4 22 (9.36%) Apache Avro
5 21 (8.94%) AngularJS
5 21 (8.94%) Node.js
6 19 (8.09%) HTML
7 18 (7.66%) CSS
7 18 (7.66%) JSON
8 15 (6.38%) Web Services
9 11 (4.68%) RabbitMQ
9 11 (4.68%) Spring Boot
10 10 (4.26%) Akka
10 10 (4.26%) SOAP
10 10 (4.26%) XML
11 9 (3.83%) Apache Flex
11 9 (3.83%) D3.js
12 8 (3.40%) Natural Language Processing
12 8 (3.40%) Spark Streaming
13 7 (2.98%) Spring MVC
Miscellaneous
1 48 (20.43%) Analytical Skills
2 36 (15.32%) Computer Science
3 27 (11.49%) Greenfield Project
4 19 (8.09%) Data Structures
5 18 (7.66%) Algorithms
6 13 (5.53%) Fintech
7 7 (2.98%) Foreign Exchange (FX)
7 7 (2.98%) Social Network
8 4 (1.70%) Derivative
8 4 (1.70%) Fixed Income
8 4 (1.70%) Virtual Team
9 3 (1.28%) Clustering
9 3 (1.28%) Linux Command Line
9 3 (1.28%) Market Risk
9 3 (1.28%) Replication
10 2 (0.85%) Credit Risk
10 2 (0.85%) FMCG
10 2 (0.85%) Online Games
11 1 (0.43%) Blog
11 1 (0.43%) Enterprise Software
Operating Systems
1 54 (22.98%) Linux
2 17 (7.23%) Unix
3 16 (6.81%) Windows
4 2 (0.85%) CoreOS
4 2 (0.85%) Ubuntu
5 1 (0.43%) Mesosphere
Processes & Methodologies
1 108 (45.96%) MapReduce
2 101 (42.98%) Agile Software Development
3 88 (37.45%) Analytics
4 57 (24.26%) ETL
5 42 (17.87%) Business Intelligence
5 42 (17.87%) TDD
6 40 (17.02%) Continuous Integration
7 33 (14.04%) Software Engineering
8 31 (13.19%) Data Management
8 31 (13.19%) OO
9 28 (11.91%) Machine Learning
10 25 (10.64%) Scrum
11 24 (10.21%) Visualisation
12 21 (8.94%) Data Architecture
12 21 (8.94%) Data Modelling
12 21 (8.94%) Microservices
12 21 (8.94%) Use Case
13 20 (8.51%) Data Analytics
14 19 (8.09%) Business Analytics
15 17 (7.23%) Technical Architecture
Programming Languages
1 159 (67.66%) Java
2 94 (40.00%) Python
3 89 (37.87%) Scala
4 77 (32.77%) SQL
5 44 (18.72%) JavaScript
6 38 (16.17%) R
7 23 (9.79%) T-SQL
8 10 (4.26%) C++
9 9 (3.83%) C
10 6 (2.55%) PHP
11 5 (2.13%) C#
11 5 (2.13%) Go
11 5 (2.13%) Shell Script
12 4 (1.70%) Perl
12 4 (1.70%) Ruby
13 3 (1.28%) Bash Shell
13 3 (1.28%) Clojure
14 2 (0.85%) HiveQL
14 2 (0.85%) MATLAB
14 2 (0.85%) PHP 7
Qualifications
1 44 (18.72%) Degree
2 24 (10.21%) Computer Science Degree
3 7 (2.98%) PhD
4 4 (1.70%) Security Cleared
5 3 (1.28%) DV Cleared
6 2 (0.85%) Master's Degree
6 2 (0.85%) Numerate Degree
7 1 (0.43%) CTC Cleared
7 1 (0.43%) SC Cleared
Quality Assurance & Compliance
1 10 (4.26%) Data Quality
2 5 (2.13%) QA
3 2 (0.85%) GCP
4 1 (0.43%) GDPR
4 1 (0.43%) SLA
System Software
1 44 (18.72%) Apache Flume
2 25 (10.64%) Apache ZooKeeper
3 19 (8.09%) Docker
4 2 (0.85%) VirtualBox
4 2 (0.85%) VMware Infrastructure
5 1 (0.43%) Cronacle
5 1 (0.43%) MapR-FS
Systems Management
1 36 (15.32%) Oozie
2 35 (14.89%) Hadoop YARN
3 19 (8.09%) Puppet
4 10 (4.26%) Ansible
5 9 (3.83%) Ambari
5 9 (3.83%) Kubernetes
6 8 (3.40%) Opscode Chef
7 7 (2.98%) Terraform
8 6 (2.55%) Nagios
9 4 (1.70%) Graphite
9 4 (1.70%) Mesos
10 3 (1.28%) Salt
11 2 (0.85%) ZABBIX
12 1 (0.43%) Control-M
12 1 (0.43%) Docker Swarm
12 1 (0.43%) yum
Vendors
1 50 (21.28%) Cloudera
2 35 (14.89%) Hortonworks
3 25 (10.64%) MapR
4 20 (8.51%) Oracle
5 13 (5.53%) Google
6 9 (3.83%) Teradata
7 8 (3.40%) Tableau
7 8 (3.40%) Talend
8 7 (2.98%) Microsoft
9 6 (2.55%) IBM
10 5 (2.13%) Greenplum
10 5 (2.13%) SAP
11 4 (1.70%) Vertica
12 3 (1.28%) Atlassian
13 2 (0.85%) Datameer
13 2 (0.85%) Informatica
13 2 (0.85%) New Relic
13 2 (0.85%) Rackspace
13 2 (0.85%) Splunk
13 2 (0.85%) VMware