Period
to 22 August 2017

The following table provides summary statistics for permanent job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited HDFS over the 6 months to 22 August 2017 with a comparison to the same period in the previous 2 years.

Hadoop Distributed File System (HDFS)
UK
6 months to
22 Aug 2017
Same period 2016 Same period 2015
Rank 947 927 1128
Rank change year-on-year -20 +201 +86
Permanent jobs citing HDFS 240 319 186
As % of all permanent IT jobs advertised in the UK 0.14% 0.16% 0.085%
As % of the System Software category 1.07% 1.30% 0.73%
Number of salaries quoted 182 255 120
UK median annual salary £70,000 £77,500 £70,000
Median salary % change year-on-year -9.68% +10.71% +6.06%
10th Percentile £45,000 £45,000 £45,000
90th Percentile £102,500 £116,500 £102,500
UK excluding London median annual salary £55,000 £52,500 £50,000
% change year-on-year +4.76% +5.00% +17.65%

HDFS is in the system software skills category. The next table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for system software skills.

System Software Skills
UK
Permanent job vacancies with a requirement for system software skills 22448 24610 25436
As % of all permanent IT job vacancies advertised in the UK 12.75% 12.49% 11.66%
Number of salaries quoted 18896 20860 21057
UK median annual salary £45,000 £42,500 £40,000
Median salary % change year-on-year +5.88% +6.25% -
10th Percentile £24,000 £23,000 £23,000
90th Percentile £75,000 £71,250 £65,350
UK excluding London median annual salary £38,000 £37,500 £37,500
% change year-on-year +1.33% - +7.14%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a percentage of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing HDFS.

Salary trend for HDFS in the UK

HDFS
Salary Histogram

The salary distribution of IT jobs citing HDFS over the 6 months to 22 August 2017.

Salary histogram for HDFS in the UK

HDFS
Top 13 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing HDFS within the UK over the 6 months to 22 August 2017. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -10 229 £70,000 -6.67% 18
London -16 165 £70,000 -17.65% 16
UK excluding London +25 74 £55,000 +4.76% 3
South East +48 25 £55,000 +6.80%
North of England -8 23 £70,000 +47.37%
North West -11 20 £72,500 +102.80%
East of England - 11 £55,000 - 1
Scotland +17 10 £65,750 -
South West - 4 £55,000 -
North East - 2 £50,000 -
Midlands +12 1 £50,000 -9.09% 1
Yorkshire +10 1 - -
East Midlands - 1 £50,000 -

For the 6 months to 22 August 2017, IT jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all job ads with a requirement for HDFS.

1 206 (85.83%) Hadoop
2 193 (80.42%) Big Data
3 170 (70.83%) Apache Spark
4 158 (65.83%) Apache Hive
5 147 (61.25%) Java
6 119 (49.58%) HBase
7 114 (47.50%) MapReduce
8 111 (46.25%) Kafka
9 105 (43.75%) Python
10 104 (43.33%) Agile Software Development
11 86 (35.83%) Analytics
12 78 (32.50%) NoSQL
13 74 (30.83%) SQL
14 72 (30.00%) Scala
15 71 (29.58%) Finance
16 70 (29.17%) Apache Pig
17 69 (28.75%) Apache
18 62 (25.83%) Linux
19 59 (24.58%) ETL
20 54 (22.50%) Cloudera
21 48 (20.00%) Oozie
22 44 (18.33%) Apache Cassandra
23 43 (17.92%) Hadoop YARN
24 42 (17.50%) Analytical Skills
25 41 (17.08%) Relational Database
26 40 (16.67%) Hortonworks
27 38 (15.83%) Impala
27 38 (15.83%) SQL Server
27 38 (15.83%) TDD
28 37 (15.42%) Apache Flume

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 170 (70.83%) Apache Spark
2 70 (29.17%) Apache Pig
3 69 (28.75%) Apache
4 24 (10.00%) Apache Solr
5 14 (5.83%) Tomcat
6 10 (4.17%) SAS
7 9 (3.75%) Elasticsearch
8 6 (2.50%) Confluence
9 4 (1.67%) Cloud Foundry
9 4 (1.67%) GemFire
10 3 (1.25%) JBoss
10 3 (1.25%) OpenStack
11 2 (0.83%) Apache Geronimo
11 2 (0.83%) GlassFish
11 2 (0.83%) Mule
11 2 (0.83%) WebSphere
12 1 (0.42%) Oracle Coherence
Applications
1 2 (0.83%) Microsoft Excel
1 2 (0.83%) Weka
Cloud Services
1 36 (15.00%) Amazon AWS
2 16 (6.67%) Cloud Computing
3 12 (5.00%) Microsoft Azure
4 11 (4.58%) GitHub
4 11 (4.58%) Google Cloud Platform
5 9 (3.75%) PaaS
6 7 (2.92%) IaaS
7 4 (1.67%) Amazon S3
7 4 (1.67%) SaaS
8 3 (1.25%) Loggly
8 3 (1.25%) Sumo Logic
9 2 (0.83%) AWS Lambda
9 2 (0.83%) Cortana Intelligence Suite
10 1 (0.42%) Amazon EMR
10 1 (0.42%) Amazon Kinesis
10 1 (0.42%) DigitalOcean
Communications & Networking
1 14 (5.83%) Kerberos
2 13 (5.42%) Internet
3 6 (2.50%) LDAP
4 4 (1.67%) HTTP
5 3 (1.25%) DNS
6 2 (0.83%) Broadband
7 1 (0.42%) Network Security
7 1 (0.42%) SSL
Database & Business Intelligence
1 206 (85.83%) Hadoop
2 193 (80.42%) Big Data
3 158 (65.83%) Apache Hive
4 119 (49.58%) HBase
5 78 (32.50%) NoSQL
6 44 (18.33%) Apache Cassandra
7 41 (17.08%) Relational Database
8 38 (15.83%) Impala
8 38 (15.83%) SQL Server
9 34 (14.17%) Sqoop
10 31 (12.92%) MongoDB
11 23 (9.58%) MySQL
12 22 (9.17%) Data Warehouse
13 19 (7.92%) SQL Server Integration Services
14 17 (7.08%) PostgreSQL
15 12 (5.00%) RDBMS
16 11 (4.58%) Parquet
17 9 (3.75%) Amazon Redshift
17 9 (3.75%) Data Mining
18 8 (3.33%) Redis
Development Applications
1 27 (11.25%) Git (software)
2 26 (10.83%) Jenkins
3 22 (9.17%) Subversion
4 17 (7.08%) RStudio
4 17 (7.08%) SQL Server Management Studio (SSMS)
5 14 (5.83%) Eclipse
5 14 (5.83%) Gradle
6 13 (5.42%) Maven
7 12 (5.00%) JIRA
8 8 (3.33%) Talend Open Studio
9 5 (2.08%) Cucumber
10 4 (1.67%) JUnit
10 4 (1.67%) Vagrant
11 3 (1.25%) IntelliJ
12 2 (0.83%) Ant
12 2 (0.83%) Atlassian Bamboo
12 2 (0.83%) Grunt
12 2 (0.83%) NetBeans
13 1 (0.42%) Bitbucket
13 1 (0.42%) NeoLoad
General
1 71 (29.58%) Finance
2 23 (9.58%) Retail
3 13 (5.42%) Banking
4 8 (3.33%) Front Office
4 8 (3.33%) Marketing
4 8 (3.33%) Telecoms
5 7 (2.92%) Legal
6 6 (2.50%) Investment Banking
7 5 (2.08%) Games
8 3 (1.25%) Electronics
8 3 (1.25%) Manufacturing
9 1 (0.42%) Financial Institution
9 1 (0.42%) Retail Banking
Job Titles
1 64 (26.67%) Developer
2 52 (21.67%) Data Engineer
3 46 (19.17%) Hadoop Engineer
4 37 (15.42%) Big Data Engineer
5 32 (13.33%) Architect
6 28 (11.67%) Data Architect
7 26 (10.83%) Java Developer
8 21 (8.75%) Big Data Architect
9 20 (8.33%) Data Manager
10 18 (7.50%) Big Data Developer
11 12 (5.00%) Hadoop Developer
11 12 (5.00%) Senior Developer
12 11 (4.58%) Data Scientist
12 11 (4.58%) Software Engineer
13 10 (4.17%) Scala Developer
14 8 (3.33%) Administrator
14 8 (3.33%) Big Data Software Engineer
14 8 (3.33%) E-Commerce Director
14 8 (3.33%) Hadoop Administrator
14 8 (3.33%) Hadoop Architect
Libraries, Frameworks & Software Standards
1 111 (46.25%) Kafka
2 30 (12.50%) Spring
3 25 (10.42%) JSON
4 24 (10.00%) REST
5 16 (6.67%) Apache Avro
6 13 (5.42%) HTML
6 13 (5.42%) Node.js
7 12 (5.00%) Akka
7 12 (5.00%) RabbitMQ
7 12 (5.00%) Spring Boot
8 9 (3.75%) CSS
8 9 (3.75%) Middleware
8 9 (3.75%) RESTful
8 9 (3.75%) Spark Streaming
8 9 (3.75%) Spring MVC
9 8 (3.33%) AngularJS
9 8 (3.33%) Apache Flex
9 8 (3.33%) MLlib
9 8 (3.33%) Web Services
10 7 (2.92%) Natural Language Processing
Miscellaneous
1 42 (17.50%) Analytical Skills
2 25 (10.42%) Computer Science
3 20 (8.33%) Algorithms
4 18 (7.50%) Greenfield Project
5 17 (7.08%) Virtual Team
6 15 (6.25%) Data Structures
7 14 (5.83%) Linux Command Line
8 8 (3.33%) Foreign Exchange (FX)
9 7 (2.92%) Replication
9 7 (2.92%) Social Network
10 5 (2.08%) Derivative
10 5 (2.08%) Fixed Income
11 3 (1.25%) Clustering
11 3 (1.25%) Fintech
11 3 (1.25%) Management Information System
12 2 (0.83%) Data Centre
12 2 (0.83%) FMCG
12 2 (0.83%) Grid Computing
12 2 (0.83%) Market Risk
13 1 (0.42%) Credit Risk
Operating Systems
1 62 (25.83%) Linux
2 32 (13.33%) Unix
3 8 (3.33%) Windows
4 6 (2.50%) Red Hat Enterprise Linux
5 3 (1.25%) CoreOS
6 1 (0.42%) Mesosphere
6 1 (0.42%) Ubuntu
Processes & Methodologies
1 114 (47.50%) MapReduce
2 104 (43.33%) Agile Software Development
3 86 (35.83%) Analytics
4 59 (24.58%) ETL
5 38 (15.83%) TDD
6 33 (13.75%) Business Intelligence
7 30 (12.50%) Software Engineering
8 29 (12.08%) Data Management
9 27 (11.25%) Machine Learning
10 25 (10.42%) Scrum
11 24 (10.00%) E-Commerce
11 24 (10.00%) Use Case
12 23 (9.58%) OO
13 22 (9.17%) BDD
13 22 (9.17%) Business Analytics
14 21 (8.75%) Continuous Integration
15 20 (8.33%) Technical Architecture
16 19 (7.92%) Information Management
16 19 (7.92%) Requirements Analysis
16 19 (7.92%) Visualisation
Programming Languages
1 147 (61.25%) Java
2 105 (43.75%) Python
3 74 (30.83%) SQL
4 72 (30.00%) Scala
5 29 (12.08%) R
6 25 (10.42%) JavaScript
7 22 (9.17%) T-SQL
8 16 (6.67%) Shell Script
9 15 (6.25%) C++
10 10 (4.17%) C
10 10 (4.17%) PHP
11 8 (3.33%) Perl
12 6 (2.50%) C#
13 5 (2.08%) Ruby
14 4 (1.67%) Bash Shell
14 4 (1.67%) Go
15 3 (1.25%) Clojure
15 3 (1.25%) PHP 5
15 3 (1.25%) PHP 7
16 2 (0.83%) Haskell
Qualifications
1 30 (12.50%) Degree
2 15 (6.25%) Computer Science Degree
3 7 (2.92%) PhD
4 3 (1.25%) Security Cleared
5 2 (0.83%) DV Cleared
5 2 (0.83%) Master's Degree
5 2 (0.83%) Numerate Degree
6 1 (0.42%) SC Cleared
Quality Assurance & Compliance
1 9 (3.75%) QA
2 7 (2.92%) GCP
3 4 (1.67%) Data Quality
4 2 (0.83%) SLA
System Software
1 37 (15.42%) Apache Flume
2 21 (8.75%) Apache ZooKeeper
3 14 (5.83%) Docker
4 4 (1.67%) VMware Infrastructure
5 3 (1.25%) VirtualBox
6 2 (0.83%) Cronacle
6 2 (0.83%) Oracle RAC
7 1 (0.42%) KVM
7 1 (0.42%) vCloud
7 1 (0.42%) Virtual Machines
7 1 (0.42%) Xen
Systems Management
1 48 (20.00%) Oozie
2 43 (17.92%) Hadoop YARN
3 15 (6.25%) Ansible
4 12 (5.00%) Puppet
5 11 (4.58%) Ambari
5 11 (4.58%) Opscode Chef
6 9 (3.75%) Kubernetes
6 9 (3.75%) Terraform
7 5 (2.08%) Graphite
8 4 (1.67%) Mesos
8 4 (1.67%) Salt
9 3 (1.25%) Nagios
9 3 (1.25%) ZABBIX
10 2 (0.83%) Control-M
11 1 (0.42%) Consul
11 1 (0.42%) Docker Swarm
11 1 (0.42%) Packer
11 1 (0.42%) Rancher
11 1 (0.42%) yum
Vendors
1 54 (22.50%) Cloudera
2 40 (16.67%) Hortonworks
3 28 (11.67%) MapR
4 20 (8.33%) Oracle
5 15 (6.25%) Google
6 8 (3.33%) Tableau
6 8 (3.33%) Talend
6 8 (3.33%) Teradata
7 6 (2.50%) IBM
8 5 (2.08%) Greenplum
8 5 (2.08%) SAP
9 4 (1.67%) Red Hat
9 4 (1.67%) Vertica
9 4 (1.67%) VMware
10 3 (1.25%) Microsoft
10 3 (1.25%) MuleSoft
10 3 (1.25%) New Relic
10 3 (1.25%) Rackspace
10 3 (1.25%) SaltStack
10 3 (1.25%) Splunk