Period
to 23 November 2017

The following table provides summary statistics for permanent job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited HDFS over the 6 months to 23 November 2017 with a comparison to the same period in the previous 2 years.

Hadoop Distributed File System (HDFS)
UK
6 months to
23 Nov 2017
Same period 2016 Same period 2015
Rank 943 948 1065
Rank change year-on-year +5 +117 +89
Permanent jobs citing HDFS 246 291 231
As % of all permanent IT jobs advertised in the UK 0.14% 0.15% 0.11%
As % of the System Software category 1.06% 1.24% 0.94%
Number of salaries quoted 198 223 167
UK median annual salary £70,000 £80,000 £70,000
Median salary % change year-on-year -12.50% +14.29% +16.67%
10th Percentile £45,000 £49,750 £45,750
90th Percentile £104,125 £105,000 £100,000
UK excluding London median annual salary £57,500 £59,000 £60,000
% change year-on-year -2.54% -1.67% -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for system software skills.

All System Software Skills
UK
Permanent vacancies with a requirement for system software skills 23308 23451 24638
As % of all permanent IT jobs advertised in the UK 12.94% 12.35% 11.33%
Number of salaries quoted 19491 19678 20563
UK median annual salary £45,000 £42,500 £40,000
Median salary % change year-on-year +5.88% +6.25% +1.53%
10th Percentile £23,750 £23,500 £22,500
90th Percentile £75,000 £72,500 £67,500
UK excluding London median annual salary £40,000 £37,500 £37,500
% change year-on-year +6.67% - +7.14%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a percentage of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing HDFS.

Salary trend for HDFS in the UK

HDFS
Salary Histogram

The salary distribution of IT jobs citing HDFS over the 6 months to 23 November 2017.

Salary histogram for HDFS in the UK

HDFS
Top 12 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing HDFS within the UK over the 6 months to 23 November 2017. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +15 244 £70,000 -12.50% 26
London -12 173 £72,500 -9.38% 16
UK excluding London +26 72 £57,500 -2.54% 12
South East +28 43 £57,500 +4.55% 2
East of England - 10 £105,000 -
North of England +9 7 £45,000 -14.29% 5
North West -12 7 £45,000 +21.62% 2
South West +13 6 £50,000 -33.33% 3
Midlands +13 4 £46,125 -7.75%
West Midlands +3 3 £42,250 -15.50%
Scotland -12 2 £57,500 - 2
East Midlands - 1 £50,000 -

For the 6 months to 23 November 2017, IT jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all job ads with a requirement for HDFS.

1 202 (82.11%) Hadoop
2 195 (79.27%) Big Data
3 185 (75.20%) Apache Spark
4 156 (63.41%) Java
5 129 (52.44%) Apache Hive
6 111 (45.12%) HBase
7 110 (44.72%) Agile Software Development
8 101 (41.06%) MapReduce
9 96 (39.02%) Python
9 96 (39.02%) Kafka
10 89 (36.18%) Analytics
11 82 (33.33%) Scala
12 78 (31.71%) SQL
13 70 (28.46%) Linux
14 69 (28.05%) Apache
15 66 (26.83%) NoSQL
16 63 (25.61%) Finance
17 59 (23.98%) ETL
18 58 (23.58%) Degree
19 54 (21.95%) Amazon AWS
20 53 (21.54%) JavaScript
21 52 (21.14%) SQL Server
22 50 (20.33%) Continuous Integration
22 50 (20.33%) TDD
23 46 (18.70%) Apache Pig
23 46 (18.70%) Apache Flume
24 45 (18.29%) Git (software)
25 43 (17.48%) REST
25 43 (17.48%) Relational Database
26 42 (17.07%) Spring

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 185 (75.20%) Apache Spark
2 69 (28.05%) Apache
3 46 (18.70%) Apache Pig
4 30 (12.20%) Apache Solr
5 27 (10.98%) Elasticsearch
6 8 (3.25%) Confluence
6 8 (3.25%) Tomcat
7 4 (1.63%) Oracle Coherence
8 3 (1.22%) JBoss
8 3 (1.22%) OpenStack
9 2 (0.81%) Cloud Foundry
9 2 (0.81%) GemFire
9 2 (0.81%) SAS
10 1 (0.41%) Apache Airflow
10 1 (0.41%) Oracle SOA Suite
10 1 (0.41%) WebLogic
10 1 (0.41%) WebSphere
Applications
1 2 (0.81%) Microsoft Excel
Business Applications
1 1 (0.41%) Summit
Cloud Services
1 54 (21.95%) Amazon AWS
2 15 (6.10%) Google Cloud Platform
3 14 (5.69%) Cloud Computing
4 13 (5.28%) Microsoft Azure
5 10 (4.07%) Amazon EMR
6 5 (2.03%) Amazon S3
7 4 (1.63%) Azure Data Factory
7 4 (1.63%) GitHub
7 4 (1.63%) IaaS
8 3 (1.22%) Loggly
8 3 (1.22%) PaaS
8 3 (1.22%) Sumo Logic
9 2 (0.81%) AWS Lambda
9 2 (0.81%) Cortana Intelligence Suite
9 2 (0.81%) SaaS
9 2 (0.81%) Serverless
10 1 (0.41%) Amazon EC2
10 1 (0.41%) Amazon Kinesis
Communications & Networking
1 20 (8.13%) Internet
2 16 (6.50%) Kerberos
3 6 (2.44%) HTTP
4 5 (2.03%) DNS
5 4 (1.63%) LDAP
5 4 (1.63%) Network Security
6 3 (1.22%) TCP/IP
7 2 (0.81%) Broadband
8 1 (0.41%) Firewall
8 1 (0.41%) SSL
Database & Business Intelligence
1 202 (82.11%) Hadoop
2 195 (79.27%) Big Data
3 129 (52.44%) Apache Hive
4 111 (45.12%) HBase
5 66 (26.83%) NoSQL
6 52 (21.14%) SQL Server
7 43 (17.48%) Relational Database
8 37 (15.04%) Impala
9 34 (13.82%) Sqoop
10 33 (13.41%) MongoDB
11 28 (11.38%) SQL Server Integration Services
12 27 (10.98%) Apache Cassandra
13 21 (8.54%) PostgreSQL
14 20 (8.13%) MySQL
15 19 (7.72%) Redis
16 18 (7.32%) SQL Server Analysis Services
17 17 (6.91%) Data Warehouse
18 14 (5.69%) RDBMS
18 14 (5.69%) SQL Server Reporting Services
19 13 (5.28%) Data Lake
Development Applications
1 45 (18.29%) Git (software)
2 32 (13.01%) Jenkins
3 22 (8.94%) JIRA
4 21 (8.54%) Subversion
5 20 (8.13%) Maven
5 20 (8.13%) RStudio
5 20 (8.13%) SQL Server Management Studio (SSMS)
6 14 (5.69%) Eclipse
7 11 (4.47%) Grunt
8 9 (3.66%) Gerrit
8 9 (3.66%) IntelliJ
9 8 (3.25%) Cucumber
9 8 (3.25%) Jasmine
9 8 (3.25%) Selenium
10 7 (2.85%) Bitbucket
11 6 (2.44%) NetBeans
11 6 (2.44%) Talend Open Studio
12 5 (2.03%) Gradle
12 5 (2.03%) JIRA Agile
13 3 (1.22%) SonarQube
General
1 63 (25.61%) Finance
2 29 (11.79%) Banking
3 20 (8.13%) Retail
4 9 (3.66%) Front Office
5 7 (2.85%) Telecoms
6 5 (2.03%) Advertising
6 5 (2.03%) Games
6 5 (2.03%) Marketing
7 4 (1.63%) Investment Banking
7 4 (1.63%) Legal
8 3 (1.22%) Electronics
8 3 (1.22%) Manufacturing
9 1 (0.41%) Financial Institution
9 1 (0.41%) iGaming
9 1 (0.41%) Retail Banking
Job Titles
1 97 (39.43%) Developer
2 44 (17.89%) Java Developer
3 38 (15.45%) Data Engineer
4 37 (15.04%) Big Data Engineer
5 33 (13.41%) Big Data Developer
6 32 (13.01%) Senior Developer
7 27 (10.98%) Architect
7 27 (10.98%) Hadoop Engineer
7 27 (10.98%) Senior Java Developer
8 24 (9.76%) Data Manager
9 23 (9.35%) Data Architect
10 19 (7.72%) Big Data Architect
11 17 (6.91%) Scala Developer
12 14 (5.69%) Software Engineer
13 11 (4.47%) Big Data Software Engineer
14 10 (4.07%) Hadoop Developer
15 9 (3.66%) Angular Developer
16 7 (2.85%) JavaScript Developer
16 7 (2.85%) Node.js Developer
16 7 (2.85%) Technical Leader
Libraries, Frameworks & Software Standards
1 96 (39.02%) Kafka
2 43 (17.48%) REST
3 42 (17.07%) Spring
4 28 (11.38%) Node.js
5 27 (10.98%) AngularJS
5 27 (10.98%) HTML
6 26 (10.57%) CSS
7 23 (9.35%) JSON
8 21 (8.54%) Apache Avro
9 18 (7.32%) Web Services
10 17 (6.91%) RabbitMQ
10 17 (6.91%) Spring Boot
11 14 (5.69%) Elastic Stack
12 12 (4.88%) SOAP
13 10 (4.07%) Java EE
13 10 (4.07%) Samza
13 10 (4.07%) Spark Streaming
14 9 (3.66%) D3.js
14 9 (3.66%) Drools
14 9 (3.66%) XML
Miscellaneous
1 38 (15.45%) Analytical Skills
2 36 (14.63%) Computer Science
3 27 (10.98%) Greenfield Project
4 22 (8.94%) Algorithms
5 18 (7.32%) Data Structures
6 14 (5.69%) Fintech
7 7 (2.85%) Virtual Team
8 5 (2.03%) Credit Risk
8 5 (2.03%) Linux Command Line
8 5 (2.03%) Replication
8 5 (2.03%) Social Network
9 4 (1.63%) Clustering
9 4 (1.63%) Enterprise Software
10 3 (1.22%) Derivative
10 3 (1.22%) Driving Licence
10 3 (1.22%) Fixed Income
10 3 (1.22%) Market Risk
11 2 (0.81%) FMCG
11 2 (0.81%) Foreign Exchange (FX)
11 2 (0.81%) Online Games
Operating Systems
1 70 (28.46%) Linux
2 23 (9.35%) Windows
3 19 (7.72%) Unix
4 3 (1.22%) CoreOS
5 2 (0.81%) Mesosphere
5 2 (0.81%) Ubuntu
Processes & Methodologies
1 110 (44.72%) Agile Software Development
2 101 (41.06%) MapReduce
3 89 (36.18%) Analytics
4 59 (23.98%) ETL
5 50 (20.33%) Continuous Integration
5 50 (20.33%) TDD
6 40 (16.26%) Business Intelligence
7 37 (15.04%) OO
8 33 (13.41%) Software Engineering
9 30 (12.20%) Scrum
10 27 (10.98%) Data Management
10 27 (10.98%) Machine Learning
10 27 (10.98%) Microservices
11 25 (10.16%) Visualisation
12 24 (9.76%) Data Architecture
13 23 (9.35%) BDD
13 23 (9.35%) Data Modelling
14 21 (8.54%) Use Case
15 17 (6.91%) Business Analytics
15 17 (6.91%) Test Automation
Programming Languages
1 156 (63.41%) Java
2 96 (39.02%) Python
3 82 (33.33%) Scala
4 78 (31.71%) SQL
5 53 (21.54%) JavaScript
6 36 (14.63%) R
7 27 (10.98%) T-SQL
8 16 (6.50%) Ruby
9 15 (6.10%) C#
10 14 (5.69%) Bash Shell
11 13 (5.28%) C
12 8 (3.25%) Shell Script
13 7 (2.85%) C++
14 6 (2.44%) Go
14 6 (2.44%) PHP
15 3 (1.22%) CoffeeScript
15 3 (1.22%) F#
15 3 (1.22%) VBA
16 2 (0.81%) Clojure
16 2 (0.81%) MATLAB
Qualifications
1 58 (23.58%) Degree
2 29 (11.79%) Computer Science Degree
3 12 (4.88%) Security Cleared
4 8 (3.25%) CEH
4 8 (3.25%) CISSP
4 8 (3.25%) GCIH
4 8 (3.25%) OSCP
5 7 (2.85%) DV Cleared
6 5 (2.03%) SC Cleared
7 3 (1.22%) Master's Degree
7 3 (1.22%) PhD
8 2 (0.81%) Numerate Degree
9 1 (0.41%) CTC Cleared
Quality Assurance & Compliance
1 10 (4.07%) Data Quality
2 8 (3.25%) GPG13
2 8 (3.25%) ISO/IEC 27001
2 8 (3.25%) QA
3 3 (1.22%) GCP
4 1 (0.41%) GDPR
4 1 (0.41%) MiFID
4 1 (0.41%) SLA
System Software
1 46 (18.70%) Apache Flume
2 29 (11.79%) Apache ZooKeeper
3 28 (11.38%) Docker
4 8 (3.25%) Snort
5 3 (1.22%) VirtualBox
5 3 (1.22%) VMware Infrastructure
6 2 (0.81%) MapR-FS
6 2 (0.81%) Virtual Machines
7 1 (0.41%) Cronacle
Systems Management
1 36 (14.63%) Hadoop YARN
2 29 (11.79%) Oozie
3 22 (8.94%) Puppet
4 12 (4.88%) Ansible
4 12 (4.88%) Kubernetes
5 9 (3.66%) Opscode Chef
5 9 (3.66%) Terraform
6 8 (3.25%) Ambari
6 8 (3.25%) Suricata
7 7 (2.85%) Nagios
8 6 (2.44%) Mesos
8 6 (2.44%) QRadar
9 5 (2.03%) Graphite
9 5 (2.03%) Kibana
9 5 (2.03%) Salt
10 3 (1.22%) Fluentd
10 3 (1.22%) logstash
10 3 (1.22%) ZABBIX
11 2 (0.81%) Docker Swarm
11 2 (0.81%) yum
Vendors
1 36 (14.63%) Cloudera
2 26 (10.57%) Oracle
3 25 (10.16%) Hortonworks
4 17 (6.91%) MapR
5 16 (6.50%) Google
6 11 (4.47%) Microsoft
7 10 (4.07%) Tableau
8 9 (3.66%) Splunk
8 9 (3.66%) Teradata
9 8 (3.25%) FireEye
9 8 (3.25%) Fortinet
9 8 (3.25%) Palo Alto
9 8 (3.25%) Sophos
9 8 (3.25%) Sourcefire
9 8 (3.25%) TippingPoint
10 7 (2.85%) Atlassian
11 6 (2.44%) AlienVault
11 6 (2.44%) ArcSight
11 6 (2.44%) LogRhythm
11 6 (2.44%) Talend