Period
to 23 February 2019

The following table provides summary statistics for permanent job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited HDFS over the 6 months to 23 February 2019 with a comparison to the same period in the previous 2 years.

Hadoop Distributed File System (HDFS)
UK
6 months to
23 Feb 2019
Same period 2018 Same period 2017
Rank 836 897 899
Rank change year-on-year +61 +2 +107
Permanent jobs citing HDFS 278 322 288
As % of all permanent IT jobs advertised in the UK 0.18% 0.18% 0.16%
As % of the System Software category 1.18% 1.37% 1.34%
Number of salaries quoted 214 258 227
UK median annual salary £72,500 £70,000 £75,000
Median salary % change year-on-year +3.57% -6.67% +7.14%
10th Percentile £49,337 £42,500 £49,750
90th Percentile £97,500 £100,750 £115,000
UK excluding London median annual salary £59,000 £60,000 £63,750
% change year-on-year -1.67% -5.88% +10.87%

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for system software skills.

All System Software Skills
UK
Permanent vacancies with a requirement for system software skills 23,532 23,585 21,522
As % of all permanent IT jobs advertised in the UK 15.64% 13.38% 12.26%
Number of salaries quoted 19,243 19,465 17,852
UK median annual salary £50,000 £45,000 £42,500
Median salary % change year-on-year +11.11% +5.88% +6.25%
10th Percentile £25,750 £24,250 £23,250
90th Percentile £82,500 £77,500 £72,500
UK excluding London median annual salary £42,500 £40,000 £37,500
% change year-on-year +6.25% +6.67% -

HDFS
Job Vacancy Trend

Job postings citing HDFS as a percentage of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing HDFS.

Salary trend for HDFS in the UK

HDFS
Salary Histogram

The salary distribution of IT jobs citing HDFS over the 6 months to 23 February 2019.

Salary histogram for HDFS in the UK

HDFS
Top 15 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing HDFS within the UK over the 6 months to 23 February 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +51 267 £72,500 +3.57% 41
London +59 164 £75,000 - 26
UK excluding London +34 111 £59,000 -1.67% 17
North of England +60 33 £49,337 +9.64% 2
East of England +53 28 £72,500 -3.33% 5
South West +10 19 £50,000 - 1
Yorkshire +38 16 £49,337 +16.09% 1
Midlands +44 12 £75,000 +25.00%
South East -16 11 £37,500 -34.78% 7
East Midlands - 10 £75,000 -
North East - 9 £49,337 -
North West +36 8 £70,000 +55.56% 1
Scotland +14 5 £63,750 +6.69% 2
Wales - 3 £54,000 -
West Midlands +31 2 - -

For the 6 months to 23 February 2019, IT jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for HDFS.

1 233 (83.81%) Hadoop
2 210 (75.54%) Big Data
3 202 (72.66%) Apache Spark
4 192 (69.06%) Apache Hive
4 192 (69.06%) Java
5 169 (60.79%) Agile Software Development
6 158 (56.83%) HBase
7 156 (56.12%) Python
8 133 (47.84%) Kafka
9 116 (41.73%) Scala
10 114 (41.01%) NoSQL
11 109 (39.21%) Linux
12 102 (36.69%) SQL
13 100 (35.97%) ETL
14 94 (33.81%) Apache Cassandra
15 92 (33.09%) Amazon AWS
15 92 (33.09%) Analytics
15 92 (33.09%) Data Science
16 88 (31.65%) Machine Learning
16 88 (31.65%) MongoDB
17 82 (29.50%) Oracle
17 82 (29.50%) Sqoop
18 74 (26.62%) Apache Flume
18 74 (26.62%) Hortonworks
19 70 (25.18%) Cloudera
19 70 (25.18%) Oozie
20 64 (23.02%) Data Engineering
21 61 (21.94%) MapReduce
21 61 (21.94%) Data Analysis
21 61 (21.94%) Unix

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 202 (72.66%) Apache Spark
2 55 (19.78%) Apache
3 49 (17.63%) Apache Pig
4 48 (17.27%) Elasticsearch
5 38 (13.67%) Apache Solr
6 22 (7.91%) SAS
7 18 (6.47%) JBoss
8 14 (5.04%) Tomcat
8 14 (5.04%) WebSphere
8 14 (5.04%) WildFly
9 13 (4.68%) Apache Geronimo
9 13 (4.68%) GlassFish
10 8 (2.88%) Cloud Foundry
11 6 (2.16%) Jetty
12 4 (1.44%) Apache Airflow
13 1 (0.36%) Confluence
13 1 (0.36%) GemFire
13 1 (0.36%) OpenStack
13 1 (0.36%) WebLogic
Applications
1 13 (4.68%) Weka
2 1 (0.36%) Microsoft Office
Business Applications
1 1 (0.36%) Adobe Campaign
Cloud Services
1 92 (33.09%) Amazon AWS
2 54 (19.42%) GitHub
3 39 (14.03%) Microsoft Azure
4 29 (10.43%) Cloud Computing
5 28 (10.07%) Amazon EMR
6 17 (6.12%) Amazon S3
7 16 (5.76%) Google Cloud Platform
8 12 (4.32%) Serverless
9 10 (3.60%) Amazon EC2
9 10 (3.60%) AWS Lambda
10 7 (2.52%) Amazon Kinesis
11 6 (2.16%) SaaS
12 4 (1.44%) AWS CloudFormation
12 4 (1.44%) Azure Data Factory
13 3 (1.08%) Azure Machine Learning
13 3 (1.08%) IaaS
13 3 (1.08%) PaaS
14 2 (0.72%) AWS Data Pipeline
15 1 (0.36%) IBM Cloud
15 1 (0.36%) Sumo Logic
Communications & Networking
1 11 (3.96%) Kerberos
2 10 (3.60%) Internet
3 7 (2.52%) Firewall
3 7 (2.52%) SSL
4 6 (2.16%) DNS
4 6 (2.16%) HTTP
4 6 (2.16%) TCP/IP
5 3 (1.08%) Network Security
6 2 (0.72%) Sourcefire
6 2 (0.72%) VPN
7 1 (0.36%) HTTPS
7 1 (0.36%) VLAN
7 1 (0.36%) WAN
Database & Business Intelligence
1 233 (83.81%) Hadoop
2 210 (75.54%) Big Data
3 192 (69.06%) Apache Hive
4 158 (56.83%) HBase
5 114 (41.01%) NoSQL
6 94 (33.81%) Apache Cassandra
7 88 (31.65%) MongoDB
8 82 (29.50%) Sqoop
9 56 (20.14%) Data Warehouse
9 56 (20.14%) Impala
10 51 (18.35%) Relational Database
11 48 (17.27%) PostgreSQL
12 42 (15.11%) MySQL
13 41 (14.75%) SQL Server
14 35 (12.59%) Spark SQL
15 32 (11.51%) Redis
16 31 (11.15%) RDBMS
17 29 (10.43%) Data Lake
17 29 (10.43%) Neo4j
18 26 (9.35%) Flink
Development Applications
1 48 (17.27%) Git (software)
2 31 (11.15%) Jenkins
3 30 (10.79%) Maven
4 26 (9.35%) IntelliJ
5 18 (6.47%) Eclipse
5 18 (6.47%) Gradle
6 13 (4.68%) Ant
7 11 (3.96%) NetBeans
8 10 (3.60%) JIRA
9 6 (2.16%) JUnit
9 6 (2.16%) Team Foundation Server
10 4 (1.44%) Selenium
10 4 (1.44%) Subversion
11 2 (0.72%) Cucumber
11 2 (0.72%) ERwin
11 2 (0.72%) FindBugs
11 2 (0.72%) TeamCity
12 1 (0.36%) Grunt
12 1 (0.36%) Protractor
12 1 (0.36%) Vagrant
General
1 55 (19.78%) Finance
2 27 (9.71%) Retail
3 23 (8.27%) Banking
4 9 (3.24%) Telecoms
5 7 (2.52%) Marketing
6 5 (1.80%) Advertising
6 5 (1.80%) Legal
7 3 (1.08%) Electronics
7 3 (1.08%) Investment Banking
7 3 (1.08%) Law
8 2 (0.72%) Financial Institution
8 2 (0.72%) Manufacturing
8 2 (0.72%) Pharmaceutical
Job Titles
1 106 (38.13%) Data Engineer
2 84 (30.22%) Developer
3 69 (24.82%) Java Developer
4 60 (21.58%) Big Data Engineer
5 29 (10.43%) Big Data Developer
5 29 (10.43%) Senior Developer
6 27 (9.71%) Python Engineer
7 26 (9.35%) Architect
8 24 (8.63%) Senior Java Developer
9 23 (8.27%) Senior Data Engineer
10 19 (6.83%) Data Scientist
11 18 (6.47%) Lead Developer
11 18 (6.47%) Lead Java Developer
12 17 (6.12%) Senior Data Scientist
13 13 (4.68%) Core Java Developer
13 13 (4.68%) Data Architect
14 11 (3.96%) Big Data Architect
14 11 (3.96%) Software Engineer
15 10 (3.60%) Senior Software Engineer
16 9 (3.24%) Senior Big Data Developer
Libraries, Frameworks & Software Standards
1 133 (47.84%) Kafka
2 50 (17.99%) Spring
3 42 (15.11%) Web Services
4 39 (14.03%) Spark Streaming
5 33 (11.87%) REST
6 31 (11.15%) Hibernate
7 26 (9.35%) Spring Boot
8 24 (8.63%) JPA
8 24 (8.63%) SOAP
9 23 (8.27%) PySpark
10 21 (7.55%) Apache Avro
11 19 (6.83%) HTML
11 19 (6.83%) RabbitMQ
11 19 (6.83%) React
12 18 (6.47%) CSS
13 17 (6.12%) Apache NiFi
14 16 (5.76%) Node.js
15 14 (5.04%) JAX-RS
15 14 (5.04%) Redux
16 13 (4.68%) TensorFlow
Miscellaneous
1 37 (13.31%) Analytical Skills
2 20 (7.19%) Algorithms
3 14 (5.04%) Data Structures
3 14 (5.04%) Greenfield Project
4 10 (3.60%) Fintech
4 10 (3.60%) Virtual Team
5 8 (2.88%) Data Centre
5 8 (2.88%) Linux Command Line
6 7 (2.52%) Public Cloud
6 7 (2.52%) User Experience
7 6 (2.16%) Enterprise Software
8 5 (1.80%) Cloud Native
9 3 (1.08%) Life Science
10 2 (0.72%) Mainframe
10 2 (0.72%) Replication
10 2 (0.72%) Security Operations Centre
10 2 (0.72%) Self-Motivation
11 1 (0.36%) FMCG
11 1 (0.36%) Robotics
Operating Systems
1 109 (39.21%) Linux
2 61 (21.94%) Unix
3 17 (6.12%) Windows
4 13 (4.68%) Ubuntu
5 6 (2.16%) VMS
6 3 (1.08%) Red Hat Enterprise Linux
7 1 (0.36%) CentOS
7 1 (0.36%) CoreOS
7 1 (0.36%) DC/OS
Processes & Methodologies
1 169 (60.79%) Agile Software Development
2 100 (35.97%) ETL
3 92 (33.09%) Analytics
3 92 (33.09%) Data Science
4 88 (31.65%) Machine Learning
5 64 (23.02%) Data Engineering
6 61 (21.94%) Data Analysis
6 61 (21.94%) MapReduce
7 57 (20.50%) Data Integration
8 51 (18.35%) Software Engineering
9 48 (17.27%) Data Modelling
9 48 (17.27%) Open Source
10 46 (16.55%) Continuous Integration
11 45 (16.19%) DevOps
12 43 (15.47%) Mentoring
13 41 (14.75%) Computer Science
14 40 (14.39%) TDD
15 39 (14.03%) Use Case
16 38 (13.67%) Data Transformation
17 34 (12.23%) Coaching
Programming Languages
1 192 (69.06%) Java
2 156 (56.12%) Python
3 116 (41.73%) Scala
4 102 (36.69%) SQL
5 53 (19.06%) JavaScript
6 38 (13.67%) R
7 34 (12.23%) Ruby
8 24 (8.63%) T-SQL
9 23 (8.27%) Shell Script
10 20 (7.19%) Java 8
11 14 (5.04%) PL/SQL
12 13 (4.68%) Groovy
13 10 (3.60%) C#
13 10 (3.60%) C++
14 7 (2.52%) Bash Shell
15 4 (1.44%) Go
15 4 (1.44%) PHP
16 3 (1.08%) HiveQL
16 3 (1.08%) Perl
17 2 (0.72%) MATLAB
Qualifications
1 57 (20.50%) Degree
2 27 (9.71%) Computer Science Degree
3 4 (1.44%) AWS Certification
4 2 (0.72%) CEH
4 2 (0.72%) CISSP
4 2 (0.72%) GCIH
4 2 (0.72%) OSCP
4 2 (0.72%) Security Cleared
5 1 (0.36%) DV Cleared
5 1 (0.36%) PhD
5 1 (0.36%) SC Cleared
Quality Assurance & Compliance
1 27 (9.71%) QA
2 15 (5.40%) GDPR
3 10 (3.60%) Data Quality
4 6 (2.16%) IFRS
4 6 (2.16%) ISO/IEC 27001
5 4 (1.44%) PCI DSS
6 2 (0.72%) GPG13
7 1 (0.36%) EMIR
7 1 (0.36%) MiFID
System Software
1 74 (26.62%) Apache Flume
2 40 (14.39%) Docker
3 30 (10.79%) Apache ZooKeeper
4 9 (3.24%) Virtual Machines
5 8 (2.88%) Active Directory
6 2 (0.72%) Snort
7 1 (0.36%) VirtualBox
7 1 (0.36%) VMware Infrastructure
Systems Management
1 70 (25.18%) Oozie
2 51 (18.35%) Hadoop YARN
3 31 (11.15%) Kubernetes
4 23 (8.27%) Kibana
5 20 (7.19%) Puppet
6 12 (4.32%) logstash
7 10 (3.60%) Mesos
8 7 (2.52%) Ansible
8 7 (2.52%) Grafana
8 7 (2.52%) Prometheus
9 6 (2.16%) Ambari
10 4 (1.44%) Rundeck
11 2 (0.72%) QRadar
11 2 (0.72%) Suricata
11 2 (0.72%) Terraform
12 1 (0.36%) Apache Brooklyn
12 1 (0.36%) Graphite
12 1 (0.36%) Marathon
12 1 (0.36%) Salt
12 1 (0.36%) Single Sign-On
Vendors
1 82 (29.50%) Oracle
2 74 (26.62%) Hortonworks
3 70 (25.18%) Cloudera
4 31 (11.15%) Tableau
5 19 (6.83%) MapR
6 15 (5.40%) Google
7 13 (4.68%) Alteryx
8 10 (3.60%) Microsoft
9 9 (3.24%) Databricks
9 9 (3.24%) Informatica
9 9 (3.24%) SAP
9 9 (3.24%) Talend
9 9 (3.24%) Teradata
10 6 (2.16%) IBM
11 5 (1.80%) CA
11 5 (1.80%) Salesforce.com
12 4 (1.44%) Cognizant
12 4 (1.44%) Splunk
13 3 (1.08%) WorldPay
14 2 (0.72%) FireEye