Period
to 19 June 2018

The following table provides summary statistics for permanent job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited HDFS over the 6 months to 19 June 2018 with a comparison to the same period in the previous 2 years.

Hadoop Distributed File System (HDFS)
UK
6 months to
19 Jun 2018
Same period 2017 Same period 2016
Rank 738 924 910
Rank change year-on-year +186 -14 +249
Permanent jobs citing HDFS 463 253 366
As % of all permanent IT jobs advertised in the UK 0.27% 0.15% 0.18%
As % of the System Software category 1.88% 1.18% 1.47%
Number of salaries quoted 320 201 287
UK median annual salary £67,500 £70,000 £70,000
Median salary % change year-on-year -3.57% - -
10th Percentile £37,500 £48,750 £45,000
90th Percentile £96,375 £115,000 £120,000
UK excluding London median annual salary £60,000 £55,000 £55,000
% change year-on-year +9.09% - +22.22%

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for system software skills.

All System Software Skills
UK
Permanent vacancies with a requirement for system software skills 24,617 21,472 24,874
As % of all permanent IT jobs advertised in the UK 14.30% 12.48% 12.46%
Number of salaries quoted 19,975 18,050 21,062
UK median annual salary £47,500 £42,500 £41,500
Median salary % change year-on-year +11.76% +2.41% +3.75%
10th Percentile £25,000 £23,750 £23,000
90th Percentile £80,000 £75,000 £70,000
UK excluding London median annual salary £41,500 £37,500 £37,500
% change year-on-year +10.67% - +2.74%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a percentage of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing HDFS.

Salary trend for HDFS in the UK

HDFS
Salary Histogram

The salary distribution of IT jobs citing HDFS over the 6 months to 19 June 2018.

Salary histogram for HDFS in the UK

HDFS
Top 14 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing HDFS within the UK over the 6 months to 19 June 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +178 439 £70,000 - 36
London +66 251 £76,250 +1.67% 22
UK excluding London +137 207 £60,000 +9.09% 20
South East +74 92 £57,500 +4.55% 6
East of England +48 41 £72,500 +31.82% 2
North of England +8 32 £43,750 -20.45% 4
North West -8 18 £40,000 -27.27% 3
Scotland -28 17 £50,000 -37.50% 6
West Midlands +27 13 £60,000 - 1
Midlands +20 13 £60,000 - 1
Yorkshire +33 12 £47,500 -13.64% 1
South West +8 10 £58,750 -2.08% 1
Wales +17 2 £58,000 -10.77%
North East -20 2 £67,500 +35.00%

For the 6 months to 19 June 2018, IT jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for HDFS.

1 432 (93.30%) Hadoop
2 319 (68.90%) Apache Spark
3 302 (65.23%) Big Data
4 286 (61.77%) Apache Hive
5 268 (57.88%) Python
6 222 (47.95%) Java
7 210 (45.36%) SQL
8 207 (44.71%) HBase
9 192 (41.47%) Analytics
10 172 (37.15%) Scala
11 166 (35.85%) Linux
12 159 (34.34%) Kafka
13 156 (33.69%) Amazon AWS
14 149 (32.18%) Data Science
15 147 (31.75%) Machine Learning
16 137 (29.59%) ETL
17 133 (28.73%) Agile Software Development
18 131 (28.29%) Sqoop
19 122 (26.35%) Impala
20 119 (25.70%) MapReduce
21 111 (23.97%) R
22 106 (22.89%) Apache Flume
23 101 (21.81%) Oozie
23 101 (21.81%) Data Warehouse
24 95 (20.52%) Data Lake
25 91 (19.65%) Visualisation
26 90 (19.44%) Degree
27 84 (18.14%) SQL Server
28 83 (17.93%) Hortonworks
29 82 (17.71%) Oracle

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 319 (68.90%) Apache Spark
2 68 (14.69%) Apache
3 59 (12.74%) Apache Pig
4 42 (9.07%) Elasticsearch
5 33 (7.13%) Apache Solr
6 18 (3.89%) SAS
7 8 (1.73%) OpenStack
8 6 (1.30%) Tomcat
9 4 (0.86%) Apache Geronimo
9 4 (0.86%) GlassFish
9 4 (0.86%) JBoss
9 4 (0.86%) Oracle SOA Suite
9 4 (0.86%) WebLogic
9 4 (0.86%) WebSphere
10 3 (0.65%) nginx
10 3 (0.65%) Oracle Workflow
11 2 (0.43%) Confluence
12 1 (0.22%) Apache Airflow
12 1 (0.22%) IIS
12 1 (0.22%) WildFly
Applications
1 14 (3.02%) MS Visio
2 4 (0.86%) Weka
3 2 (0.43%) Microsoft Project
4 1 (0.22%) CPLEX
4 1 (0.22%) SPSS
Business Applications
1 4 (0.86%) Oracle 11i
1 4 (0.86%) Oracle Assets
1 4 (0.86%) Oracle EBS
1 4 (0.86%) Oracle EBS R12
1 4 (0.86%) Oracle General Ledger
1 4 (0.86%) Oracle iProcurement
1 4 (0.86%) Oracle Payables
1 4 (0.86%) Oracle Receivables
Cloud Services
1 156 (33.69%) Amazon AWS
2 76 (16.41%) Microsoft Azure
3 58 (12.53%) Amazon EMR
4 42 (9.07%) Amazon S3
5 35 (7.56%) Amazon Kinesis
6 30 (6.48%) Azure Data Factory
7 29 (6.26%) Serverless
8 28 (6.05%) AWS Lambda
9 21 (4.54%) Google Cloud Platform
10 14 (3.02%) GitHub
11 10 (2.16%) Amazon EC2
11 10 (2.16%) IaaS
12 9 (1.94%) AWS CloudFormation
13 8 (1.73%) Loggly
13 8 (1.73%) Sumo Logic
14 5 (1.08%) Amazon ELB
14 5 (1.08%) Cloud Computing
14 5 (1.08%) CloudFront
14 5 (1.08%) Slack
14 5 (1.08%) Virtual Private Cloud
Communications & Networking
1 26 (5.62%) Internet
1 26 (5.62%) Kerberos
2 19 (4.10%) Sourcefire
3 12 (2.59%) Network Security
4 11 (2.38%) DNS
5 9 (1.94%) HTTP
6 6 (1.30%) Firewall
7 3 (0.65%) SSL
8 2 (0.43%) TCP/IP
9 1 (0.22%) Broadband
9 1 (0.22%) Wireshark
Database & Business Intelligence
1 432 (93.30%) Hadoop
2 302 (65.23%) Big Data
3 286 (61.77%) Apache Hive
4 207 (44.71%) HBase
5 131 (28.29%) Sqoop
6 122 (26.35%) Impala
7 101 (21.81%) Data Warehouse
8 95 (20.52%) Data Lake
9 84 (18.14%) SQL Server
10 80 (17.28%) NoSQL
11 62 (13.39%) Amazon Redshift
12 59 (12.74%) MongoDB
13 55 (11.88%) Apache Cassandra
13 55 (11.88%) Relational Database
14 48 (10.37%) SQL Server Integration Services
15 42 (9.07%) Parquet
16 38 (8.21%) SQL Server Analysis Services
17 36 (7.78%) Power BI
18 35 (7.56%) MySQL
19 34 (7.34%) PostgreSQL
Development Applications
1 56 (12.10%) Jenkins
2 41 (8.86%) Git (software)
3 25 (5.40%) TeamCity
4 23 (4.97%) JUnit
5 15 (3.24%) Maven
6 12 (2.59%) JIRA
7 10 (2.16%) GitLab
8 8 (1.73%) Vagrant
9 6 (1.30%) Ant
9 6 (1.30%) git-flow
10 5 (1.08%) ERwin
10 5 (1.08%) Grunt
10 5 (1.08%) Liquibase
11 4 (0.86%) Sonatype Nexus
11 4 (0.86%) Team Foundation Server
12 3 (0.65%) CA Release Automation
12 3 (0.65%) NeoLoad
12 3 (0.65%) Octopus Deploy
12 3 (0.65%) The Grinder
12 3 (0.65%) Travis CI
General
1 67 (14.47%) Finance
2 50 (10.80%) Retail
3 32 (6.91%) Marketing
4 31 (6.70%) Advertising
5 24 (5.18%) Banking
6 17 (3.67%) Investment Banking
7 12 (2.59%) Financial Institution
8 11 (2.38%) Front Office
9 9 (1.94%) Military
10 5 (1.08%) Legal
11 4 (0.86%) Automotive
11 4 (0.86%) Telecoms
12 1 (0.22%) Aerospace
12 1 (0.22%) Electronics
12 1 (0.22%) Games
12 1 (0.22%) Law
12 1 (0.22%) Manufacturing
Job Titles
1 149 (32.18%) Data Engineer
2 89 (19.22%) Big Data Engineer
3 74 (15.98%) Data Scientist
4 65 (14.04%) Developer
5 45 (9.72%) Hadoop Engineer
6 44 (9.50%) Architect
7 41 (8.86%) Consultant
7 41 (8.86%) Data Architect
8 36 (7.78%) Java Developer
9 35 (7.56%) Senior Data Scientist
10 29 (6.26%) Software Engineer
11 28 (6.05%) Systems Engineer
12 24 (5.18%) Big Data Architect
13 22 (4.75%) Java Engineer
14 20 (4.32%) Data Consultant
14 20 (4.32%) Machine Learning Scientist
14 20 (4.32%) Technical Architect
15 19 (4.10%) Senior Developer
16 18 (3.89%) Cloud Architect
16 18 (3.89%) Data Analytics Architect
Libraries, Frameworks & Software Standards
1 159 (34.34%) Kafka
2 49 (10.58%) Elastic Stack
3 43 (9.29%) Web Services
4 36 (7.78%) Node.js
4 36 (7.78%) TensorFlow
5 34 (7.34%) JSON
6 32 (6.91%) MLlib
7 28 (6.05%) .NET
7 28 (6.05%) Spring
8 26 (5.62%) Apache Avro
8 26 (5.62%) REST
9 25 (5.40%) PySpark
10 23 (4.97%) Natural Language Processing
11 22 (4.75%) CSS
11 22 (4.75%) HTML
12 21 (4.54%) NumPy
12 21 (4.54%) Pandas
12 21 (4.54%) scikit-learn
13 20 (4.32%) SciPy
13 20 (4.32%) Theano
Miscellaneous
1 67 (14.47%) Fintech
2 63 (13.61%) Algorithms
3 58 (12.53%) Data Structures
4 56 (12.10%) Computer Science
5 50 (10.80%) Analytical Skills
6 28 (6.05%) Greenfield Project
7 23 (4.97%) Driving Licence
8 22 (4.75%) Replication
9 21 (4.54%) Linux Command Line
10 12 (2.59%) Tandem
11 10 (2.16%) Equities
11 10 (2.16%) Management Information System
12 9 (1.94%) Enterprise Software
12 9 (1.94%) PKI
13 8 (1.73%) Cyberthreat
13 8 (1.73%) Derivative
13 8 (1.73%) Security Operations Centre
14 7 (1.51%) Clustering
14 7 (1.51%) Public Cloud
15 6 (1.30%) Taxonomies
Operating Systems
1 166 (35.85%) Linux
2 32 (6.91%) Windows
3 30 (6.48%) Unix
4 8 (1.73%) CentOS
4 8 (1.73%) CoreOS
5 7 (1.51%) Ubuntu
6 5 (1.08%) Amazon Linux
7 4 (0.86%) Red Hat Enterprise Linux
8 2 (0.43%) VMS
8 2 (0.43%) Windows Server
9 1 (0.22%) DC/OS
9 1 (0.22%) Windows Server 2008
Processes & Methodologies
1 192 (41.47%) Analytics
2 149 (32.18%) Data Science
3 147 (31.75%) Machine Learning
4 137 (29.59%) ETL
5 133 (28.73%) Agile Software Development
6 119 (25.70%) MapReduce
7 91 (19.65%) Visualisation
8 72 (15.55%) Business Intelligence
9 66 (14.25%) Data Engineering
9 66 (14.25%) Problem-Solving
10 62 (13.39%) Data Analysis
11 61 (13.17%) Continuous Integration
12 59 (12.74%) Software Engineering
13 57 (12.31%) Data Visualisation
14 56 (12.10%) DevOps
15 55 (11.88%) Predictive Modelling
16 51 (11.02%) Data Analytics
17 50 (10.80%) Data Modelling
18 46 (9.94%) Open Source
19 43 (9.29%) Data Transformation
Programming Languages
1 268 (57.88%) Python
2 222 (47.95%) Java
3 210 (45.36%) SQL
4 172 (37.15%) Scala
5 111 (23.97%) R
6 47 (10.15%) HiveQL
7 39 (8.42%) Bash Shell
8 38 (8.21%) JavaScript
9 35 (7.56%) Perl
9 35 (7.56%) Ruby
10 33 (7.13%) C
11 27 (5.83%) Go
12 22 (4.75%) C++
13 21 (4.54%) PL/SQL
14 17 (3.67%) C#
14 17 (3.67%) Shell Script
15 16 (3.46%) PHP
15 16 (3.46%) T-SQL
16 15 (3.24%) MATLAB
17 11 (2.38%) Java 8
Qualifications
1 90 (19.44%) Degree
2 29 (6.26%) Computer Science Degree
3 28 (6.05%) Numerate Degree
4 23 (4.97%) Security Cleared
5 20 (4.32%) DV Cleared
6 19 (4.10%) CEH
6 19 (4.10%) CISSP
6 19 (4.10%) GCIH
6 19 (4.10%) OSCP
7 9 (1.94%) Master's Degree
8 4 (0.86%) ISTQB
8 4 (0.86%) PhD
8 4 (0.86%) PRINCE2 Certification
9 3 (0.65%) Microsoft Certification
9 3 (0.65%) SC Cleared
10 2 (0.43%) MCSA
10 2 (0.43%) MCSD
11 1 (0.22%) CTC Cleared
11 1 (0.22%) ITIL Foundation Certificate
Quality Assurance & Compliance
1 30 (6.48%) QA
2 14 (3.02%) Data Quality
3 8 (1.73%) GPG13
3 8 (1.73%) ISO/IEC 27001
4 5 (1.08%) GCP
5 1 (0.22%) GDPR
5 1 (0.22%) SLA
System Software
1 106 (22.89%) Apache Flume
2 36 (7.78%) Docker
3 21 (4.54%) Apache ZooKeeper
4 19 (4.10%) Active Directory
4 19 (4.10%) Snort
5 18 (3.89%) VMware Infrastructure
6 12 (2.59%) Hyper-V
7 11 (2.38%) Virtual Machines
8 8 (1.73%) VirtualBox
9 3 (0.65%) Oracle RAC
9 3 (0.65%) Xen
10 1 (0.22%) Cronacle
10 1 (0.22%) vSphere
Systems Management
1 101 (21.81%) Oozie
2 59 (12.74%) Hadoop YARN
3 28 (6.05%) Puppet
4 27 (5.83%) Ansible
5 25 (5.40%) Kubernetes
6 22 (4.75%) Kibana
6 22 (4.75%) Mesos
7 19 (4.10%) Ambari
7 19 (4.10%) Suricata
8 13 (2.81%) Opscode Chef
9 10 (2.16%) Nagios
9 10 (2.16%) Salt
10 9 (1.94%) ZABBIX
11 8 (1.73%) logstash
11 8 (1.73%) QRadar
12 6 (1.30%) Terraform
13 5 (1.08%) Prometheus
14 4 (0.86%) Docker Swarm
14 4 (0.86%) Grafana
15 3 (0.65%) Rancher
Vendors
1 83 (17.93%) Hortonworks
2 82 (17.71%) Oracle
3 78 (16.85%) Tableau
4 74 (15.98%) Cloudera
5 60 (12.96%) WorldPay
6 53 (11.45%) Microsoft
7 28 (6.05%) Sybase
8 27 (5.83%) Google
9 26 (5.62%) SAP
9 26 (5.62%) Talend
10 19 (4.10%) Informatica
10 19 (4.10%) TippingPoint
11 18 (3.89%) Atlassian
11 18 (3.89%) VMware
12 17 (3.67%) Splunk
13 16 (3.46%) Cognos
14 15 (3.24%) MapR
15 14 (3.02%) Netezza
16 11 (2.38%) Salesforce.com
17 9 (1.94%) SaltStack