HDFS Jobs

The following table provides summary statistics for permanent job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited HDFS over the 3 months to 26 June 2017 with a comparison to the same period in the previous 2 years.

Hadoop Distributed File System (HDFS) UK
Location	3 months to 26 Jun 2017	Same period 2016	Same period 2015
Rank	746	777	903
Rank change year-on-year	+31	+126
Permanent jobs citing HDFS	128	156	81
As % of all permanent IT jobs advertised in the UK	0.14%	0.16%	0.073%
As % of the System Software category	1.11%	1.26%	0.62%
Number of salaries quoted	98	129	62
Median salary	£70,000	£80,000	£75,000
Median salary % change year-on-year	-12.50%	+6.66%
90% offered a salary of more than	£55,000	£47,000	£50,000
10% offered a salary of more than	£105,000	£120,000	£124,000
UK excluding London median salary	£56,250	£51,250	£50,000
% change year-on-year	+9.75%	+2.50%

HDFS is in the system software skills category. The next table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for system software skills.

System Software Skills UK
Permanent job vacancies with a requirement for system software skills	11512	12340	13028
As % of all permanent IT job vacancies advertised in the UK	12.87%	12.71%	11.75%
Number of salaries quoted	9775	10512	10778
Median salary	£44,000	£42,500	£41,000
Median salary % change year-on-year	+3.52%	+3.65%
90% offered a salary of more than	£26,000	£25,000	£25,000
10% offered a salary of more than	£70,000	£65,000	£67,500
UK excluding London median salary	£38,000	£39,000	£37,500
% change year-on-year	-2.56%	+4.00%

HDFS
Job Vacancy Trend

The job posting trend of jobs advertised citing HDFS as a proportion of all permanent or contract IT jobs with a match in the System Software category.

HDFS
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing HDFS.

HDFS
Salary Histogram

The salary distribution of IT jobs citing HDFS over the 3 months to 26 June 2017.

HDFS
Top 30 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing HDFS within the UK over the 3 months to 26 June 2017. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 3 month period last year.

Location	Rank Change on Same Period Last Year	Matching Permanent IT Job Ads	Median Salary Last 3 Months	Median Salary % Change on Same Period Last Year	Live Job Vacancies
England	+10	126	£70,000	-12.50%	23
London	+17	102	£70,000	-17.64%	19
City of London	+32	21	£67,500	-32.50%	2
South East	+47	9	£55,000	+10.00%	2
North West	-30	9	£78,750	+120.27%
Manchester	-31	5	£59,250	+65.73%
Hampshire	+16	4	£55,000	+10.00%	1
Surrey	-	4	£50,000	-
Farnborough	-	4	£55,000	-	1
Cheshire	-	4	£80,000	-
East of England	-	3	-	-
West End of London	-	3	£57,500	-	1
Guildford	-	3	£50,000	-
South West	-	2	£60,000	-
Scotland	-	2	£55,000	-	1
Cambridgeshire	-	2	-	-
Reading	+7	1	-	-	1
Berkshire	+6	1	-	-	1
Yorkshire	-12	1	-	-	1
Edinburgh	-	1	£55,000	-
Glasgow	-	1	-	-	1
Norfolk	-	1	-	-
Malmesbury	-	1	£60,000	-
North Yorkshire	-	1	-	-
Bristol	-	1	£60,000	-
Wiltshire	-	1	£60,000	-
Cambridge	-	1	-	-
Uxbridge	-	1	-	-
Norwich	-	1	-	-
York	-	1	-	-

HDFS
Top 30 Co-occurring IT Skills

For the 6 months to 26 June 2017, IT jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all job ads with a requirement for HDFS.

1	219 (85.88%)	Hadoop
2	213 (83.53%)	Big Data
3	188 (73.73%)	Apache Hive
4	165 (64.71%)	Apache Spark
5	141 (55.29%)	Java
6	131 (51.37%)	HBase
7	114 (44.71%)	Kafka
8	111 (43.53%)	MapReduce
9	97 (38.04%)	Python
10	89 (34.90%)	Apache Pig
11	88 (34.51%)	Analytics
12	87 (34.12%)	NoSQL
13	86 (33.73%)	Agile Software Development
14	73 (28.63%)	Finance
15	70 (27.45%)	Apache

15	70 (27.45%)	Linux
15	70 (27.45%)	Scala
16	65 (25.49%)	SQL
17	58 (22.75%)	Analytical Skills
18	56 (21.96%)	Oozie
19	53 (20.78%)	Sqoop
20	50 (19.61%)	Hortonworks
20	50 (19.61%)	Cloudera
21	46 (18.04%)	ETL
22	44 (17.25%)	Unix
23	42 (16.47%)	Apache Cassandra
24	40 (15.69%)	Impala
25	35 (13.73%)	Hadoop YARN
25	35 (13.73%)	Relational Database
25	35 (13.73%)	Apache Flume

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
Applications
Cloud Services
Communications & Networking
Database & Business Intelligence
Development Applications
General
Job Titles
Libraries, Frameworks & Software Standards
Miscellaneous
Operating Systems
Processes & Methodologies
Programming Languages
Qualifications
Quality Assurance & Compliance
System Software
Systems Management
Vendors

Application Platforms
1	165 (64.71%)	Apache Spark
2	89 (34.90%)	Apache Pig
3	70 (27.45%)	Apache
4	17 (6.67%)	Elasticsearch
5	14 (5.49%)	Apache Solr
6	9 (3.53%)	SAS
6	9 (3.53%)	Tomcat
7	4 (1.57%)	Apache Geronimo
7	4 (1.57%)	Cloud Foundry
7	4 (1.57%)	GemFire
7	4 (1.57%)	GlassFish
7	4 (1.57%)	JBoss
7	4 (1.57%)	WebSphere
8	3 (1.18%)	OpenStack
9	2 (0.78%)	Mule
10	1 (0.39%)	Confluence
10	1 (0.39%)	Oracle Coherence

Applications
1	4 (1.57%)	Weka
2	2 (0.78%)	Microsoft Excel

Cloud Services
1	27 (10.59%)	Amazon AWS
2	12 (4.71%)	Cloud Computing
2	12 (4.71%)	GitHub
2	12 (4.71%)	Microsoft Azure
3	11 (4.31%)	Google Cloud Platform
4	9 (3.53%)	PaaS
5	7 (2.75%)	IaaS
6	4 (1.57%)	SaaS
7	3 (1.18%)	Loggly
7	3 (1.18%)	Sumo Logic
8	2 (0.78%)	Amazon EC2
9	1 (0.39%)	Amazon EMR
9	1 (0.39%)	AWS Lambda
9	1 (0.39%)	DigitalOcean

Communications & Networking
1	9 (3.53%)	Kerberos
2	8 (3.14%)	Internet
3	6 (2.35%)	LDAP
4	4 (1.57%)	HTTP
4	4 (1.57%)	Network Security
5	3 (1.18%)	DNS
6	2 (0.78%)	Broadband
7	1 (0.39%)	SSL

Database & Business Intelligence
1	219 (85.88%)	Hadoop
2	213 (83.53%)	Big Data
3	188 (73.73%)	Apache Hive
4	131 (51.37%)	HBase
5	87 (34.12%)	NoSQL
6	53 (20.78%)	Sqoop
7	42 (16.47%)	Apache Cassandra
8	40 (15.69%)	Impala
9	35 (13.73%)	Relational Database
10	33 (12.94%)	MongoDB
11	24 (9.41%)	SQL Server
12	21 (8.24%)	Data Warehouse
12	21 (8.24%)	MySQL
13	15 (5.88%)	RDBMS
14	13 (5.10%)	SQL Server Integration Services
15	12 (4.71%)	PostgreSQL
16	11 (4.31%)	Data Mining
16	11 (4.31%)	Parquet
17	10 (3.92%)	CouchDB
18	9 (3.53%)	HAWQ

Development Applications
1	24 (9.41%)	Jenkins
2	23 (9.02%)	Git (software)
3	17 (6.67%)	Subversion
4	16 (6.27%)	Gradle
5	14 (5.49%)	Maven
6	13 (5.10%)	RStudio
7	12 (4.71%)	SQL Server Management Studio (SSMS)
8	7 (2.75%)	Talend Open Studio
9	6 (2.35%)	Eclipse
10	5 (1.96%)	JIRA
10	5 (1.96%)	JUnit
11	4 (1.57%)	Ant
11	4 (1.57%)	Vagrant
12	2 (0.78%)	Atlassian Bamboo
12	2 (0.78%)	FindBugs
12	2 (0.78%)	Grunt
12	2 (0.78%)	TeamCity
13	1 (0.39%)	Artifactory
13	1 (0.39%)	Bitbucket
13	1 (0.39%)	JMeter

General
1	73 (28.63%)	Finance
2	25 (9.80%)	Retail
3	19 (7.45%)	Banking
4	10 (3.92%)	Advertising
4	10 (3.92%)	Games
5	8 (3.14%)	Telecoms
6	7 (2.75%)	Front Office
6	7 (2.75%)	Legal
7	6 (2.35%)	Investment Banking
8	5 (1.96%)	Marketing
9	3 (1.18%)	Manufacturing
10	1 (0.39%)	Automotive
10	1 (0.39%)	Financial Institution

Job Titles
1	61 (23.92%)	Developer
2	53 (20.78%)	Architect
3	47 (18.43%)	Data Engineer
4	45 (17.65%)	Data Architect
5	37 (14.51%)	Hadoop Engineer
6	34 (13.33%)	Big Data Engineer
7	29 (11.37%)	Big Data Architect
8	25 (9.80%)	Java Developer
9	17 (6.67%)	Scala Developer
10	16 (6.27%)	Big Data Developer
10	16 (6.27%)	Data Manager
11	15 (5.88%)	Administrator
11	15 (5.88%)	Hadoop Administrator
11	15 (5.88%)	Hadoop Architect
11	15 (5.88%)	Senior Developer
12	13 (5.10%)	Data Scientist
12	13 (5.10%)	Hadoop Developer
13	12 (4.71%)	E-Commerce Director
13	12 (4.71%)	Senior Data Engineer
13	12 (4.71%)	Technology Director

Libraries, Frameworks & Software Standards
1	114 (44.71%)	Kafka
2	20 (7.84%)	Spring
3	18 (7.06%)	Akka
3	18 (7.06%)	Apache Avro
3	18 (7.06%)	JSON
4	17 (6.67%)	REST
5	11 (4.31%)	AngularJS
6	10 (3.92%)	Middleware
7	9 (3.53%)	jQuery
7	9 (3.53%)	Natural Language Processing
7	9 (3.53%)	RabbitMQ
7	9 (3.53%)	Spark Streaming
8	8 (3.14%)	Apache Flex
8	8 (3.14%)	Elastic Stack
8	8 (3.14%)	Node.js
8	8 (3.14%)	RESTful
9	7 (2.75%)	HTML
9	7 (2.75%)	J2EE
9	7 (2.75%)	Web Services
9	7 (2.75%)	XML

Miscellaneous
1	58 (22.75%)	Analytical Skills
2	31 (12.16%)	Computer Science
3	22 (8.63%)	Virtual Team
4	19 (7.45%)	Algorithms
5	14 (5.49%)	Linux Command Line
6	12 (4.71%)	Greenfield Project
7	11 (4.31%)	Data Structures
8	10 (3.92%)	Replication
9	9 (3.53%)	Foreign Exchange (FX)
10	7 (2.75%)	Social Network
11	5 (1.96%)	Clustering
11	5 (1.96%)	Derivative
12	4 (1.57%)	Data Centre
12	4 (1.57%)	Fintech
12	4 (1.57%)	Fixed Income
12	4 (1.57%)	Management Information System
13	3 (1.18%)	Security Operations Centre
14	2 (0.78%)	Grid Computing
14	2 (0.78%)	PKI
14	2 (0.78%)	User Experience

Operating Systems
1	70 (27.45%)	Linux
2	44 (17.25%)	Unix
3	6 (2.35%)	Red Hat Enterprise Linux
4	3 (1.18%)	CoreOS
5	2 (0.78%)	Windows
6	1 (0.39%)	Mesosphere
6	1 (0.39%)	Ubuntu

Processes & Methodologies
1	111 (43.53%)	MapReduce
2	88 (34.51%)	Analytics
3	86 (33.73%)	Agile Software Development
4	46 (18.04%)	ETL
5	33 (12.94%)	E-Commerce
6	29 (11.37%)	Mentoring
7	28 (10.98%)	Open Source
7	28 (10.98%)	TDD
8	27 (10.59%)	Machine Learning
8	27 (10.59%)	Software Engineering
9	26 (10.20%)	Data Architecture
10	25 (9.80%)	Visualisation
11	24 (9.41%)	Business Intelligence
11	24 (9.41%)	Stakeholder Management
12	22 (8.63%)	OO
13	21 (8.24%)	Data Analytics
14	20 (7.84%)	DevOps
14	20 (7.84%)	Scrum
15	19 (7.45%)	BDD
15	19 (7.45%)	Data Management

Programming Languages
1	141 (55.29%)	Java
2	97 (38.04%)	Python
3	70 (27.45%)	Scala
4	65 (25.49%)	SQL
5	21 (8.24%)	C++
6	20 (7.84%)	R
7	19 (7.45%)	JavaScript
8	17 (6.67%)	T-SQL
9	16 (6.27%)	PHP
9	16 (6.27%)	Ruby
10	15 (5.88%)	Shell Script
11	12 (4.71%)	Perl
12	11 (4.31%)	C
13	10 (3.92%)	C#
14	6 (2.35%)	Bash Shell
15	5 (1.96%)	PL/SQL
16	3 (1.18%)	F#
16	3 (1.18%)	PHP 5
16	3 (1.18%)	PHP 7
17	2 (0.78%)	Haskell

Qualifications
1	34 (13.33%)	Degree
2	17 (6.67%)	PhD
3	13 (5.10%)	Computer Science Degree
4	10 (3.92%)	Postgraduate
5	5 (1.96%)	CISSP
6	3 (1.18%)	CEH
6	3 (1.18%)	GCIH
6	3 (1.18%)	OSCP
6	3 (1.18%)	Security Cleared
7	2 (0.78%)	Numerate Degree
7	2 (0.78%)	OCA
7	2 (0.78%)	OCP
7	2 (0.78%)	Oracle Certification
7	2 (0.78%)	SC Cleared
8	1 (0.39%)	CBAP
8	1 (0.39%)	IIBA

Quality Assurance & Compliance
1	11 (4.31%)	QA
2	7 (2.75%)	GCP
3	5 (1.96%)	Data Quality
4	2 (0.78%)	SLA
5	1 (0.39%)	Basel III
5	1 (0.39%)	Dodd-Frank
5	1 (0.39%)	GPG13
5	1 (0.39%)	ISO/IEC 27001

System Software
1	35 (13.73%)	Apache Flume
2	22 (8.63%)	Apache ZooKeeper
3	13 (5.10%)	Docker
4	8 (3.14%)	VMware Infrastructure
5	3 (1.18%)	Snort
5	3 (1.18%)	vCloud
5	3 (1.18%)	VirtualBox
6	2 (0.78%)	Oracle RAC
7	1 (0.39%)	Cronacle
7	1 (0.39%)	KVM
7	1 (0.39%)	Virtual Machines
7	1 (0.39%)	Xen

Systems Management
1	56 (21.96%)	Oozie
2	35 (13.73%)	Hadoop YARN
3	15 (5.88%)	Ansible
3	15 (5.88%)	Puppet
4	14 (5.49%)	Ambari
5	12 (4.71%)	Opscode Chef
6	8 (3.14%)	Terraform
7	6 (2.35%)	Graphite
7	6 (2.35%)	Kubernetes
7	6 (2.35%)	Mesos
8	5 (1.96%)	Kibana
8	5 (1.96%)	logstash
9	4 (1.57%)	Nagios
9	4 (1.57%)	Salt
10	3 (1.18%)	Fluentd
10	3 (1.18%)	QRadar
10	3 (1.18%)	ZABBIX
11	1 (0.39%)	Consul
11	1 (0.39%)	CSIRT
11	1 (0.39%)	Rancher

Vendors
1	50 (19.61%)	Cloudera
1	50 (19.61%)	Hortonworks
2	31 (12.16%)	MapR
3	22 (8.63%)	Oracle
4	13 (5.10%)	Google
5	10 (3.92%)	Tableau
6	8 (3.14%)	Splunk
6	8 (3.14%)	VMware
7	7 (2.75%)	IBM
7	7 (2.75%)	Talend
8	6 (2.35%)	Red Hat
9	5 (1.96%)	Greenplum
9	5 (1.96%)	Vertica
10	4 (1.57%)	Alteryx
11	3 (1.18%)	ArcSight
11	3 (1.18%)	LogRhythm
11	3 (1.18%)	MuleSoft
11	3 (1.18%)	New Relic
11	3 (1.18%)	Rackspace
11	3 (1.18%)	SaltStack