HDFS Contracts

The following table provides summary statistics for contract job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 3 months to 28 June 2017 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS) UK
Location	3 months to 28 Jun 2017	Same period 2016	Same period 2015
Rank	482	587	601
Rank change year-on-year	+105	+14
Contract jobs citing HDFS	128	64	44
As % of all contract IT jobs advertised in the UK	0.27%	0.12%	0.080%
As % of the System Software category	2.43%	0.94%	0.76%
Number of daily rates quoted	85	47	25
Median daily rate	£500	£513	£525
Median daily rate % change year-on-year	-2.43%	-2.38%
90% offered a daily rate of more than	£376	£408	£409
10% offered a daily rate of more than	£675	£698	£778
UK excluding London median daily rate	£401	£500	£450
% change year-on-year	-19.90%	+11.11%
Number of hourly rates quoted	2	0	4
Median hourly rate	£50.00	-	£26.00
UK excluding London median hourly rate	£50.00	-	£26.00

HDFS is in the system software skills category. The next table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for system software skills.

Contract System Software Skills UK
Contract job vacancies with a requirement for system software skills	5277	6781	5827
As % of all contract IT job vacancies advertised in the UK	11.20%	12.59%	10.57%
Number of daily rates quoted	5009	7094	5670
Median daily rate	£410	£400	£350
Median daily rate % change year-on-year	+2.50%	+14.28%
90% offered a daily rate of more than	£175	£200	£180
10% offered a daily rate of more than	£575	£575	£500
UK excluding London median daily rate	£375	£350	£330
% change year-on-year	+7.14%	+6.06%
Number of hourly rates quoted	431	681	778
Median hourly rate	£16.92	£20.00	£17.98
Median hourly rate % change year-on-year	-15.40%	+11.23%
90% offered a hourly rate of more than	£10.50	£11.25	£10.50
10% offered a hourly rate of more than	£49.19	£42.50	£40.00
UK excluding London median hourly rate	£16.00	£22.50	£17.50
% change year-on-year	-28.88%	+28.57%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a percentage of all IT jobs advertised.

HDFS
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing HDFS.

Contractor daily rate trend for HDFS in the UK

HDFS
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing HDFS over the 3 months to 28 June 2017.

Contractor daily rate histogram for HDFS in the UK

HDFS
Top 8 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the UK over the 3 months to 28 June 2017. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 3 month period last year.

Location	Rank Change on Same Period Last Year	Matching Contract IT Job Ads	Median Daily Rate Last 3 Months	Median Daily Rate % Change on Same Period Last Year	Live Job Vacancies
England	+101	126	£500	-2.43%	20
London	+55	85	£550	-	14
UK excluding London	+57	42	£401	-19.90%	7
South East	+50	31	£401	-9.74%	2
South West	0	5	£388	-35.41%
East of England	-	3	-	-	2
North of England	+58	2	£405	-	2
Northern Ireland	-48	1	-	-

HDFS
Top 30 Co-occurring IT Skills

For the 6 months to 28 June 2017, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all contract ads with a requirement for HDFS.

1	266 (98.15%)	Hadoop
2	199 (73.43%)	Big Data
3	178 (65.68%)	Apache Spark
4	161 (59.41%)	Apache Hive
5	154 (56.83%)	HBase
6	148 (54.61%)	Java
7	116 (42.80%)	Kafka
8	113 (41.70%)	Linux
8	113 (41.70%)	Python
9	103 (38.01%)	Hadoop YARN
10	92 (33.95%)	Scala
11	91 (33.58%)	Agile Software Development
12	89 (32.84%)	Cloudera
13	84 (31.00%)	DevOps
14	70 (25.83%)	MapReduce

15	69 (25.46%)	Unix
16	68 (25.09%)	Apache ZooKeeper
17	63 (23.25%)	Finance
18	60 (22.14%)	Impala
18	60 (22.14%)	Apache Pig
19	57 (21.03%)	Apache Flume
19	57 (21.03%)	Ansible
20	55 (20.30%)	Apache Solr
20	55 (20.30%)	SQL
21	53 (19.56%)	TDD
22	52 (19.19%)	Amazon AWS
22	52 (19.19%)	Puppet
23	51 (18.82%)	Sqoop
23	51 (18.82%)	Oozie
24	49 (18.08%)	Apache Cassandra

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
Applications
Business Applications
Cloud Services
Communications & Networking
Database & Business Intelligence
Development Applications
General
Job Titles
Libraries, Frameworks & Software Standards
Miscellaneous
Operating Systems
Processes & Methodologies
Programming Languages
Qualifications
Quality Assurance & Compliance
System Software
Systems Management
Vendors

Application Platforms
1	178 (65.68%)	Apache Spark
2	60 (22.14%)	Apache Pig
3	55 (20.30%)	Apache Solr
4	35 (12.92%)	Apache
5	33 (12.18%)	Elasticsearch
6	17 (6.27%)	Confluence
7	13 (4.80%)	Tomcat
8	5 (1.85%)	WebSphere
9	4 (1.48%)	Cloud Foundry
9	4 (1.48%)	JBoss
10	3 (1.11%)	Mule
10	3 (1.11%)	nginx
10	3 (1.11%)	OpenStack
10	3 (1.11%)	SAS
11	2 (0.74%)	Adobe Experience Manager
11	2 (0.74%)	Apache Airflow
11	2 (0.74%)	CMS
12	1 (0.37%)	GemFire
12	1 (0.37%)	Jetty

Applications
1	1 (0.37%)	Microsoft Excel
1	1 (0.37%)	MS Visio

Business Applications
1	4 (1.48%)	SAP Oil and Gas

Cloud Services
1	52 (19.19%)	Amazon AWS
2	26 (9.59%)	Amazon S3
3	19 (7.01%)	Amazon EC2
4	12 (4.43%)	Cloud Computing
4	12 (4.43%)	Virtual Private Cloud
5	10 (3.69%)	GitHub
6	9 (3.32%)	Amazon EMR
7	8 (2.95%)	Amazon CloudWatch
7	8 (2.95%)	Amazon ELB
7	8 (2.95%)	Google Cloud Platform
8	7 (2.58%)	IaaS
9	6 (2.21%)	Microsoft Azure
10	5 (1.85%)	AWS CloudFormation
11	3 (1.11%)	AWS CloudTrail
11	3 (1.11%)	AWS Lambda
11	3 (1.11%)	PaaS
12	1 (0.37%)	Loggly
12	1 (0.37%)	Sumo Logic

Communications & Networking
1	40 (14.76%)	Kerberos
2	11 (4.06%)	LDAP
3	8 (2.95%)	Network Security
3	8 (2.95%)	SSL
4	3 (1.11%)	Firewall
4	3 (1.11%)	FTP
4	3 (1.11%)	OpenLDAP
4	3 (1.11%)	tcpdump
4	3 (1.11%)	Wireshark
5	2 (0.74%)	HTTP
6	1 (0.37%)	DNS
6	1 (0.37%)	Internet
6	1 (0.37%)	TCP/IP

Database & Business Intelligence
1	266 (98.15%)	Hadoop
2	199 (73.43%)	Big Data
3	161 (59.41%)	Apache Hive
4	154 (56.83%)	HBase
5	60 (22.14%)	Impala
6	51 (18.82%)	Sqoop
7	49 (18.08%)	Apache Cassandra
8	45 (16.61%)	NoSQL
9	31 (11.44%)	MySQL
10	29 (10.70%)	MongoDB
11	27 (9.96%)	PostgreSQL
12	22 (8.12%)	Data Warehouse
13	15 (5.54%)	Relational Database
14	14 (5.17%)	Neo4j
15	12 (4.43%)	RDBMS
16	10 (3.69%)	SQL Server
17	7 (2.58%)	Amazon Redshift
17	7 (2.58%)	Apache Zeppelin
18	6 (2.21%)	Data Lake
18	6 (2.21%)	IBM BigInsights

Development Applications
1	47 (17.34%)	Jenkins
2	38 (14.02%)	Git (software)
3	24 (8.86%)	JIRA
4	19 (7.01%)	Cucumber
5	12 (4.43%)	Maven
6	10 (3.69%)	Gradle
7	8 (2.95%)	IntelliJ
8	7 (2.58%)	Subversion
9	6 (2.21%)	webpack
10	5 (1.85%)	GitLab
10	5 (1.85%)	JUnit
11	4 (1.48%)	Eclipse
11	4 (1.48%)	Hudson
11	4 (1.48%)	Sonatype Nexus
11	4 (1.48%)	Talend Open Studio
12	3 (1.11%)	Artifactory
13	2 (0.74%)	Babel
13	2 (0.74%)	gulp
13	2 (0.74%)	Karma Test Runner
13	2 (0.74%)	Protractor

General
1	63 (23.25%)	Finance
2	41 (15.13%)	Banking
3	12 (4.43%)	Front Office
4	10 (3.69%)	Marketing
5	9 (3.32%)	Investment Banking
5	9 (3.32%)	Telecoms
6	5 (1.85%)	Retail
7	4 (1.48%)	Electronics
7	4 (1.48%)	Manufacturing
8	3 (1.11%)	Law
9	2 (0.74%)	Billing
9	2 (0.74%)	Pharmaceutical
10	1 (0.37%)	Financial Institution
10	1 (0.37%)	French Language
10	1 (0.37%)	Games
10	1 (0.37%)	Retail Banking

Job Titles
1	89 (32.84%)	Developer
2	68 (25.09%)	Data Engineer
3	62 (22.88%)	Big Data Engineer
4	49 (18.08%)	Big Data Developer
5	39 (14.39%)	Administrator
6	38 (14.02%)	Hadoop Engineer
7	29 (10.70%)	Hadoop Administrator
8	28 (10.33%)	DevOps Engineer
9	23 (8.49%)	Hadoop Developer
9	23 (8.49%)	Linux Administrator
10	21 (7.75%)	Java Developer
11	17 (6.27%)	Apache Spark Developer
12	16 (5.90%)	Scala Developer
13	13 (4.80%)	Architect
14	12 (4.43%)	Consultant
15	11 (4.06%)	Linux Systems Administrator
15	11 (4.06%)	Systems Administrator
16	10 (3.69%)	Data Architect
16	10 (3.69%)	Python Developer
17	9 (3.32%)	Java Engineer

Libraries, Frameworks & Software Standards
1	116 (42.80%)	Kafka
2	41 (15.13%)	Spring
3	33 (12.18%)	REST
4	29 (10.70%)	Spark Streaming
5	20 (7.38%)	Spring Boot
6	18 (6.64%)	Drools
7	16 (5.90%)	Apache Camel
8	15 (5.54%)	Middleware
9	13 (4.80%)	Elastic Stack
9	13 (4.80%)	HTML
10	12 (4.43%)	HTML5
11	10 (3.69%)	Web Services
12	9 (3.32%)	JSON
12	9 (3.32%)	Spring MVC
13	8 (2.95%)	AngularJS
13	8 (2.95%)	Node.js
13	8 (2.95%)	RabbitMQ
14	7 (2.58%)	Hibernate
14	7 (2.58%)	J2EE
14	7 (2.58%)	PySpark

Miscellaneous
1	34 (12.55%)	Algorithms
2	20 (7.38%)	Computer Science
3	16 (5.90%)	Analytical Skills
4	11 (4.06%)	Foreign Exchange (FX)
5	8 (2.95%)	Derivative
6	6 (2.21%)	Fintech
6	6 (2.21%)	Fixed Income
6	6 (2.21%)	Greenfield Project
6	6 (2.21%)	Linux Command Line
7	5 (1.85%)	User Experience
7	5 (1.85%)	Virtual Team
8	4 (1.48%)	Distributed Denial-of-Service
9	3 (1.11%)	Data Centre
9	3 (1.11%)	Grid Computing
9	3 (1.11%)	Management Information System
9	3 (1.11%)	Market Risk
9	3 (1.11%)	Private Cloud
10	2 (0.74%)	Equities
10	2 (0.74%)	Equity Derivative
10	2 (0.74%)	Replication

Operating Systems
1	113 (41.70%)	Linux
2	69 (25.46%)	Unix
3	36 (13.28%)	Red Hat Enterprise Linux
4	22 (8.12%)	Ubuntu
5	19 (7.01%)	Debian
6	15 (5.54%)	Windows
7	9 (3.32%)	CentOS
8	6 (2.21%)	Fedora
9	4 (1.48%)	CoreOS
10	2 (0.74%)	Apple iOS
11	1 (0.37%)	Android
11	1 (0.37%)	Windows Server
11	1 (0.37%)	Windows Server 2008

Processes & Methodologies
1	91 (33.58%)	Agile Software Development
2	84 (31.00%)	DevOps
3	70 (25.83%)	MapReduce
4	53 (19.56%)	TDD
5	47 (17.34%)	ETL
6	44 (16.24%)	Continuous Integration
7	39 (14.39%)	Microservices
8	38 (14.02%)	Scrum
9	35 (12.92%)	Data Architecture
10	34 (12.55%)	Analytics
11	28 (10.33%)	BDD
12	26 (9.59%)	Continuous Delivery
13	23 (8.49%)	Machine Learning
13	23 (8.49%)	Performance Tuning
14	21 (7.75%)	Open Source
15	19 (7.01%)	Build Automation
15	19 (7.01%)	Data Analysis
15	19 (7.01%)	Data Analytics
16	18 (6.64%)	Data Engineering
16	18 (6.64%)	Desktop Virtualisation

Programming Languages
1	148 (54.61%)	Java
2	113 (41.70%)	Python
3	92 (33.95%)	Scala
4	55 (20.30%)	SQL
5	22 (8.12%)	Bash Shell
6	20 (7.38%)	Shell Script
7	19 (7.01%)	JavaScript
8	17 (6.27%)	R
9	14 (5.17%)	Ruby
10	10 (3.69%)	Perl
11	6 (2.21%)	C++
11	6 (2.21%)	ES6
12	4 (1.48%)	Korn
12	4 (1.48%)	PowerShell
12	4 (1.48%)	T-SQL
13	3 (1.11%)	C#
13	3 (1.11%)	Go
13	3 (1.11%)	PL/SQL
14	2 (0.74%)	Julia
15	1 (0.37%)	COBOL

Qualifications
1	31 (11.44%)	Security Cleared
2	22 (8.12%)	SC Cleared
3	20 (7.38%)	Degree
4	12 (4.43%)	Computer Science Degree
5	4 (1.48%)	CISA
5	4 (1.48%)	CISM
5	4 (1.48%)	CISSP
6	3 (1.11%)	DBS Check
6	3 (1.11%)	DV Cleared
7	2 (0.74%)	BPSS Clearance
7	2 (0.74%)	ISTQB
7	2 (0.74%)	ISTQB Foundation Certification
7	2 (0.74%)	Master's Degree
8	1 (0.37%)	PhD

Quality Assurance & Compliance
1	8 (2.95%)	MiFID
2	7 (2.58%)	QA
3	5 (1.85%)	SLA
4	4 (1.48%)	Data Quality
4	4 (1.48%)	FedRAMP
4	4 (1.48%)	PCI DSS
5	2 (0.74%)	CDISC
5	2 (0.74%)	Dodd-Frank
5	2 (0.74%)	EMIR

System Software
1	68 (25.09%)	Apache ZooKeeper
2	57 (21.03%)	Apache Flume
3	36 (13.28%)	Docker
4	9 (3.32%)	Active Directory
5	7 (2.58%)	VMware Infrastructure
6	5 (1.85%)	vCloud
7	3 (1.11%)	Oracle RAC
7	3 (1.11%)	Squid
8	2 (0.74%)	vSphere
9	1 (0.37%)	VirtualBox
9	1 (0.37%)	VMware ESXi

Systems Management
1	103 (38.01%)	Hadoop YARN
2	57 (21.03%)	Ansible
3	52 (19.19%)	Puppet
4	51 (18.82%)	Oozie
5	40 (14.76%)	Terraform
6	32 (11.81%)	Opscode Chef
7	26 (9.59%)	Kubernetes
8	11 (4.06%)	Ambari
8	11 (4.06%)	Kibana
8	11 (4.06%)	Nagios
9	9 (3.32%)	Mesos
10	7 (2.58%)	logstash
11	4 (1.48%)	ZABBIX
12	3 (1.11%)	Consul
12	3 (1.11%)	Grafana
12	3 (1.11%)	Graphite
12	3 (1.11%)	Packer
12	3 (1.11%)	rkt
12	3 (1.11%)	Salt
12	3 (1.11%)	Sysdig

Vendors
1	89 (32.84%)	Cloudera
2	37 (13.65%)	Hortonworks
3	21 (7.75%)	Oracle
4	12 (4.43%)	Red Hat
4	12 (4.43%)	VMware
5	10 (3.69%)	Google
6	9 (3.32%)	IBM
7	8 (2.95%)	Tableau
8	6 (2.21%)	Ab Initio
9	5 (1.85%)	Splunk
10	4 (1.48%)	MapR
10	4 (1.48%)	MicroStrategy
10	4 (1.48%)	SaltStack
10	4 (1.48%)	SAP
10	4 (1.48%)	Talend
11	3 (1.11%)	Alteryx
11	3 (1.11%)	ForgeRock
11	3 (1.11%)	Informatica
12	2 (0.74%)	Adobe
12	2 (0.74%)	Cadence