Period
to 24 November 2017

The following table provides summary statistics for contract job vacancies advertised in England with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 24 November 2017 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
UK > England
6 months to
24 Nov 2017
Same period 2016 Same period 2015
Rank 476 594 697
Rank change year-on-year +118 +103 +80
Contract jobs citing HDFS 425 269 169
As % of all contract IT jobs advertised in England 0.45% 0.28% 0.16%
As % of the System Software category 3.51% 2.24% 1.57%
Number of daily rates quoted 306 153 112
England median daily rate £550 £550 £619
Median daily rate % change year-on-year - -11.11% -20.16%
10th Percentile £395 £410 £414
90th Percentile £700 £713 £675
UK median daily rate £550 £550 £619
% change year-on-year - -11.11% -20.16%
Number of hourly rates quoted 0 0 1
England median hourly rate - - £37.50
UK median hourly rate - - £37.50

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in England with a requirement for system software skills.

All System Software Skills
England
Contract vacancies with a requirement for system software skills 12124 12020 10775
As % of all contract IT jobs advertised in England 12.79% 12.32% 10.30%
Number of daily rates quoted 7943 7609 6233
England median daily rate £425 £413 £375
Median daily rate % change year-on-year +3.03% +10.00% +8.70%
10th Percentile £161 £175 £160
90th Percentile £613 £575 £550
UK median daily rate £425 £400 £370
% change year-on-year +6.25% +8.11% +9.63%
Number of hourly rates quoted 659 810 1168
England median hourly rate £16.50 £16.00 £16.00
Median hourly rate % change year-on-year +3.13% - +6.67%
10th Percentile £10.40 £10.50 £10.21
90th Percentile £47.30 £39.78 £38.75
UK median hourly rate £16.50 £15.88 £16.00
% change year-on-year +3.94% -0.78% +6.67%

HDFS
Job Vacancy Trend in England

Job postings citing HDFS as a percentage of all IT jobs advertised in England.

Job vacancy trend for HDFS in England

HDFS
Contractor Daily Rate Trend in England

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing HDFS in England.

Contractor daily rate trend for HDFS in England

HDFS
Contractor Daily Rate Histogram in England

The daily rate distribution of IT jobs citing HDFS in England over the 6 months to 24 November 2017.

Contractor daily rate histogram for HDFS in England

HDFS
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the England region over the 6 months to 24 November 2017. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London +80 316 £575 - 16
South East +44 80 £513 +7.89% 2
East of England +8 12 £488 +16.07%
North of England +53 9 £475 -4.52% 4
Yorkshire +24 7 £475 -3.06% 2
South West +1 7 £425 -29.17% 4
North West +27 2 £405 -29.10% 2
HDFS
UK

For the 6 months to 24 November 2017, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all contract ads across the England region with a requirement for HDFS.

1 413 (97.18%) Hadoop
2 386 (90.82%) Big Data
3 358 (84.24%) Apache Spark
4 297 (69.88%) Java
5 280 (65.88%) HBase
6 279 (65.65%) Kafka
7 231 (54.35%) Apache Hive
8 161 (37.88%) MapReduce
9 149 (35.06%) Apache Flume
10 133 (31.29%) Linux
10 133 (31.29%) Agile Software Development
10 133 (31.29%) TDD
11 126 (29.65%) Spring
12 120 (28.24%) Python
13 118 (27.76%) Hadoop YARN
14 115 (27.06%) PostgreSQL
15 113 (26.59%) Oozie
16 108 (25.41%) BDD
17 105 (24.71%) DevOps
17 105 (24.71%) Apache ZooKeeper
18 103 (24.24%) Sqoop
18 103 (24.24%) Scala
19 102 (24.00%) Kerberos
20 101 (23.76%) Apache Solr
21 99 (23.29%) Spring Boot
22 98 (23.06%) Data Architecture
22 98 (23.06%) REST
22 98 (23.06%) SQL
23 92 (21.65%) Microservices
24 91 (21.41%) Scrum

HDFS
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 358 (84.24%) Apache Spark
2 101 (23.76%) Apache Solr
3 85 (20.00%) Apache
4 51 (12.00%) Confluence
5 49 (11.53%) Elasticsearch
6 48 (11.29%) Apache Pig
7 28 (6.59%) Tomcat
8 15 (3.53%) Cloud Foundry
8 15 (3.53%) GemFire
9 7 (1.65%) JBoss
10 4 (0.94%) OpenStack
10 4 (0.94%) SAS
11 3 (0.71%) Jetty
11 3 (0.71%) Oracle Coherence
11 3 (0.71%) WebLogic
12 1 (0.24%) Mule
12 1 (0.24%) nginx
12 1 (0.24%) Oracle Workflow
12 1 (0.24%) WebSphere
Business Applications
1 2 (0.47%) SAP CRM
Cloud Services
1 73 (17.18%) Amazon AWS
2 70 (16.47%) GitHub
3 36 (8.47%) Bluemix
3 36 (8.47%) IBM Cloud
4 19 (4.47%) PaaS
5 11 (2.59%) Google Analytics
6 8 (1.88%) Amazon EMR
6 8 (1.88%) Amazon S3
7 6 (1.41%) Google Cloud Platform
8 5 (1.18%) IaaS
9 4 (0.94%) Amazon EC2
9 4 (0.94%) Amazon Kinesis
9 4 (0.94%) Cloud Computing
10 3 (0.71%) AWS CloudFormation
10 3 (0.71%) Microsoft Azure
10 3 (0.71%) Route 53
11 2 (0.47%) Google Tag Manager
12 1 (0.24%) Data Lake Analytics
12 1 (0.24%) Loggly
12 1 (0.24%) Sumo Logic
Communications & Networking
1 102 (24.00%) Kerberos
2 7 (1.65%) Internet
3 5 (1.18%) SSL
4 4 (0.94%) HTTP
5 3 (0.71%) Network Security
6 2 (0.47%) tcpdump
6 2 (0.47%) Wireshark
7 1 (0.24%) DNS
7 1 (0.24%) LDAP
7 1 (0.24%) OpenLDAP
Database & Business Intelligence
1 413 (97.18%) Hadoop
2 386 (90.82%) Big Data
3 280 (65.88%) HBase
4 231 (54.35%) Apache Hive
5 115 (27.06%) PostgreSQL
6 103 (24.24%) Sqoop
7 69 (16.24%) Impala
8 57 (13.41%) NoSQL
9 55 (12.94%) Apache Cassandra
10 39 (9.18%) MongoDB
11 30 (7.06%) SQL Server
12 28 (6.59%) Data Warehouse
13 27 (6.35%) Relational Database
14 21 (4.94%) RDBMS
15 20 (4.71%) Data Lake
16 19 (4.47%) DataStax
17 18 (4.24%) Oracle ODI
18 16 (3.76%) IBM BigInsights
18 16 (3.76%) Oracle Database
19 13 (3.06%) Parquet
Development Applications
1 83 (19.53%) Jenkins
2 75 (17.65%) Cucumber
3 58 (13.65%) JIRA
4 52 (12.24%) Git (software)
5 26 (6.12%) IntelliJ
6 24 (5.65%) Gradle
7 13 (3.06%) JUnit
8 12 (2.82%) GitLab
8 12 (2.82%) Team Foundation Server
9 11 (2.59%) Maven
10 7 (1.65%) Talend Open Studio
11 6 (1.41%) Eclipse
12 5 (1.18%) SonarQube
12 5 (1.18%) Subversion
12 5 (1.18%) TeamCity
13 3 (0.71%) Codeship
13 3 (0.71%) Mercurial
13 3 (0.71%) Travis CI
14 2 (0.47%) Atlassian Bamboo
14 2 (0.47%) Sonatype Nexus
General
1 85 (20.00%) Finance
2 38 (8.94%) Banking
3 26 (6.12%) Telecoms
4 11 (2.59%) Marketing
5 10 (2.35%) Electronics
5 10 (2.35%) Manufacturing
6 7 (1.65%) Investment Banking
7 6 (1.41%) Retail
8 5 (1.18%) Front Office
8 5 (1.18%) Retail Banking
9 4 (0.94%) Pharmaceutical
10 2 (0.47%) Legal
11 1 (0.24%) Games
11 1 (0.24%) Law
11 1 (0.24%) Publishing
Job Titles
1 134 (31.53%) Developer
2 122 (28.71%) Data Engineer
3 106 (24.94%) Big Data Engineer
4 81 (19.06%) Big Data Developer
5 76 (17.88%) Hadoop Engineer
6 53 (12.47%) Architect
7 47 (11.06%) Java Developer
8 42 (9.88%) Data Architect
9 38 (8.94%) Hadoop Developer
10 36 (8.47%) Big Data Architect
11 24 (5.65%) Java Engineer
11 24 (5.65%) Scala Developer
12 23 (5.41%) Administrator
12 23 (5.41%) Senior Hadoop Engineer
13 20 (4.71%) Apache Spark Developer
13 20 (4.71%) Senior Data Engineer
14 19 (4.47%) Hadoop Administrator
15 14 (3.29%) Hadoop Architect
15 14 (3.29%) Senior Developer
15 14 (3.29%) Senior Java Engineer
Libraries, Frameworks & Software Standards
1 279 (65.65%) Kafka
2 126 (29.65%) Spring
3 99 (23.29%) Spring Boot
4 98 (23.06%) REST
5 70 (16.47%) Drools
6 60 (14.12%) PySpark
7 58 (13.65%) Apache Camel
8 46 (10.82%) RESTful
9 41 (9.65%) Spark Streaming
10 32 (7.53%) Web Services
11 30 (7.06%) RabbitMQ
12 29 (6.82%) J2EE
12 29 (6.82%) JSON
13 25 (5.88%) Middleware
14 23 (5.41%) Apache Avro
15 17 (4.00%) SOAP
16 16 (3.76%) TinkerPop
17 14 (3.29%) Akka
17 14 (3.29%) XML
18 13 (3.06%) Spring Integration
Miscellaneous
1 66 (15.53%) Algorithms
2 52 (12.24%) Analytical Skills
3 27 (6.35%) Greenfield Project
4 19 (4.47%) Linux Command Line
4 19 (4.47%) Virtual Team
5 18 (4.24%) Replication
6 11 (2.59%) Computer Science
6 11 (2.59%) Market Risk
7 7 (1.65%) Derivative
8 5 (1.18%) Interest Rate Derivative
9 4 (0.94%) Credit Risk
9 4 (0.94%) Foreign Exchange (FX)
10 3 (0.71%) Credit Derivative
10 3 (0.71%) Data Centre
10 3 (0.71%) Equities
11 2 (0.47%) Data Structures
11 2 (0.47%) Management Information System
12 1 (0.24%) Clustering
12 1 (0.24%) Grid Computing
12 1 (0.24%) Media Streaming
Operating Systems
1 133 (31.29%) Linux
2 72 (16.94%) Unix
3 20 (4.71%) Red Hat Enterprise Linux
4 14 (3.29%) Ubuntu
5 6 (1.41%) CentOS
6 5 (1.18%) Windows
7 3 (0.71%) AIX
8 2 (0.47%) Android
8 2 (0.47%) Apple iOS
9 1 (0.24%) CoreOS
9 1 (0.24%) Debian
9 1 (0.24%) Mac OS X
Processes & Methodologies
1 161 (37.88%) MapReduce
2 133 (31.29%) Agile Software Development
2 133 (31.29%) TDD
3 108 (25.41%) BDD
4 105 (24.71%) DevOps
5 98 (23.06%) Data Architecture
6 92 (21.65%) Microservices
7 91 (21.41%) Scrum
8 79 (18.59%) Analytics
9 61 (14.35%) Open Source
10 57 (13.41%) ETL
11 49 (11.53%) Data Engineering
12 45 (10.59%) Continuous Integration
13 44 (10.35%) Business Intelligence
14 36 (8.47%) Data Modelling
15 27 (6.35%) Unit Testing
16 22 (5.18%) Multithreading
16 22 (5.18%) Validation
17 21 (4.94%) Benchmarking
17 21 (4.94%) Use Case
Programming Languages
1 297 (69.88%) Java
2 120 (28.24%) Python
3 103 (24.24%) Scala
4 98 (23.06%) SQL
5 50 (11.76%) Shell Script
6 18 (4.24%) PL/SQL
7 14 (3.29%) Bash Shell
8 7 (1.65%) Ruby
9 6 (1.41%) C#
10 5 (1.18%) JavaScript
10 5 (1.18%) R
11 4 (0.94%) Go
11 4 (0.94%) T-SQL
12 2 (0.47%) C++
12 2 (0.47%) COBOL
12 2 (0.47%) Perl
13 1 (0.24%) C
13 1 (0.24%) PHP
Qualifications
1 89 (20.94%) Security Cleared
2 80 (18.82%) SC Cleared
3 32 (7.53%) Degree
4 8 (1.88%) Computer Science Degree
5 6 (1.41%) DBS Check
5 6 (1.41%) Numerate Degree
6 3 (0.71%) ISTQB
7 1 (0.24%) CISA
7 1 (0.24%) CISM
7 1 (0.24%) CISSP
7 1 (0.24%) CTC Cleared
7 1 (0.24%) DV Cleared
7 1 (0.24%) ISEB
7 1 (0.24%) ISTQB Foundation Certification
7 1 (0.24%) Master's Degree
Quality Assurance & Compliance
1 15 (3.53%) QA
2 4 (0.94%) Data Quality
3 2 (0.47%) Dodd-Frank
4 1 (0.24%) Basel III
4 1 (0.24%) EMIR
4 1 (0.24%) FedRAMP
4 1 (0.24%) MiFID
4 1 (0.24%) PCI DSS
4 1 (0.24%) SLA
System Software
1 149 (35.06%) Apache Flume
2 105 (24.71%) Apache ZooKeeper
3 85 (20.00%) Docker
4 10 (2.35%) VMware Infrastructure
5 9 (2.12%) vCloud
6 2 (0.47%) Active Directory
7 1 (0.24%) Oracle RAC
7 1 (0.24%) VirtualBox
Systems Management
1 118 (27.76%) Hadoop YARN
2 113 (26.59%) Oozie
3 86 (20.24%) Ambari
4 75 (17.65%) Kubernetes
5 68 (16.00%) Ansible
6 55 (12.94%) Terraform
7 35 (8.24%) Puppet
8 21 (4.94%) Opscode Chef
9 14 (3.29%) Control-M
10 8 (1.88%) Mesos
11 4 (0.94%) Salt
11 4 (0.94%) ZABBIX
12 2 (0.47%) Grafana
12 2 (0.47%) Graphite
12 2 (0.47%) Kibana
12 2 (0.47%) Nagios
13 1 (0.24%) Apache Slider
13 1 (0.24%) logstash
13 1 (0.24%) Packer
Vendors
1 85 (20.00%) Hortonworks
2 70 (16.47%) Cloudera
3 67 (15.76%) Oracle
4 56 (13.18%) IBM
5 42 (9.88%) Tableau
6 28 (6.59%) Red Hat
7 26 (6.12%) Google
8 21 (4.94%) SAP
9 15 (3.53%) MapR
10 13 (3.06%) Salesforce.com
11 10 (2.35%) VMware
12 9 (2.12%) Teradata
13 7 (1.65%) Splunk
13 7 (1.65%) Talend
14 5 (1.18%) Informatica
15 3 (0.71%) Atlassian
15 3 (0.71%) Palantir
15 3 (0.71%) SaltStack
16 2 (0.47%) Tealium
17 1 (0.24%) New Relic