Period
to 23 November 2017

The following table provides summary statistics for contract job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 23 November 2017 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
UK
6 months to
23 Nov 2017
Same period 2016 Same period 2015
Rank 499 622 727
Rank change year-on-year +123 +105 +116
Contract jobs citing HDFS 429 275 168
As % of all contract IT jobs advertised in the UK 0.42% 0.26% 0.15%
As % of the System Software category 3.30% 2.12% 1.45%
Number of daily rates quoted 309 159 112
UK median daily rate £550 £550 £619
Median daily rate % change year-on-year - -11.11% -20.16%
10th Percentile £390 £408 £414
90th Percentile £700 £725 £675
UK excluding London median daily rate £500 £519 £450
% change year-on-year -3.66% +15.33% -
Number of hourly rates quoted 1 0 1
UK median hourly rate £50.00 - £37.50
UK excluding London median hourly rate £50.00 - -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for system software skills.

All System Software Skills
UK
Contract vacancies with a requirement for system software skills 13000 12984 11597
As % of all contract IT jobs advertised in the UK 12.61% 12.33% 10.20%
Number of daily rates quoted 8435 8137 6604
UK median daily rate £425 £400 £370
Median daily rate % change year-on-year +6.25% +8.11% +9.63%
10th Percentile £156 £170 £158
90th Percentile £613 £575 £550
UK excluding London median daily rate £375 £350 £335
% change year-on-year +7.14% +4.48% +4.69%
Number of hourly rates quoted 728 885 1246
UK median hourly rate £16.50 £16.00 £16.00
Median hourly rate % change year-on-year +3.13% - +6.67%
10th Percentile £10.40 £10.50 £10.07
90th Percentile £47.08 £42.00 £38.75
UK excluding London median hourly rate £15.00 £15.00 £16.00
% change year-on-year - -6.25% +6.58%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a percentage of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing HDFS.

Contractor daily rate trend for HDFS in the UK

HDFS
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing HDFS over the 6 months to 23 November 2017.

Contractor daily rate histogram for HDFS in the UK

HDFS
Top 10 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the UK over the 6 months to 23 November 2017. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +129 426 £550 - 26
London +72 315 £573 -0.43% 15
UK excluding London +75 111 £500 -3.66% 13
South East +47 82 £506 +6.58% 2
East of England -9 12 £488 +16.07%
North of England +57 9 £475 -4.52% 5
Yorkshire +19 7 £475 -3.06% 3
South West -2 7 £425 -29.17% 4
North West +20 2 £405 -29.10% 2
Northern Ireland -38 1 - -

For the 6 months to 23 November 2017, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all contract ads with a requirement for HDFS.

1 418 (97.44%) Hadoop
2 389 (90.68%) Big Data
3 359 (83.68%) Apache Spark
4 300 (69.93%) Java
5 281 (65.50%) HBase
6 279 (65.03%) Kafka
7 233 (54.31%) Apache Hive
8 162 (37.76%) MapReduce
9 151 (35.20%) Apache Flume
10 136 (31.70%) Agile Software Development
11 133 (31.00%) Linux
11 133 (31.00%) TDD
12 127 (29.60%) Spring
13 121 (28.21%) Python
14 118 (27.51%) Hadoop YARN
15 116 (27.04%) PostgreSQL
16 115 (26.81%) Oozie
17 108 (25.17%) BDD
18 107 (24.94%) Apache ZooKeeper
19 105 (24.48%) DevOps
20 104 (24.24%) Sqoop
21 103 (24.01%) Apache Solr
21 103 (24.01%) Kerberos
22 102 (23.78%) Scala
23 100 (23.31%) Data Architecture
23 100 (23.31%) Spring Boot
23 100 (23.31%) SQL
24 99 (23.08%) REST
25 92 (21.45%) Microservices
25 92 (21.45%) Scrum

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 359 (83.68%) Apache Spark
2 103 (24.01%) Apache Solr
3 85 (19.81%) Apache
4 51 (11.89%) Confluence
5 49 (11.42%) Elasticsearch
6 48 (11.19%) Apache Pig
7 28 (6.53%) Tomcat
8 15 (3.50%) Cloud Foundry
8 15 (3.50%) GemFire
9 7 (1.63%) JBoss
10 5 (1.17%) SAS
11 4 (0.93%) OpenStack
12 3 (0.70%) Jetty
12 3 (0.70%) Oracle Coherence
12 3 (0.70%) WebLogic
13 1 (0.23%) Mule
13 1 (0.23%) nginx
13 1 (0.23%) Oracle Workflow
13 1 (0.23%) WebSphere
Business Applications
1 2 (0.47%) SAP CRM
Cloud Services
1 73 (17.02%) Amazon AWS
2 70 (16.32%) GitHub
3 36 (8.39%) Bluemix
3 36 (8.39%) IBM Cloud
4 19 (4.43%) PaaS
5 11 (2.56%) Google Analytics
6 8 (1.86%) Amazon EMR
6 8 (1.86%) Amazon S3
7 6 (1.40%) Google Cloud Platform
8 5 (1.17%) IaaS
9 4 (0.93%) Amazon EC2
9 4 (0.93%) Amazon Kinesis
9 4 (0.93%) Cloud Computing
10 3 (0.70%) AWS CloudFormation
10 3 (0.70%) Microsoft Azure
10 3 (0.70%) Route 53
11 2 (0.47%) Google Tag Manager
12 1 (0.23%) Data Lake Analytics
12 1 (0.23%) Loggly
12 1 (0.23%) Sumo Logic
Communications & Networking
1 103 (24.01%) Kerberos
2 7 (1.63%) Internet
3 5 (1.17%) SSL
4 4 (0.93%) HTTP
5 3 (0.70%) Network Security
6 2 (0.47%) tcpdump
6 2 (0.47%) Wireshark
7 1 (0.23%) DNS
7 1 (0.23%) LDAP
7 1 (0.23%) OpenLDAP
Database & Business Intelligence
1 418 (97.44%) Hadoop
2 389 (90.68%) Big Data
3 281 (65.50%) HBase
4 233 (54.31%) Apache Hive
5 116 (27.04%) PostgreSQL
6 104 (24.24%) Sqoop
7 70 (16.32%) Impala
8 57 (13.29%) NoSQL
9 56 (13.05%) Apache Cassandra
10 40 (9.32%) MongoDB
11 31 (7.23%) SQL Server
12 28 (6.53%) Data Warehouse
13 27 (6.29%) Relational Database
14 21 (4.90%) RDBMS
15 20 (4.66%) Data Lake
16 19 (4.43%) DataStax
17 18 (4.20%) Oracle ODI
18 16 (3.73%) IBM BigInsights
18 16 (3.73%) Oracle Database
19 14 (3.26%) Parquet
Development Applications
1 83 (19.35%) Jenkins
2 75 (17.48%) Cucumber
3 58 (13.52%) JIRA
4 52 (12.12%) Git (software)
5 26 (6.06%) IntelliJ
6 24 (5.59%) Gradle
7 13 (3.03%) JUnit
8 12 (2.80%) GitLab
8 12 (2.80%) Team Foundation Server
9 11 (2.56%) Maven
10 7 (1.63%) Talend Open Studio
11 6 (1.40%) Eclipse
12 5 (1.17%) SonarQube
12 5 (1.17%) Subversion
12 5 (1.17%) TeamCity
13 3 (0.70%) Codeship
13 3 (0.70%) Mercurial
13 3 (0.70%) Travis CI
14 2 (0.47%) Atlassian Bamboo
14 2 (0.47%) Sonatype Nexus
General
1 86 (20.05%) Finance
2 38 (8.86%) Banking
3 26 (6.06%) Telecoms
4 11 (2.56%) Marketing
5 10 (2.33%) Electronics
5 10 (2.33%) Manufacturing
6 7 (1.63%) Investment Banking
7 6 (1.40%) Retail
8 5 (1.17%) Front Office
8 5 (1.17%) Pharmaceutical
8 5 (1.17%) Retail Banking
9 2 (0.47%) Legal
10 1 (0.23%) Games
10 1 (0.23%) Law
10 1 (0.23%) Publishing
Job Titles
1 135 (31.47%) Developer
2 122 (28.44%) Data Engineer
3 106 (24.71%) Big Data Engineer
4 82 (19.11%) Big Data Developer
5 77 (17.95%) Hadoop Engineer
6 54 (12.59%) Architect
7 47 (10.96%) Java Developer
8 43 (10.02%) Data Architect
9 39 (9.09%) Hadoop Developer
10 37 (8.62%) Big Data Architect
11 25 (5.83%) Java Engineer
12 24 (5.59%) Administrator
12 24 (5.59%) Scala Developer
13 23 (5.36%) Senior Hadoop Engineer
14 20 (4.66%) Apache Spark Developer
14 20 (4.66%) Hadoop Administrator
14 20 (4.66%) Senior Data Engineer
15 15 (3.50%) Hadoop Architect
16 14 (3.26%) Senior Developer
16 14 (3.26%) Senior Java Engineer
Libraries, Frameworks & Software Standards
1 279 (65.03%) Kafka
2 127 (29.60%) Spring
3 100 (23.31%) Spring Boot
4 99 (23.08%) REST
5 70 (16.32%) Drools
6 60 (13.99%) PySpark
7 58 (13.52%) Apache Camel
8 46 (10.72%) RESTful
9 42 (9.79%) Spark Streaming
10 32 (7.46%) Web Services
11 30 (6.99%) RabbitMQ
12 29 (6.76%) J2EE
12 29 (6.76%) JSON
13 25 (5.83%) Middleware
14 24 (5.59%) Apache Avro
15 17 (3.96%) SOAP
16 16 (3.73%) TinkerPop
17 14 (3.26%) Akka
17 14 (3.26%) XML
18 13 (3.03%) Spring Integration
Miscellaneous
1 67 (15.62%) Algorithms
2 52 (12.12%) Analytical Skills
3 27 (6.29%) Greenfield Project
4 19 (4.43%) Linux Command Line
4 19 (4.43%) Virtual Team
5 18 (4.20%) Replication
6 11 (2.56%) Computer Science
6 11 (2.56%) Market Risk
7 7 (1.63%) Derivative
8 5 (1.17%) Interest Rate Derivative
9 4 (0.93%) Credit Risk
9 4 (0.93%) Foreign Exchange (FX)
10 3 (0.70%) Credit Derivative
10 3 (0.70%) Data Centre
10 3 (0.70%) Equities
11 2 (0.47%) Data Structures
11 2 (0.47%) Management Information System
12 1 (0.23%) Grid Computing
12 1 (0.23%) Media Streaming
12 1 (0.23%) PDA
Operating Systems
1 133 (31.00%) Linux
2 73 (17.02%) Unix
3 21 (4.90%) Red Hat Enterprise Linux
4 15 (3.50%) Ubuntu
5 6 (1.40%) CentOS
6 5 (1.17%) Windows
7 3 (0.70%) AIX
8 2 (0.47%) Android
8 2 (0.47%) Apple iOS
8 2 (0.47%) Debian
9 1 (0.23%) CoreOS
9 1 (0.23%) Mac OS X
Processes & Methodologies
1 162 (37.76%) MapReduce
2 136 (31.70%) Agile Software Development
3 133 (31.00%) TDD
4 108 (25.17%) BDD
5 105 (24.48%) DevOps
6 100 (23.31%) Data Architecture
7 92 (21.45%) Microservices
7 92 (21.45%) Scrum
8 79 (18.41%) Analytics
9 61 (14.22%) Open Source
10 56 (13.05%) ETL
11 49 (11.42%) Data Engineering
12 45 (10.49%) Business Intelligence
12 45 (10.49%) Continuous Integration
13 36 (8.39%) Data Modelling
14 27 (6.29%) Unit Testing
15 22 (5.13%) Multithreading
15 22 (5.13%) Validation
16 21 (4.90%) Benchmarking
16 21 (4.90%) Use Case
Programming Languages
1 300 (69.93%) Java
2 121 (28.21%) Python
3 102 (23.78%) Scala
4 100 (23.31%) SQL
5 50 (11.66%) Shell Script
6 18 (4.20%) PL/SQL
7 14 (3.26%) Bash Shell
8 7 (1.63%) Ruby
9 6 (1.40%) C#
10 5 (1.17%) JavaScript
10 5 (1.17%) R
11 4 (0.93%) Go
11 4 (0.93%) T-SQL
12 2 (0.47%) C++
12 2 (0.47%) COBOL
12 2 (0.47%) Perl
13 1 (0.23%) C
13 1 (0.23%) PHP
Qualifications
1 91 (21.21%) Security Cleared
2 81 (18.88%) SC Cleared
3 32 (7.46%) Degree
4 8 (1.86%) Computer Science Degree
5 6 (1.40%) DBS Check
5 6 (1.40%) Numerate Degree
6 3 (0.70%) ISTQB
7 1 (0.23%) CISA
7 1 (0.23%) CISM
7 1 (0.23%) CISSP
7 1 (0.23%) CTC Cleared
7 1 (0.23%) DV Cleared
7 1 (0.23%) ISEB
7 1 (0.23%) ISTQB Foundation Certification
7 1 (0.23%) Master's Degree
Quality Assurance & Compliance
1 15 (3.50%) QA
2 4 (0.93%) Data Quality
3 2 (0.47%) Dodd-Frank
4 1 (0.23%) Basel III
4 1 (0.23%) CDISC
4 1 (0.23%) EMIR
4 1 (0.23%) FedRAMP
4 1 (0.23%) MiFID
4 1 (0.23%) PCI DSS
4 1 (0.23%) SLA
System Software
1 151 (35.20%) Apache Flume
2 107 (24.94%) Apache ZooKeeper
3 85 (19.81%) Docker
4 10 (2.33%) VMware Infrastructure
5 9 (2.10%) vCloud
6 2 (0.47%) Active Directory
7 1 (0.23%) Oracle RAC
7 1 (0.23%) VirtualBox
Systems Management
1 118 (27.51%) Hadoop YARN
2 115 (26.81%) Oozie
3 86 (20.05%) Ambari
4 75 (17.48%) Kubernetes
5 68 (15.85%) Ansible
6 55 (12.82%) Terraform
7 35 (8.16%) Puppet
8 21 (4.90%) Opscode Chef
9 14 (3.26%) Control-M
10 8 (1.86%) Mesos
11 4 (0.93%) Salt
11 4 (0.93%) ZABBIX
12 2 (0.47%) Grafana
12 2 (0.47%) Graphite
12 2 (0.47%) Kibana
12 2 (0.47%) Nagios
13 1 (0.23%) Apache Slider
13 1 (0.23%) logstash
13 1 (0.23%) Packer
Vendors
1 85 (19.81%) Hortonworks
2 72 (16.78%) Cloudera
3 68 (15.85%) Oracle
4 56 (13.05%) IBM
5 42 (9.79%) Tableau
6 28 (6.53%) Red Hat
7 26 (6.06%) Google
8 21 (4.90%) SAP
9 15 (3.50%) MapR
10 13 (3.03%) Salesforce.com
11 10 (2.33%) VMware
12 9 (2.10%) Teradata
13 7 (1.63%) Splunk
13 7 (1.63%) Talend
14 5 (1.17%) Informatica
15 3 (0.70%) Atlassian
15 3 (0.70%) Palantir
15 3 (0.70%) SaltStack
16 2 (0.47%) Tealium
17 1 (0.23%) Jaspersoft