Period
to 25 February 2018

The following table provides summary statistics for contract job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 25 February 2018 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
UK
6 months to
25 Feb 2018
Same period 2017 Same period 2016
Rank 516 553 736
Rank change year-on-year +37 +183 +41
Contract jobs citing HDFS 375 313 146
As % of all contract IT jobs advertised in the UK 0.38% 0.33% 0.14%
As % of the System Software category 2.95% 2.71% 1.34%
Number of daily rates quoted 262 187 93
UK median daily rate £575 £575 £600
Median daily rate % change year-on-year - -4.17% +2.13%
10th Percentile £425 £405 £375
90th Percentile £675 £713 £675
UK excluding London median daily rate £575 £475 £450
% change year-on-year +21.05% +5.56% -14.29%
Number of hourly rates quoted 0 0 1
UK median hourly rate - - £37.50
UK excluding London median hourly rate - - -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for system software skills.

All System Software Skills
UK
Contract vacancies with a requirement for system software skills 12,697 11,552 10,915
As % of all contract IT jobs advertised in the UK 12.70% 12.14% 10.11%
Number of daily rates quoted 8,296 7,224 6,412
UK median daily rate £450 £405 £376
Median daily rate % change year-on-year +11.11% +7.64% +7.50%
10th Percentile £163 £163 £158
90th Percentile £625 £588 £550
UK excluding London median daily rate £388 £350 £345
% change year-on-year +10.71% +1.45% +2.60%
Number of hourly rates quoted 692 832 1,102
UK median hourly rate £16.00 £16.00 £15.00
Median hourly rate % change year-on-year - +6.67% -3.23%
10th Percentile £10.50 £10.29 £10.00
90th Percentile £46.23 £44.48 £40.25
UK excluding London median hourly rate £15.00 £15.00 £15.25
% change year-on-year - -1.64% +1.67%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a percentage of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing HDFS.

Contractor daily rate trend for HDFS in the UK

HDFS
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing HDFS over the 6 months to 25 February 2018.

Contractor daily rate histogram for HDFS in the UK

HDFS
Top 12 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the UK over the 6 months to 25 February 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +37 370 £575 - 48
London -23 236 £575 - 33
UK excluding London +56 135 £575 +21.05% 19
South East +55 111 £575 +12.09% 9
South West +18 6 £488 +21.88% 1
Midlands +16 6 £600 -
North of England +15 6 £588 +27.03% 4
East Midlands +4 6 £600 -
East of England -18 4 £630 +57.50% 1
North West +17 3 £575 +15.58% 2
Yorkshire -18 3 £600 +29.73% 2
Scotland -6 2 £488 +62.50% 4

For the 6 months to 25 February 2018, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for HDFS.

1 360 (96.00%) Hadoop
2 314 (83.73%) Big Data
3 284 (75.73%) Apache Spark
4 230 (61.33%) HBase
5 227 (60.53%) Java
6 219 (58.40%) Kafka
7 201 (53.60%) Apache Hive
8 168 (44.80%) Linux
9 147 (39.20%) Apache Flume
10 128 (34.13%) Hadoop YARN
11 126 (33.60%) Oozie
12 122 (32.53%) Agile Software Development
13 114 (30.40%) Python
14 108 (28.80%) Sqoop
15 105 (28.00%) Hortonworks
16 100 (26.67%) Cloudera
17 98 (26.13%) MapReduce
18 97 (25.87%) Analytics
18 97 (25.87%) SQL
19 92 (24.53%) Scala
20 85 (22.67%) Apache ZooKeeper
21 82 (21.87%) DevOps
22 80 (21.33%) Jenkins
22 80 (21.33%) Shell Script
22 80 (21.33%) PostgreSQL
23 79 (21.07%) Spring
24 78 (20.80%) TDD
25 76 (20.27%) Apache
26 67 (17.87%) Finance
26 67 (17.87%) Git (software)

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 284 (75.73%) Apache Spark
2 76 (20.27%) Apache
3 49 (13.07%) Apache Solr
4 48 (12.80%) Apache Pig
5 24 (6.40%) Elasticsearch
6 16 (4.27%) Tomcat
7 15 (4.00%) Cloud Foundry
8 11 (2.93%) Confluence
9 9 (2.40%) GemFire
10 8 (2.13%) JBoss
10 8 (2.13%) nginx
11 4 (1.07%) SAS
12 3 (0.80%) Oracle Coherence
13 1 (0.27%) Apache Airflow
13 1 (0.27%) OpenStack
Business Applications
1 2 (0.53%) SAP CRM
2 1 (0.27%) Summit
Cloud Services
1 66 (17.60%) Amazon AWS
2 62 (16.53%) GitHub
3 21 (5.60%) Bluemix
3 21 (5.60%) Google Tag Manager
3 21 (5.60%) IBM Cloud
4 14 (3.73%) Microsoft Azure
5 13 (3.47%) PaaS
6 11 (2.93%) Amazon S3
7 10 (2.67%) AWS Lambda
7 10 (2.67%) Serverless
8 8 (2.13%) Amazon EMR
9 7 (1.87%) IaaS
10 5 (1.33%) Amazon CloudWatch
10 5 (1.33%) Amazon EC2
10 5 (1.33%) Amazon ELB
10 5 (1.33%) AWS CloudTrail
10 5 (1.33%) Virtual Private Cloud
11 4 (1.07%) Amazon Kinesis
11 4 (1.07%) Google Cloud Platform
12 3 (0.80%) Google Analytics
Communications & Networking
1 63 (16.80%) Kerberos
2 19 (5.07%) SSL
3 14 (3.73%) Internet
4 6 (1.60%) OpenLDAP
5 5 (1.33%) HTTP
6 3 (0.80%) Network Security
7 2 (0.53%) Firewall
8 1 (0.27%) DNS
8 1 (0.27%) FTP
8 1 (0.27%) Multicast
Database & Business Intelligence
1 360 (96.00%) Hadoop
2 314 (83.73%) Big Data
3 230 (61.33%) HBase
4 201 (53.60%) Apache Hive
5 108 (28.80%) Sqoop
6 80 (21.33%) PostgreSQL
7 61 (16.27%) Apache Cassandra
8 57 (15.20%) Impala
9 56 (14.93%) NoSQL
10 43 (11.47%) Data Warehouse
11 42 (11.20%) Data Lake
12 41 (10.93%) MongoDB
13 40 (10.67%) SQL Server
14 34 (9.07%) Parquet
15 31 (8.27%) Relational Database
16 26 (6.93%) RDBMS
17 16 (4.27%) Oracle Database
18 14 (3.73%) SAP BW
18 14 (3.73%) SAP Data Services
19 11 (2.93%) Amazon Redshift
Development Applications
1 80 (21.33%) Jenkins
2 67 (17.87%) Git (software)
3 38 (10.13%) Gradle
4 23 (6.13%) JUnit
5 19 (5.07%) Cucumber
6 18 (4.80%) JIRA
7 13 (3.47%) TeamCity
8 12 (3.20%) Maven
9 10 (2.67%) IntelliJ
10 7 (1.87%) Sonatype Nexus
11 5 (1.33%) Artifactory
11 5 (1.33%) SonarQube
12 4 (1.07%) ERwin
12 4 (1.07%) GitLab
12 4 (1.07%) VersionOne
13 2 (0.53%) JBehave
13 2 (0.53%) Subversion
14 1 (0.27%) Atlassian Bamboo
14 1 (0.27%) git-flow
14 1 (0.27%) Vagrant
General
1 67 (17.87%) Finance
2 26 (6.93%) Banking
3 12 (3.20%) Retail
4 9 (2.40%) Telecoms
5 7 (1.87%) Front Office
5 7 (1.87%) Retail Banking
6 6 (1.60%) Investment Banking
7 3 (0.80%) French Language
7 3 (0.80%) Marketing
8 2 (0.53%) Games
8 2 (0.53%) Legal
8 2 (0.53%) Pharmaceutical
9 1 (0.27%) Back Office
Job Titles
1 112 (29.87%) Developer
2 79 (21.07%) Data Engineer
3 67 (17.87%) Big Data Developer
4 62 (16.53%) Architect
5 57 (15.20%) Java Developer
6 55 (14.67%) Big Data Engineer
7 47 (12.53%) Data Architect
8 41 (10.93%) Hadoop Engineer
9 37 (9.87%) Big Data Architect
10 25 (6.67%) Consultant
11 24 (6.40%) Hadoop Developer
12 20 (5.33%) DevOps Engineer
13 16 (4.27%) Administrator
13 16 (4.27%) Senior Developer
14 15 (4.00%) Hadoop Architect
14 15 (4.00%) Hadoop Consultant
15 14 (3.73%) Senior Java Developer
16 13 (3.47%) Java Engineer
17 12 (3.20%) Hadoop Administrator
18 10 (2.67%) Cloud Engineer
Libraries, Frameworks & Software Standards
1 219 (58.40%) Kafka
2 79 (21.07%) Spring
3 59 (15.73%) RESTful
4 50 (13.33%) REST
5 48 (12.80%) JSON
5 48 (12.80%) RabbitMQ
6 47 (12.53%) Web Services
7 45 (12.00%) Apache Avro
8 41 (10.93%) PySpark
9 38 (10.13%) Middleware
10 35 (9.33%) J2EE
11 32 (8.53%) Spring Boot
12 28 (7.47%) SOAP
13 23 (6.13%) Spark Streaming
14 20 (5.33%) Drools
15 19 (5.07%) XML
16 16 (4.27%) JMS
17 13 (3.47%) Hibernate
18 12 (3.20%) ActiveMQ
18 12 (3.20%) Apache Camel
Miscellaneous
1 45 (12.00%) Greenfield Project
2 34 (9.07%) Virtual Team
3 33 (8.80%) Linux Command Line
4 28 (7.47%) Algorithms
5 21 (5.60%) Computer Science
6 20 (5.33%) Analytical Skills
7 9 (2.40%) Derivative
7 9 (2.40%) Replication
8 6 (1.60%) Market Risk
9 5 (1.33%) Interest Rate Derivative
10 3 (0.80%) Credit Risk
10 3 (0.80%) Data Centre
11 2 (0.53%) Enterprise Software
11 2 (0.53%) Foreign Exchange (FX)
11 2 (0.53%) Life Science
11 2 (0.53%) Management Information System
11 2 (0.53%) Media Streaming
11 2 (0.53%) Mobile App
11 2 (0.53%) MPEG
11 2 (0.53%) Over-the-Top Content
Operating Systems
1 168 (44.80%) Linux
2 66 (17.60%) Unix
3 36 (9.60%) Red Hat Enterprise Linux
4 30 (8.00%) CentOS
5 7 (1.87%) Windows
6 6 (1.60%) CoreOS
7 4 (1.07%) Ubuntu
8 3 (0.80%) DC/OS
8 3 (0.80%) Debian
9 1 (0.27%) Windows Server
9 1 (0.27%) Windows Server 2008
Processes & Methodologies
1 122 (32.53%) Agile Software Development
2 98 (26.13%) MapReduce
3 97 (25.87%) Analytics
4 82 (21.87%) DevOps
5 78 (20.80%) TDD
6 64 (17.07%) ETL
7 62 (16.53%) Continuous Integration
8 50 (13.33%) BDD
9 49 (13.07%) Data Engineering
10 44 (11.73%) Scrum
11 43 (11.47%) Open Source
12 41 (10.93%) Data Architecture
13 40 (10.67%) Data Science
14 35 (9.33%) Benchmarking
15 33 (8.80%) Business Intelligence
15 33 (8.80%) Data Strategy
16 29 (7.73%) Data Analytics
17 28 (7.47%) Microservices
18 25 (6.67%) Machine Learning
18 25 (6.67%) Unit Testing
Programming Languages
1 227 (60.53%) Java
2 114 (30.40%) Python
3 97 (25.87%) SQL
4 92 (24.53%) Scala
5 80 (21.33%) Shell Script
6 45 (12.00%) Bash Shell
7 22 (5.87%) PL/SQL
8 15 (4.00%) R
9 8 (2.13%) Ruby
10 7 (1.87%) C#
11 6 (1.60%) Go
11 6 (1.60%) Perl
12 5 (1.33%) C++
13 4 (1.07%) Gremlin
14 2 (0.53%) C
14 2 (0.53%) JavaScript
14 2 (0.53%) T-SQL
14 2 (0.53%) XPath
15 1 (0.27%) PHP
15 1 (0.27%) TypeScript
Qualifications
1 54 (14.40%) Security Cleared
2 51 (13.60%) Degree
3 48 (12.80%) SC Cleared
4 16 (4.27%) Computer Science Degree
5 12 (3.20%) Numerate Degree
6 3 (0.80%) DV Cleared
7 2 (0.53%) DBS Check
8 1 (0.27%) ISTQB
8 1 (0.27%) Master's Degree
8 1 (0.27%) PRINCE2 Certification
Quality Assurance & Compliance
1 17 (4.53%) QA
2 5 (1.33%) Data Quality
2 5 (1.33%) MiFID
3 4 (1.07%) ISO/IEC 20000
3 4 (1.07%) Sarbanes-Oxley
4 3 (0.80%) Dodd-Frank
4 3 (0.80%) EMIR
System Software
1 147 (39.20%) Apache Flume
2 85 (22.67%) Apache ZooKeeper
3 41 (10.93%) Docker
4 17 (4.53%) Active Directory
5 10 (2.67%) VMware Infrastructure
6 9 (2.40%) vCloud
7 5 (1.33%) Squid
8 1 (0.27%) etcd
8 1 (0.27%) Virtual Machines
8 1 (0.27%) VirtualBox
Systems Management
1 128 (34.13%) Hadoop YARN
2 126 (33.60%) Oozie
3 60 (16.00%) Ambari
4 35 (9.33%) Kubernetes
5 28 (7.47%) Puppet
6 27 (7.20%) Ansible
7 16 (4.27%) Terraform
8 14 (3.73%) Apache Slider
9 12 (3.20%) Mesos
10 11 (2.93%) Opscode Chef
11 8 (2.13%) Kibana
12 7 (1.87%) Nagios
13 6 (1.60%) Consul
13 6 (1.60%) Prometheus
13 6 (1.60%) ZABBIX
14 5 (1.33%) logstash
14 5 (1.33%) Packer
14 5 (1.33%) rkt
14 5 (1.33%) Sysdig
15 3 (0.80%) Salt
Vendors
1 105 (28.00%) Hortonworks
2 100 (26.67%) Cloudera
3 45 (12.00%) Oracle
4 41 (10.93%) MapR
5 39 (10.40%) Tableau
6 28 (7.47%) Google
7 26 (6.93%) Salesforce.com
7 26 (6.93%) Talend
8 25 (6.67%) IBM
9 21 (5.60%) Tealium
10 17 (4.53%) SAP
11 15 (4.00%) Red Hat
12 10 (2.67%) VMware
13 8 (2.13%) Microsoft
14 6 (1.60%) Atlassian
14 6 (1.60%) Informatica
15 5 (1.33%) ForgeRock
15 5 (1.33%) Teradata
16 4 (1.07%) Ab Initio
16 4 (1.07%) SaltStack