Job Vacancy Trend Daily Rate Trend Daily Rate Histogram Job Locations Skill Sets
Period
to 21 October 2017

The following table provides summary statistics for contract job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 21 October 2017 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
UK
6 months to
21 Oct 2017		 Same period 2016 Same period 2015
Rank 506 646 754
Rank change year-on-year +140 +108 +62
Contract jobs citing HDFS 403 238 146
As % of all contract IT jobs advertised in the UK 0.41% 0.22% 0.13%
As % of the System Software category 3.32% 1.81% 1.24%
Number of daily rates quoted 288 149 101
UK median daily rate £550 £550 £600
Median daily rate % change year-on-year - -8.33% -22.58%
10th Percentile £376 £408 £413
90th Percentile £702 £733 £675
UK excluding London median daily rate £475 £513 £450
% change year-on-year -7.41% +14.00% -
Number of hourly rates quoted 2 0 2
UK median hourly rate £50.00 - £31.75
UK excluding London median hourly rate £50.00 - £26.00

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for system software skills.

All System Software Skills
UK
Contract vacancies with a requirement for system software skills 12134 13139 11727
As % of all contract IT jobs advertised in the UK 12.22% 12.37% 10.28%
Number of daily rates quoted 7818 8236 6608
UK median daily rate £425 £400 £358
Median daily rate % change year-on-year +6.25% +11.89% +5.93%
10th Percentile £158 £175 £160
90th Percentile £600 £575 £538
UK excluding London median daily rate £375 £360 £338
% change year-on-year +4.17% +6.67% +5.47%
Number of hourly rates quoted 677 900 1273
UK median hourly rate £16.66 £16.00 £16.00
Median hourly rate % change year-on-year +4.13% - +6.67%
10th Percentile £10.40 £10.50 £10.06
90th Percentile £47.25 £40.30 £38.75
UK excluding London median hourly rate £15.50 £15.46 £15.75
% change year-on-year +0.24% -1.83% +5.00%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a percentage of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing HDFS.

Contractor daily rate trend for HDFS in the UK

HDFS
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing HDFS over the 6 months to 21 October 2017.

Contractor daily rate histogram for HDFS in the UK

HDFS
Top 10 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the UK over the 6 months to 21 October 2017. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year		 Matching
Contract
IT Job Ads		 Median
Daily Rate
Past 6 Months		 Median Daily Rate
% Change
on Same Period
Last Year		 Live
Job
Vacancies
England +175 401 £550 - 20
London +112 300 £550 -4.35% 18
UK excluding London +76 102 £475 -7.41% 2
South East +56 75 £401 -19.90% 2
East of England -12 11 £488 +16.07%
North of England +76 9 £475 -6.40%
Yorkshire +33 7 £475 -2.06%
South West +20 6 £500 -16.67%
North West +18 2 £405 -29.10%
Northern Ireland -48 1 - -

For the 6 months to 21 October 2017, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all contract ads with a requirement for HDFS.

1 396 (98.26%) Hadoop
2 352 (87.34%) Big Data
3 333 (82.63%) Apache Spark
4 273 (67.74%) Java
5 253 (62.78%) HBase
6 247 (61.29%) Kafka
7 218 (54.09%) Apache Hive
8 160 (39.70%) MapReduce
9 136 (33.75%) Apache Flume
10 132 (32.75%) Python
11 130 (32.26%) Linux
12 126 (31.27%) Agile Software Development
13 123 (30.52%) TDD
13 123 (30.52%) Hadoop YARN
14 116 (28.78%) Spring
15 110 (27.30%) Apache Solr
16 108 (26.80%) PostgreSQL
17 102 (25.31%) Oozie
18 101 (25.06%) BDD
18 101 (25.06%) Kerberos
19 99 (24.57%) Data Architecture
20 98 (24.32%) Apache ZooKeeper
20 98 (24.32%) DevOps
21 97 (24.07%) Sqoop
21 97 (24.07%) SQL
22 96 (23.82%) REST
23 95 (23.57%) Scala
23 95 (23.57%) Spring Boot
24 92 (22.83%) Microservices
25 88 (21.84%) Scrum

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 333 (82.63%) Apache Spark
2 110 (27.30%) Apache Solr
3 75 (18.61%) Apache
4 54 (13.40%) Apache Pig
5 51 (12.66%) Confluence
6 43 (10.67%) Elasticsearch
7 30 (7.44%) Tomcat
8 12 (2.98%) Cloud Foundry
9 10 (2.48%) GemFire
10 8 (1.99%) JBoss
11 5 (1.24%) SAS
12 3 (0.74%) Jetty
12 3 (0.74%) nginx
12 3 (0.74%) OpenStack
12 3 (0.74%) WebLogic
13 2 (0.50%) Mule
13 2 (0.50%) WebSphere
14 1 (0.25%) CMS
14 1 (0.25%) Oracle Coherence
14 1 (0.25%) Oracle Workflow
Business Applications
1 2 (0.50%) SAP CRM
Cloud Services
1 71 (17.62%) Amazon AWS
2 59 (14.64%) GitHub
3 35 (8.68%) Bluemix
3 35 (8.68%) IBM Cloud
4 12 (2.98%) PaaS
5 8 (1.99%) Amazon EC2
5 8 (1.99%) Amazon S3
5 8 (1.99%) Google Analytics
6 6 (1.49%) Cloud Computing
7 4 (0.99%) Amazon EMR
7 4 (0.99%) Virtual Private Cloud
8 3 (0.74%) Amazon CloudWatch
8 3 (0.74%) Amazon ELB
8 3 (0.74%) AWS CloudFormation
8 3 (0.74%) AWS CloudTrail
8 3 (0.74%) AWS Lambda
8 3 (0.74%) Google Cloud Platform
8 3 (0.74%) Microsoft Azure
8 3 (0.74%) Route 53
8 3 (0.74%) Serverless
Communications & Networking
1 101 (25.06%) Kerberos
2 5 (1.24%) SSL
3 4 (0.99%) Network Security
4 3 (0.74%) LDAP
4 3 (0.74%) OpenLDAP
5 2 (0.50%) HTTP
5 2 (0.50%) tcpdump
5 2 (0.50%) Wireshark
6 1 (0.25%) Internet
Database & Business Intelligence
1 396 (98.26%) Hadoop
2 352 (87.34%) Big Data
3 253 (62.78%) HBase
4 218 (54.09%) Apache Hive
5 108 (26.80%) PostgreSQL
6 97 (24.07%) Sqoop
7 70 (17.37%) Impala
8 54 (13.40%) Apache Cassandra
9 50 (12.41%) NoSQL
10 37 (9.18%) MongoDB
11 29 (7.20%) SQL Server
12 22 (5.46%) Data Warehouse
13 21 (5.21%) MySQL
13 21 (5.21%) Relational Database
14 20 (4.96%) DataStax
15 18 (4.47%) Oracle ODI
16 17 (4.22%) IBM BigInsights
16 17 (4.22%) RDBMS
17 12 (2.98%) Data Lake
17 12 (2.98%) Oracle Database
Development Applications
1 76 (18.86%) Jenkins
2 72 (17.87%) Cucumber
3 59 (14.64%) JIRA
4 46 (11.41%) Git (software)
5 23 (5.71%) IntelliJ
6 16 (3.97%) Gradle
7 12 (2.98%) GitLab
7 12 (2.98%) Team Foundation Server
8 10 (2.48%) JUnit
9 9 (2.23%) Eclipse
10 6 (1.49%) Maven
11 5 (1.24%) Sonatype Nexus
11 5 (1.24%) Subversion
12 4 (0.99%) Talend Open Studio
13 3 (0.74%) Artifactory
13 3 (0.74%) Codeship
13 3 (0.74%) Mercurial
13 3 (0.74%) SonarQube
13 3 (0.74%) TeamCity
13 3 (0.74%) Travis CI
General
1 83 (20.60%) Finance
2 38 (9.43%) Banking
3 24 (5.96%) Telecoms
4 10 (2.48%) Electronics
4 10 (2.48%) Investment Banking
4 10 (2.48%) Manufacturing
4 10 (2.48%) Marketing
5 6 (1.49%) Pharmaceutical
5 6 (1.49%) Retail
6 4 (0.99%) Front Office
6 4 (0.99%) Retail Banking
7 1 (0.25%) Billing
7 1 (0.25%) Law
7 1 (0.25%) Publishing
Job Titles
1 118 (29.28%) Developer
2 115 (28.54%) Data Engineer
3 101 (25.06%) Big Data Engineer
4 76 (18.86%) Big Data Developer
5 70 (17.37%) Hadoop Engineer
6 45 (11.17%) Architect
7 35 (8.68%) Data Architect
7 35 (8.68%) Hadoop Developer
8 33 (8.19%) Java Developer
9 31 (7.69%) Administrator
10 30 (7.44%) Big Data Architect
11 27 (6.70%) Hadoop Administrator
12 24 (5.96%) Java Engineer
13 23 (5.71%) Scala Developer
14 21 (5.21%) Senior Hadoop Engineer
15 20 (4.96%) Senior Data Engineer
16 19 (4.71%) Apache Spark Developer
17 12 (2.98%) Hadoop Architect
17 12 (2.98%) Senior Java Engineer
18 10 (2.48%) DevOps Engineer
Libraries, Frameworks & Software Standards
1 247 (61.29%) Kafka
2 116 (28.78%) Spring
3 96 (23.82%) REST
4 95 (23.57%) Spring Boot
5 70 (17.37%) Drools
6 55 (13.65%) Apache Camel
7 54 (13.40%) PySpark
8 38 (9.43%) Spark Streaming
9 29 (7.20%) RESTful
10 21 (5.21%) J2EE
10 21 (5.21%) Web Services
11 20 (4.96%) JSON
11 20 (4.96%) RabbitMQ
12 18 (4.47%) Middleware
13 16 (3.97%) TinkerPop
14 13 (3.23%) SOAP
14 13 (3.23%) XML
15 11 (2.73%) Akka
15 11 (2.73%) Spring MVC
16 10 (2.48%) Spring Integration
Miscellaneous
1 68 (16.87%) Algorithms
2 47 (11.66%) Analytical Skills
3 22 (5.46%) Greenfield Project
4 14 (3.47%) Replication
5 13 (3.23%) Linux Command Line
5 13 (3.23%) Virtual Team
6 9 (2.23%) Market Risk
7 5 (1.24%) Foreign Exchange (FX)
8 4 (0.99%) Computer Science
8 4 (0.99%) Derivative
9 3 (0.74%) Credit Derivative
9 3 (0.74%) Data Centre
9 3 (0.74%) Equities
9 3 (0.74%) Interest Rate Derivative
9 3 (0.74%) Management Information System
10 2 (0.50%) Credit Risk
10 2 (0.50%) Data Structures
10 2 (0.50%) Distributed Denial-of-Service
10 2 (0.50%) Grid Computing
10 2 (0.50%) Private Cloud
Operating Systems
1 130 (32.26%) Linux
2 74 (18.36%) Unix
3 27 (6.70%) Red Hat Enterprise Linux
4 20 (4.96%) Ubuntu
5 8 (1.99%) Debian
5 8 (1.99%) Windows
6 7 (1.74%) CentOS
7 3 (0.74%) AIX
7 3 (0.74%) Apple iOS
7 3 (0.74%) CoreOS
8 2 (0.50%) Android
9 1 (0.25%) Fedora
9 1 (0.25%) Mac OS X
9 1 (0.25%) Windows Server
9 1 (0.25%) Windows Server 2008
Processes & Methodologies
1 160 (39.70%) MapReduce
2 126 (31.27%) Agile Software Development
3 123 (30.52%) TDD
4 101 (25.06%) BDD
5 99 (24.57%) Data Architecture
6 98 (24.32%) DevOps
7 92 (22.83%) Microservices
8 88 (21.84%) Scrum
9 71 (17.62%) Analytics
10 56 (13.90%) Open Source
11 52 (12.90%) ETL
12 45 (11.17%) Data Engineering
13 43 (10.67%) Continuous Integration
14 35 (8.68%) Business Intelligence
15 32 (7.94%) Data Modelling
16 22 (5.46%) Coaching
17 21 (5.21%) Unit Testing
18 20 (4.96%) Mentoring
19 19 (4.71%) Performance Tuning
19 19 (4.71%) Visualisation
Programming Languages
1 273 (67.74%) Java
2 132 (32.75%) Python
3 97 (24.07%) SQL
4 95 (23.57%) Scala
5 42 (10.42%) Shell Script
6 13 (3.23%) PL/SQL
7 12 (2.98%) Bash Shell
8 8 (1.99%) Ruby
9 6 (1.49%) Go
9 6 (1.49%) JavaScript
10 5 (1.24%) C#
10 5 (1.24%) R
11 4 (0.99%) Perl
11 4 (0.99%) T-SQL
12 2 (0.50%) COBOL
12 2 (0.50%) Julia
13 1 (0.25%) C
13 1 (0.25%) Korn
13 1 (0.25%) PowerShell
Qualifications
1 86 (21.34%) Security Cleared
2 76 (18.86%) SC Cleared
3 18 (4.47%) Degree
4 6 (1.49%) DBS Check
5 4 (0.99%) ISTQB
6 3 (0.74%) Computer Science Degree
6 3 (0.74%) DV Cleared
7 2 (0.50%) CISA
7 2 (0.50%) CISM
7 2 (0.50%) CISSP
7 2 (0.50%) ISTQB Foundation Certification
8 1 (0.25%) CTC Cleared
8 1 (0.25%) ISEB
8 1 (0.25%) Master's Degree
8 1 (0.25%) Numerate Degree
Quality Assurance & Compliance
1 12 (2.98%) QA
2 5 (1.24%) Data Quality
3 2 (0.50%) CDISC
3 2 (0.50%) Dodd-Frank
3 2 (0.50%) FedRAMP
3 2 (0.50%) PCI DSS
3 2 (0.50%) SLA
4 1 (0.25%) Basel III
4 1 (0.25%) EMIR
4 1 (0.25%) MiFID
System Software
1 136 (33.75%) Apache Flume
2 98 (24.32%) Apache ZooKeeper
3 83 (20.60%) Docker
4 8 (1.99%) vCloud
4 8 (1.99%) VMware Infrastructure
5 3 (0.74%) Squid
6 2 (0.50%) Active Directory
6 2 (0.50%) Oracle RAC
Systems Management
1 123 (30.52%) Hadoop YARN
2 102 (25.31%) Oozie
3 78 (19.35%) Ambari
4 72 (17.87%) Kubernetes
5 67 (16.63%) Ansible
6 57 (14.14%) Terraform
7 34 (8.44%) Puppet
8 18 (4.47%) Opscode Chef
9 14 (3.47%) Control-M
10 6 (1.49%) Kibana
10 6 (1.49%) Mesos
10 6 (1.49%) ZABBIX
11 5 (1.24%) logstash
12 4 (0.99%) Packer
13 3 (0.74%) Consul
13 3 (0.74%) Nagios
13 3 (0.74%) Prometheus
13 3 (0.74%) rkt
13 3 (0.74%) Salt
13 3 (0.74%) Sysdig
Vendors
1 77 (19.11%) Cloudera
2 74 (18.36%) Hortonworks
3 62 (15.38%) Oracle
4 54 (13.40%) IBM
5 42 (10.42%) Tableau
6 27 (6.70%) Red Hat
7 21 (5.21%) SAP
8 18 (4.47%) Google
9 14 (3.47%) MapR
10 9 (2.23%) Teradata
11 8 (1.99%) Salesforce.com
11 8 (1.99%) VMware
12 6 (1.49%) Splunk
13 4 (0.99%) Talend
14 3 (0.74%) ForgeRock
14 3 (0.74%) Informatica
15 2 (0.50%) Atlassian
15 2 (0.50%) Palantir
15 2 (0.50%) SaltStack
16 1 (0.25%) Jaspersoft