Job Vacancy Trend Daily Rate Trend Daily Rate Histogram Hourly Rate Trend Job Locations Skill Sets
Period
to 19 September 2018

The following table provides summary statistics for contract job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 19 September 2018 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
UK
6 months to
19 Sep 2018		 Same period 2017 Same period 2016
Rank 566 518 690
Rank change year-on-year -48 +172 +85
Contract jobs citing HDFS 333 383 217
As % of all contract IT jobs advertised in the UK 0.32% 0.39% 0.20%
As % of the System Software category 2.21% 3.24% 1.65%
Number of daily rates quoted 237 269 153
UK median daily rate £525 £540 £550
Median daily rate % change year-on-year -2.78% -1.82% -8.33%
10th Percentile £425 £376 £400
90th Percentile £675 £700 £710
UK excluding London median daily rate £500 £401 £506
% change year-on-year +24.84% -20.89% +12.50%
Number of hourly rates quoted 3 2 0
UK median hourly rate £22.00 £50.00 -
Median hourly rate % change year-on-year -56.00% - -
UK excluding London median hourly rate £22.25 £50.00 -
% change year-on-year -55.50% - -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for system software skills.

All System Software Skills
UK
Contract vacancies with a requirement for system software skills 15,036 11,813 13,130
As % of all contract IT jobs advertised in the UK 14.59% 11.96% 12.27%
Number of daily rates quoted 9,730 7,593 8,238
UK median daily rate £450 £425 £400
Median daily rate % change year-on-year +5.88% +6.25% +14.29%
10th Percentile £180 £160 £177
90th Percentile £625 £588 £575
UK excluding London median daily rate £400 £375 £365
% change year-on-year +6.67% +2.74% +8.15%
Number of hourly rates quoted 904 672 874
UK median hourly rate £16.13 £16.68 £16.50
Median hourly rate % change year-on-year -3.28% +1.09% +3.13%
10th Percentile £10.75 £10.40 £10.50
90th Percentile £43.75 £47.00 £41.00
UK excluding London median hourly rate £15.86 £15.50 £16.38
% change year-on-year +2.32% -5.34% +5.65%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a percentage of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing HDFS.

Contractor daily rate trend for HDFS in the UK

HDFS
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing HDFS over the 6 months to 19 September 2018.

Contractor daily rate histogram for HDFS in the UK

HDFS
Contractor Hourly Rate Trend

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing HDFS.

Contractor hourly rate trend for HDFS in the UK

HDFS
Top 13 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the UK over the 6 months to 19 September 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year		 Matching
Contract
IT Job Ads		 Median
Daily Rate
Past 6 Months		 Median Daily Rate
% Change
on Same Period
Last Year		 Live
Job
Vacancies
England -57 326 £525 -3.67% 34
London -27 222 £525 -4.55% 21
UK excluding London -20 107 £500 +24.84% 14
South East -10 81 £500 +24.84% 9
North of England -12 8 £500 +5.26%
East of England -16 7 £588 +20.51% 2
Scotland -17 7 £500 +5.26% 1
North West -15 5 £500 +23.46%
Yorkshire -11 3 £425 -10.53%
Midlands - 3 £425 - 1
South West -9 2 £550 +37.50% 1
East Midlands - 2 - - 1
West Midlands - 1 £425 -

For the 6 months to 19 September 2018, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for HDFS.

1 302 (90.69%) Hadoop
2 267 (80.18%) Big Data
3 232 (69.67%) Apache Spark
4 212 (63.66%) Apache Hive
5 190 (57.06%) Java
6 178 (53.45%) HBase
7 177 (53.15%) Agile Software Development
8 171 (51.35%) Kafka
9 137 (41.14%) Python
10 120 (36.04%) SQL
11 118 (35.44%) Unix
12 116 (34.83%) Linux
13 113 (33.93%) Sqoop
13 113 (33.93%) Hadoop YARN
14 107 (32.13%) Amazon AWS
15 103 (30.93%) Scrum
16 99 (29.73%) Jenkins
17 96 (28.83%) Finance
18 93 (27.93%) Impala
19 92 (27.63%) ETL
20 89 (26.73%) Cloudera
21 87 (26.13%) Scala
22 86 (25.83%) Continuous Integration
23 78 (23.42%) Shell Script
23 78 (23.42%) Test Automation
24 77 (23.12%) NoSQL
24 77 (23.12%) SQL Server
25 76 (22.82%) Ansible
26 74 (22.22%) Oozie
26 74 (22.22%) Data Analysis

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 232 (69.67%) Apache Spark
2 57 (17.12%) Apache
3 51 (15.32%) Apache Solr
4 38 (11.41%) Apache Pig
5 23 (6.91%) Confluence
5 23 (6.91%) Elasticsearch
5 23 (6.91%) GemFire
6 22 (6.61%) Cloud Foundry
7 7 (2.10%) Tomcat
8 6 (1.80%) SAS
9 4 (1.20%) WildFly
10 2 (0.60%) Apache Airflow
10 2 (0.60%) nginx
11 1 (0.30%) JBoss
11 1 (0.30%) WebLogic
Applications
1 46 (13.81%) Microsoft Excel
2 44 (13.21%) MS Visio
3 2 (0.60%) Spreadsheet
Business Applications
1 1 (0.30%) Summit
Cloud Services
1 107 (32.13%) Amazon AWS
2 61 (18.32%) Amazon S3
3 45 (13.51%) Amazon EC2
4 44 (13.21%) Virtual Private Cloud
5 30 (9.01%) GitHub
6 29 (8.71%) Microsoft Azure
6 29 (8.71%) PaaS
7 12 (3.60%) Amazon EMR
7 12 (3.60%) Cloud Computing
8 9 (2.70%) Google Cloud Platform
9 8 (2.40%) Amazon Kinesis
10 6 (1.80%) IaaS
11 5 (1.50%) Serverless
12 4 (1.20%) AWS Lambda
12 4 (1.20%) IBM Cloud
13 3 (0.90%) AWS Elastic Beanstalk
14 1 (0.30%) Azure Data Factory
14 1 (0.30%) npm
14 1 (0.30%) SaaS
14 1 (0.30%) Yarn
Communications & Networking
1 48 (14.41%) Kerberos
2 22 (6.61%) SSL
3 19 (5.71%) HTTP
4 13 (3.90%) VPN
5 8 (2.40%) Firewall
6 7 (2.10%) HTTPS
6 7 (2.10%) RTMP
7 4 (1.20%) FTP
8 3 (0.90%) Network Security
9 1 (0.30%) Internet
9 1 (0.30%) SSH
Database & Business Intelligence
1 302 (90.69%) Hadoop
2 267 (80.18%) Big Data
3 212 (63.66%) Apache Hive
4 178 (53.45%) HBase
5 113 (33.93%) Sqoop
6 93 (27.93%) Impala
7 77 (23.12%) NoSQL
7 77 (23.12%) SQL Server
8 48 (14.41%) MongoDB
9 44 (13.21%) Apache Cassandra
10 41 (12.31%) Relational Database
11 35 (10.51%) Data Warehouse
12 34 (10.21%) Data Lake
13 27 (8.11%) Parquet
13 27 (8.11%) PostgreSQL
14 15 (4.50%) Amazon Redshift
14 15 (4.50%) MySQL
15 14 (4.20%) OLTP
16 11 (3.30%) Power BI
16 11 (3.30%) RDBMS
Development Applications
1 99 (29.73%) Jenkins
2 57 (17.12%) Git (software)
3 38 (11.41%) Cucumber
4 28 (8.41%) IntelliJ
5 27 (8.11%) JIRA
6 22 (6.61%) JUnit
6 22 (6.61%) Maven
7 19 (5.71%) Eclipse
8 10 (3.00%) Gradle
9 8 (2.40%) FishEye
9 8 (2.40%) JBehave
10 7 (2.10%) Selenium
11 6 (1.80%) Sonatype Nexus
11 6 (1.80%) Subversion
12 5 (1.50%) Ant
12 5 (1.50%) TeamCity
13 4 (1.20%) CruiseControl
13 4 (1.20%) GitLab
13 4 (1.20%) Jasmine
13 4 (1.20%) Visual Studio Team System
General
1 96 (28.83%) Finance
2 31 (9.31%) Banking
3 16 (4.80%) Investment Banking
4 13 (3.90%) Legal
5 12 (3.60%) Games
6 9 (2.70%) Front Office
7 6 (1.80%) Retail
8 5 (1.50%) Marketing
9 3 (0.90%) Back Office
10 2 (0.60%) Telecoms
11 1 (0.30%) Aerospace
11 1 (0.30%) French Language
11 1 (0.30%) Manufacturing
11 1 (0.30%) Private Banking
11 1 (0.30%) Retail Banking
Job Titles
1 129 (38.74%) Developer
2 61 (18.32%) Big Data Developer
3 60 (18.02%) Data Engineer
4 50 (15.02%) Big Data Engineer
5 48 (14.41%) Java Developer
6 44 (13.21%) Platform Engineer
7 32 (9.61%) Hadoop Developer
8 29 (8.71%) Architect
9 26 (7.81%) Apache Spark Developer
10 23 (6.91%) Big Data Architect
10 23 (6.91%) Data Architect
11 22 (6.61%) Hadoop Engineer
12 16 (4.80%) Consultant
13 14 (4.20%) Senior Developer
14 13 (3.90%) DevOps Engineer
14 13 (3.90%) Java Engineer
14 13 (3.90%) Java Hadoop Developer
15 12 (3.60%) AWS Engineer
16 11 (3.30%) Senior Data Engineer
17 10 (3.00%) Hadoop Administrator
Libraries, Frameworks & Software Standards
1 171 (51.35%) Kafka
2 49 (14.71%) Spring
3 48 (14.41%) RabbitMQ
4 44 (13.21%) JSON
4 44 (13.21%) Spark Streaming
5 34 (10.21%) RESTful
6 32 (9.61%) J2EE
7 26 (7.81%) Middleware
8 25 (7.51%) REST
9 24 (7.21%) XML
10 16 (4.80%) Apache Avro
10 16 (4.80%) Spring Boot
11 14 (4.20%) HTML
12 13 (3.90%) Node.js
13 11 (3.30%) Gherkin
13 11 (3.30%) Ruby on Rails
14 10 (3.00%) AngularJS
14 10 (3.00%) Drools
14 10 (3.00%) Spring Cloud
15 9 (2.70%) Play Framework
Miscellaneous
1 47 (14.11%) Wiki
2 46 (13.81%) Greenfield Project
3 35 (10.51%) Analytical Skills
4 34 (10.21%) Algorithms
5 22 (6.61%) Virtual Team
6 19 (5.71%) Data Structures
7 17 (5.11%) Linux Command Line
8 16 (4.80%) Computer Science
9 10 (3.00%) Fintech
9 10 (3.00%) Self-Motivation
10 9 (2.70%) User Experience
11 8 (2.40%) Data Centre
12 7 (2.10%) Derivative
12 7 (2.10%) Equity Derivative
13 3 (0.90%) Management Information System
13 3 (0.90%) Replication
14 2 (0.60%) Clustering
14 2 (0.60%) Cyberthreat
14 2 (0.60%) Grid Computing
14 2 (0.60%) Private Cloud
Operating Systems
1 118 (35.44%) Unix
2 116 (34.83%) Linux
3 50 (15.02%) Windows
4 10 (3.00%) Red Hat Enterprise Linux
5 4 (1.20%) Ubuntu
6 3 (0.90%) DC/OS
7 2 (0.60%) SUSE Linux Enterprise Server
7 2 (0.60%) Windows Server
7 2 (0.60%) Windows Server 2008
8 1 (0.30%) CentOS
Processes & Methodologies
1 177 (53.15%) Agile Software Development
2 103 (30.93%) Scrum
3 92 (27.63%) ETL
4 86 (25.83%) Continuous Integration
5 78 (23.42%) Test Automation
6 74 (22.22%) Data Analysis
7 71 (21.32%) DevOps
7 71 (21.32%) TDD
8 64 (19.22%) Analytics
9 60 (18.02%) BDD
10 50 (15.02%) Business Intelligence
11 49 (14.71%) Data Extraction
12 48 (14.41%) Build Automation
13 45 (13.51%) MapReduce
14 39 (11.71%) Data Modelling
14 39 (11.71%) Data Science
15 36 (10.81%) Problem-Solving
16 35 (10.51%) Microservices
17 29 (8.71%) Open Source
17 29 (8.71%) Software Engineering
Programming Languages
1 190 (57.06%) Java
2 137 (41.14%) Python
3 120 (36.04%) SQL
4 87 (26.13%) Scala
5 78 (23.42%) Shell Script
6 62 (18.62%) Bash Shell
7 44 (13.21%) Korn
7 44 (13.21%) PowerShell
8 29 (8.71%) Java 8
9 18 (5.41%) JavaScript
10 15 (4.50%) Ruby
11 13 (3.90%) R
11 13 (3.90%) T-SQL
12 11 (3.30%) Perl
13 7 (2.10%) C#
13 7 (2.10%) C++
13 7 (2.10%) Groovy
13 7 (2.10%) TypeScript
14 5 (1.50%) PL/SQL
15 3 (0.90%) Go
Qualifications
1 22 (6.61%) Degree
2 17 (5.11%) Security Cleared
3 14 (4.20%) SC Cleared
4 10 (3.00%) ISTQB
5 8 (2.40%) Computer Science Degree
6 3 (0.90%) PhD
7 2 (0.60%) DV Cleared
7 2 (0.60%) Numerate Degree
7 2 (0.60%) OCA
7 2 (0.60%) Oracle Certification
7 2 (0.60%) Sun Certification
8 1 (0.30%) BPSS Clearance
8 1 (0.30%) CISSP
8 1 (0.30%) ISEB
8 1 (0.30%) ISTQB Foundation Certification
Quality Assurance & Compliance
1 11 (3.30%) Data Quality
1 11 (3.30%) GDPR
2 8 (2.40%) QA
3 7 (2.10%) ISO/IEC 27001
4 1 (0.30%) Dodd-Frank
4 1 (0.30%) EMIR
4 1 (0.30%) MiFID
System Software
1 61 (18.32%) Apache ZooKeeper
2 43 (12.91%) Apache Flume
3 30 (9.01%) Docker
4 7 (2.10%) Active Directory
Systems Management
1 113 (33.93%) Hadoop YARN
2 76 (22.82%) Ansible
3 74 (22.22%) Oozie
4 46 (13.81%) Puppet
5 45 (13.51%) Opscode Chef
6 24 (7.21%) Kubernetes
7 21 (6.31%) Terraform
8 19 (5.71%) Ambari
9 7 (2.10%) Apache Slider
9 7 (2.10%) Single Sign-On
10 5 (1.50%) Kibana
10 5 (1.50%) Mesos
11 2 (0.60%) Packer
11 2 (0.60%) Salt
12 1 (0.30%) Apache Brooklyn
12 1 (0.30%) FreeIPA
12 1 (0.30%) Marathon
Vendors
1 89 (26.73%) Cloudera
2 68 (20.42%) Oracle
3 46 (13.81%) Hortonworks
3 46 (13.81%) Tableau
4 15 (4.50%) MapR
5 14 (4.20%) Google
6 10 (3.00%) Informatica
7 9 (2.70%) Atlassian
8 8 (2.40%) Cognos
8 8 (2.40%) Talend
9 7 (2.10%) Splunk
10 6 (1.80%) Alteryx
10 6 (1.80%) RapidMiner
11 5 (1.50%) IBM
11 5 (1.50%) Salesforce.com
11 5 (1.50%) SAP
12 4 (1.20%) Databricks
13 3 (0.90%) Red Hat
13 3 (0.90%) Sybase
13 3 (0.90%) Teradata