Job Vacancy Trend Daily Rate Trend Daily Rate Histogram Hourly Rate Trend Job Locations Skill Sets
Period
to 21 November 2018

The following table provides summary statistics for contract job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 21 November 2018 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
UK
6 months to
21 Nov 2018		 Same period 2017 Same period 2016
Rank 567 504 631
Rank change year-on-year -63 +127 +102
Contract jobs citing HDFS 332 428 270
As % of all contract IT jobs advertised in the UK 0.32% 0.41% 0.26%
As % of the System Software category 2.11% 3.29% 2.09%
Number of daily rates quoted 225 310 156
UK median daily rate £525 £550 £550
Median daily rate % change year-on-year -4.55% - -11.11%
10th Percentile £439 £387 £405
90th Percentile £700 £700 £725
UK excluding London median daily rate £516 £488 £519
% change year-on-year +5.90% -6.07% +15.33%
Number of hourly rates quoted 3 1 0
UK median hourly rate £22.00 £50.00 -
Median hourly rate % change year-on-year -56.00% - -
UK excluding London median hourly rate £22.25 £50.00 -
% change year-on-year -55.50% - -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for system software skills.

All System Software Skills
UK
Contract vacancies with a requirement for system software skills 15,761 12,991 12,943
As % of all contract IT jobs advertised in the UK 15.14% 12.59% 12.36%
Number of daily rates quoted 10,301 8,435 8,106
UK median daily rate £450 £425 £400
Median daily rate % change year-on-year +5.88% +6.25% +9.59%
10th Percentile £181 £156 £173
90th Percentile £625 £610 £575
UK excluding London median daily rate £401 £375 £350
% change year-on-year +6.80% +7.14% +6.06%
Number of hourly rates quoted 876 726 880
UK median hourly rate £16.00 £16.50 £16.00
Median hourly rate % change year-on-year -3.03% +3.13% -
10th Percentile £10.58 £10.40 £10.50
90th Percentile £43.75 £46.82 £42.00
UK excluding London median hourly rate £15.86 £15.00 £15.00
% change year-on-year +5.73% - -6.25%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a percentage of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing HDFS.

Contractor daily rate trend for HDFS in the UK

HDFS
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing HDFS over the 6 months to 21 November 2018.

Contractor daily rate histogram for HDFS in the UK

HDFS
Contractor Hourly Rate Trend

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing HDFS.

Contractor hourly rate trend for HDFS in the UK

HDFS
Top 14 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the UK over the 6 months to 21 November 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year		 Matching
Contract
IT Job Ads		 Median
Daily Rate
Past 6 Months		 Median Daily Rate
% Change
on Same Period
Last Year		 Live
Job
Vacancies
England -59 330 £525 -4.55% 36
London -64 213 £550 - 21
UK excluding London 0 116 £516 +5.90% 15
South East -5 76 £513 +2.50% 6
North of England +9 24 £510 +7.37% 2
North West -9 15 £500 +23.46% 2
East of England 0 11 £594 +21.79% 3
Yorkshire -13 5 £510 +7.37%
North East - 4 £525 -
South West -19 2 - - 2
Scotland - 2 £469 -
Midlands - 2 £425 - 2
East Midlands - 1 - - 2
West Midlands - 1 £425 -

For the 6 months to 21 November 2018, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for HDFS.

1 298 (89.76%) Hadoop
2 273 (82.23%) Big Data
3 212 (63.86%) Apache Spark
4 196 (59.04%) Apache Hive
5 193 (58.13%) Java
6 176 (53.01%) Agile Software Development
7 170 (51.20%) Kafka
7 170 (51.20%) HBase
8 126 (37.95%) Python
9 122 (36.75%) Sqoop
10 120 (36.14%) Hadoop YARN
11 113 (34.04%) Unix
12 111 (33.43%) SQL
13 109 (32.83%) Linux
14 105 (31.63%) Jenkins
15 102 (30.72%) Impala
16 90 (27.11%) Continuous Integration
17 86 (25.90%) Amazon AWS
18 82 (24.70%) SQL Server
18 82 (24.70%) ETL
19 80 (24.10%) TDD
20 79 (23.80%) Scrum
20 79 (23.80%) Oozie
21 77 (23.19%) Ansible
21 77 (23.19%) Cloudera
22 75 (22.59%) Oracle
23 73 (21.99%) Git (software)
24 72 (21.69%) Scala
25 71 (21.39%) NoSQL
25 71 (21.39%) Shell Script

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 212 (63.86%) Apache Spark
2 67 (20.18%) Apache
3 36 (10.84%) Apache Pig
4 35 (10.54%) Apache Solr
4 35 (10.54%) Confluence
5 23 (6.93%) Elasticsearch
6 11 (3.31%) SAS
7 9 (2.71%) GemFire
8 7 (2.11%) Cloud Foundry
9 2 (0.60%) Apache Airflow
10 1 (0.30%) WebLogic
Applications
1 42 (12.65%) Microsoft Excel
2 40 (12.05%) MS Visio
3 2 (0.60%) Spreadsheet
Business Applications
1 1 (0.30%) Summit
Cloud Services
1 86 (25.90%) Amazon AWS
2 51 (15.36%) Amazon S3
3 41 (12.35%) Amazon EC2
4 40 (12.05%) Virtual Private Cloud
5 39 (11.75%) Microsoft Azure
6 37 (11.14%) GitHub
7 18 (5.42%) Google Cloud Platform
8 15 (4.52%) PaaS
9 12 (3.61%) IaaS
10 9 (2.71%) Amazon EMR
11 7 (2.11%) Cloud Computing
12 5 (1.51%) Amazon Kinesis
13 3 (0.90%) AWS Elastic Beanstalk
13 3 (0.90%) IBM Cloud
14 1 (0.30%) SaaS
Communications & Networking
1 43 (12.95%) Kerberos
2 13 (3.92%) SSL
2 13 (3.92%) VPN
3 10 (3.01%) Firewall
3 10 (3.01%) HTTP
4 9 (2.71%) HTTPS
5 5 (1.51%) LAN
5 5 (1.51%) RTMP
5 5 (1.51%) WAN
6 2 (0.60%) Network Security
7 1 (0.30%) SSH
Database & Business Intelligence
1 298 (89.76%) Hadoop
2 273 (82.23%) Big Data
3 196 (59.04%) Apache Hive
4 170 (51.20%) HBase
5 122 (36.75%) Sqoop
6 102 (30.72%) Impala
7 82 (24.70%) SQL Server
8 71 (21.39%) NoSQL
9 46 (13.86%) MongoDB
10 44 (13.25%) Parquet
11 43 (12.95%) Relational Database
12 37 (11.14%) Apache Cassandra
13 35 (10.54%) Data Lake
14 33 (9.94%) Data Warehouse
15 19 (5.72%) Apache Kudu
15 19 (5.72%) PostgreSQL
16 17 (5.12%) MySQL
17 15 (4.52%) OLTP
18 12 (3.61%) RDBMS
19 11 (3.31%) Apache Zeppelin
Development Applications
1 105 (31.63%) Jenkins
2 73 (21.99%) Git (software)
3 39 (11.75%) JIRA
4 37 (11.14%) Cucumber
5 22 (6.63%) IntelliJ
6 21 (6.33%) Gradle
7 18 (5.42%) Maven
8 15 (4.52%) JUnit
9 12 (3.61%) JBehave
10 11 (3.31%) Selenium
11 9 (2.71%) Eclipse
11 9 (2.71%) FishEye
12 7 (2.11%) GitLab
13 6 (1.81%) Atlassian Bamboo
13 6 (1.81%) Gerrit
13 6 (1.81%) jMock
13 6 (1.81%) Subversion
13 6 (1.81%) Visual Studio Team System
14 5 (1.51%) Ant
14 5 (1.51%) Sonatype Nexus
General
1 68 (20.48%) Finance
2 27 (8.13%) Banking
3 16 (4.82%) Legal
4 14 (4.22%) Front Office
5 10 (3.01%) Investment Banking
6 9 (2.71%) Retail
7 6 (1.81%) Games
8 4 (1.20%) Marketing
9 3 (0.90%) Back Office
10 2 (0.60%) Advertising
10 2 (0.60%) Telecoms
11 1 (0.30%) French Language
11 1 (0.30%) Manufacturing
11 1 (0.30%) Retail Banking
Job Titles
1 137 (41.27%) Developer
2 74 (22.29%) Java Developer
3 69 (20.78%) Data Engineer
4 60 (18.07%) Big Data Engineer
5 57 (17.17%) Big Data Developer
6 40 (12.05%) Platform Engineer
7 33 (9.94%) Apache Spark Developer
8 32 (9.64%) Hadoop Developer
9 26 (7.83%) Architect
10 22 (6.63%) Hadoop Engineer
11 19 (5.72%) Data Architect
11 19 (5.72%) DevOps Engineer
12 18 (5.42%) Big Data Architect
13 14 (4.22%) Java Hadoop Developer
14 10 (3.01%) Senior Data Engineer
14 10 (3.01%) Software Engineer
15 9 (2.71%) Consultant
15 9 (2.71%) SC Cleared Java Developer
15 9 (2.71%) Scala Developer
16 8 (2.41%) AWS Engineer
Libraries, Frameworks & Software Standards
1 170 (51.20%) Kafka
2 48 (14.46%) Spring
3 45 (13.55%) RabbitMQ
4 41 (12.35%) Spark Streaming
5 40 (12.05%) JSON
6 32 (9.64%) Apache Avro
6 32 (9.64%) REST
7 28 (8.43%) Middleware
8 26 (7.83%) J2EE
9 16 (4.82%) Spring Boot
10 15 (4.52%) RESTful
11 14 (4.22%) XML
12 10 (3.01%) HTML
12 10 (3.01%) Node.js
13 9 (2.71%) Apache Camel
13 9 (2.71%) Drools
13 9 (2.71%) LDAP
14 8 (2.41%) .NET
14 8 (2.41%) OAuth
15 7 (2.11%) MLlib
Miscellaneous
1 43 (12.95%) Wiki
2 35 (10.54%) Analytical Skills
3 31 (9.34%) Greenfield Project
4 30 (9.04%) Algorithms
5 27 (8.13%) Virtual Team
6 22 (6.63%) Linux Command Line
7 18 (5.42%) Data Structures
8 17 (5.12%) Computer Science
9 10 (3.01%) Fintech
10 9 (2.71%) Self-Motivation
11 7 (2.11%) User Experience
12 6 (1.81%) Data Centre
12 6 (1.81%) Management Information System
13 5 (1.51%) Derivative
13 5 (1.51%) Equity Derivative
14 2 (0.60%) Cyber Defence
14 2 (0.60%) Cyberthreat
14 2 (0.60%) Enterprise Software
14 2 (0.60%) Private Cloud
15 1 (0.30%) Analytical Mindset
Operating Systems
1 113 (34.04%) Unix
2 109 (32.83%) Linux
3 44 (13.25%) Windows
4 5 (1.51%) Red Hat Enterprise Linux
5 3 (0.90%) DC/OS
6 2 (0.60%) CoreOS
7 1 (0.30%) CentOS
7 1 (0.30%) Ubuntu
Processes & Methodologies
1 176 (53.01%) Agile Software Development
2 90 (27.11%) Continuous Integration
3 82 (24.70%) ETL
4 80 (24.10%) TDD
5 79 (23.80%) Scrum
6 67 (20.18%) Data Analysis
7 65 (19.58%) BDD
7 65 (19.58%) DevOps
8 64 (19.28%) Test Automation
9 62 (18.67%) Analytics
10 58 (17.47%) Business Intelligence
11 49 (14.76%) Data Extraction
12 45 (13.55%) Data Modelling
13 44 (13.25%) MapReduce
14 42 (12.65%) Build Automation
15 37 (11.14%) Data Science
16 35 (10.54%) Open Source
17 30 (9.04%) Problem-Solving
18 28 (8.43%) Software Engineering
18 28 (8.43%) Visualisation
Programming Languages
1 193 (58.13%) Java
2 126 (37.95%) Python
3 111 (33.43%) SQL
4 72 (21.69%) Scala
5 71 (21.39%) Shell Script
6 59 (17.77%) Bash Shell
7 40 (12.05%) Korn
7 40 (12.05%) PowerShell
8 31 (9.34%) Java 8
9 17 (5.12%) JavaScript
10 15 (4.52%) Ruby
11 13 (3.92%) R
12 10 (3.01%) C++
13 8 (2.41%) Groovy
14 6 (1.81%) C
14 6 (1.81%) C#
15 5 (1.51%) Perl
15 5 (1.51%) T-SQL
15 5 (1.51%) TypeScript
16 3 (0.90%) Go
Qualifications
1 33 (9.94%) Degree
2 24 (7.23%) Security Cleared
3 22 (6.63%) SC Cleared
4 8 (2.41%) Computer Science Degree
5 6 (1.81%) ISTQB
6 3 (0.90%) PhD
7 2 (0.60%) DV Cleared
7 2 (0.60%) Sun Certification
8 1 (0.30%) BPSS Clearance
8 1 (0.30%) CISSP
8 1 (0.30%) ISEB
8 1 (0.30%) ISTQB Foundation Certification
8 1 (0.30%) SAS Certification
Quality Assurance & Compliance
1 16 (4.82%) Data Quality
2 12 (3.61%) GDPR
3 9 (2.71%) QA
4 7 (2.11%) ISO/IEC 27001
5 5 (1.51%) SLA
6 1 (0.30%) Dodd-Frank
6 1 (0.30%) EMIR
6 1 (0.30%) MiFID
System Software
1 57 (17.17%) Apache ZooKeeper
2 40 (12.05%) Apache Flume
3 36 (10.84%) Docker
4 5 (1.51%) Oracle RAC
5 3 (0.90%) Active Directory
6 2 (0.60%) etcd
6 2 (0.60%) flannel
7 1 (0.30%) Cronacle
Systems Management
1 120 (36.14%) Hadoop YARN
2 79 (23.80%) Oozie
3 77 (23.19%) Ansible
4 47 (14.16%) Puppet
5 41 (12.35%) Opscode Chef
6 33 (9.94%) Kubernetes
7 24 (7.23%) Ambari
8 17 (5.12%) Terraform
9 7 (2.11%) Single Sign-On
10 5 (1.51%) Kibana
11 4 (1.20%) Mesos
12 3 (0.90%) Apache Slider
12 3 (0.90%) Tivoli
13 2 (0.60%) Apache Brooklyn
13 2 (0.60%) Grafana
13 2 (0.60%) logstash
13 2 (0.60%) Marathon
13 2 (0.60%) rkt
14 1 (0.30%) Control-M
14 1 (0.30%) Prometheus
Vendors
1 77 (23.19%) Cloudera
2 75 (22.59%) Oracle
3 48 (14.46%) Hortonworks
4 36 (10.84%) Tableau
5 21 (6.33%) Google
6 17 (5.12%) Talend
7 14 (4.22%) MapR
8 11 (3.31%) Informatica
9 10 (3.01%) Atlassian
10 9 (2.71%) Splunk
11 8 (2.41%) Cognos
12 7 (2.11%) SAP
12 7 (2.11%) Sybase
13 5 (1.51%) Alteryx
13 5 (1.51%) Cisco
13 5 (1.51%) IBM
13 5 (1.51%) RapidMiner
14 4 (1.20%) Databricks
15 3 (0.90%) Red Hat
15 3 (0.90%) Salesforce.com