Job Vacancy Trend Daily Rate Trend Daily Rate Histogram Hourly Rate Trend Job Locations Skill Sets
Period
to 20 October 2019

The following table provides summary statistics for contract job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 20 October 2019 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
UK
6 months to
20 Oct 2019		 Same period 2018 Same period 2017
Rank 608 591 520
Rank change year-on-year -17 -71 +157
Contract jobs citing HDFS 208 319 403
As % of all contract IT jobs advertised in the UK 0.25% 0.31% 0.41%
As % of the System Software category 1.58% 2.08% 3.32%
Number of daily rates quoted 116 226 288
UK median daily rate £550 £525 £550
Median daily rate % change year-on-year +4.76% -4.55% -
10th Percentile £438 £439 £376
90th Percentile £699 £694 £702
UK excluding London median daily rate £525 £500 £475
% change year-on-year +5.00% +5.26% -7.41%
Number of hourly rates quoted 0 3 2
UK median hourly rate - £22.00 £50.00
Median hourly rate % change year-on-year - -56.00% -
UK excluding London median hourly rate - £22.25 £50.00
% change year-on-year - -55.50% -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for system software skills.

All System Software Skills
UK
Contract vacancies with a requirement for system software skills 13,166 15,304 12,143
As % of all contract IT jobs advertised in the UK 15.60% 14.95% 12.22%
Number of daily rates quoted 8,683 9,968 7,825
UK median daily rate £450 £450 £425
Median daily rate % change year-on-year - +5.88% +6.25%
10th Percentile £165 £180 £158
90th Percentile £638 £625 £600
UK excluding London median daily rate £413 £400 £375
% change year-on-year +3.13% +6.67% +4.17%
Number of hourly rates quoted 666 889 677
UK median hourly rate £17.00 £16.00 £16.66
Median hourly rate % change year-on-year +6.25% -3.96% +4.13%
10th Percentile £11.17 £10.50 £10.40
90th Percentile £47.38 £43.75 £47.25
UK excluding London median hourly rate £16.00 £15.50 £15.50
% change year-on-year +3.23% - +0.94%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a percentage of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing HDFS.

Contractor daily rate trend for HDFS in the UK

HDFS
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing HDFS over the 6 months to 20 October 2019.

Contractor daily rate histogram for HDFS in the UK

HDFS
Contractor Hourly Rate Trend

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing HDFS.

Contractor hourly rate trend for HDFS in the UK

HDFS
Top 14 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the UK over the 6 months to 20 October 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year		 Matching
Contract
IT Job Ads		 Median
Daily Rate
Past 6 Months		 Median Daily Rate
% Change
on Same Period
Last Year		 Live
Job
Vacancies
England -49 165 £563 +7.14% 23
UK excluding London +56 104 £525 +5.00% 6
London -26 100 £550 +4.76% 18
South East +6 42 £550 +10.00% 1
Scotland +67 40 £525 +5.00% 1
North of England +45 10 £600 +18.81%
South West -4 6 £610 - 2
North West +16 5 £600 +14.29%
Yorkshire +41 4 £550 +17.65%
Midlands +46 2 £513 +20.59% 1
East Midlands +29 2 £513 - 1
East of England +16 2 £569 -3.19% 1
Northern Ireland - 2 - -
North East +17 1 £800 +64.10%

For the 6 months to 20 October 2019, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for HDFS.

1 179 (86.06%) Hadoop
2 151 (72.60%) Apache Spark
3 129 (62.02%) Apache Hive
4 127 (61.06%) Big Data
5 107 (51.44%) Kafka
6 102 (49.04%) HBase
7 85 (40.87%) SQL
7 85 (40.87%) Impala
8 84 (40.38%) Scala
9 80 (38.46%) Java
10 79 (37.98%) Python
11 64 (30.77%) Agile Software Development
12 56 (26.92%) Linux
13 48 (23.08%) AWS
13 48 (23.08%) Hadoop YARN
14 44 (21.15%) ETL
15 43 (20.67%) DevOps
15 43 (20.67%) Cloudera
16 42 (20.19%) Unix
17 40 (19.23%) Banking
17 40 (19.23%) Spark SQL
18 38 (18.27%) Oracle
19 36 (17.31%) NoSQL
19 36 (17.31%) Analytics
20 35 (16.83%) Data Modelling
20 35 (16.83%) Finance
21 34 (16.35%) Jenkins
22 31 (14.90%) Data Lake
22 31 (14.90%) Continuous Integration
23 29 (13.94%) Azure

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 151 (72.60%) Apache Spark
2 21 (10.10%) Apache
3 13 (6.25%) Elasticsearch
4 11 (5.29%) Jupyter
5 10 (4.81%) Apache Pig
6 6 (2.88%) Confluence
6 6 (2.88%) SAS
7 5 (2.40%) Apache Airflow
7 5 (2.40%) Apache Solr
8 2 (0.96%) Tomcat
8 2 (0.96%) Tornado
8 2 (0.96%) WebLogic
9 1 (0.48%) Cloud Foundry
9 1 (0.48%) Luigi
9 1 (0.48%) OpenStack
Applications
1 6 (2.88%) Microsoft Excel
2 1 (0.48%) Microsoft Project
Business Applications
1 2 (0.96%) Oracle Applications
Cloud Services
1 48 (23.08%) AWS
2 29 (13.94%) Azure
3 21 (10.10%) Amazon Kinesis
4 20 (9.62%) Serverless
5 18 (8.65%) Amazon EMR
5 18 (8.65%) Amazon S3
6 14 (6.73%) GCP
7 8 (3.85%) Amazon CloudWatch
7 8 (3.85%) AWS Lambda
7 8 (3.85%) Power Platform
8 5 (2.40%) Amazon ECS
8 5 (2.40%) GitHub
9 4 (1.92%) Amazon Glacier
9 4 (1.92%) AWS CloudFormation
9 4 (1.92%) AWS CloudTrail
10 2 (0.96%) Amazon ELB
10 2 (0.96%) Amazon SQS
10 2 (0.96%) Azure Machine Learning
10 2 (0.96%) Data Lake Analytics
10 2 (0.96%) Virtual Private Cloud
Communications & Networking
1 10 (4.81%) Kerberos
2 4 (1.92%) TCP/IP
3 3 (1.44%) SSL
4 2 (0.96%) InfiniBand
5 1 (0.48%) HTTP
5 1 (0.48%) Internet
5 1 (0.48%) VPN
Database & Business Intelligence
1 179 (86.06%) Hadoop
2 129 (62.02%) Apache Hive
3 127 (61.06%) Big Data
4 102 (49.04%) HBase
5 85 (40.87%) Impala
6 40 (19.23%) Spark SQL
7 36 (17.31%) NoSQL
8 31 (14.90%) Data Lake
9 25 (12.02%) Sqoop
10 24 (11.54%) SQL Server
11 22 (10.58%) Data Warehouse
11 22 (10.58%) Parquet
12 16 (7.69%) Relational Database
13 11 (5.29%) MongoDB
14 10 (4.81%) Cloudera CDH
15 8 (3.85%) Apache Cassandra
15 8 (3.85%) Oracle Database
15 8 (3.85%) Power BI
16 6 (2.88%) Neo4j
16 6 (2.88%) PostgreSQL
Development Applications
1 34 (16.35%) Jenkins
2 19 (9.13%) Git (software)
2 19 (9.13%) JIRA
3 9 (4.33%) Travis CI
4 8 (3.85%) Atlassian Bamboo
5 7 (3.37%) JUnit
5 7 (3.37%) Maven
6 6 (2.88%) Jasmine
7 5 (2.40%) Sonatype Nexus
7 5 (2.40%) Subversion
8 4 (1.92%) Eclipse
8 4 (1.92%) FindBugs
8 4 (1.92%) IntelliJ
8 4 (1.92%) TeamCity
9 3 (1.44%) Cucumber
9 3 (1.44%) Selenium
10 2 (0.96%) Bitbucket
10 2 (0.96%) Gatling
10 2 (0.96%) JMeter
10 2 (0.96%) WebDriver
General
1 40 (19.23%) Banking
2 35 (16.83%) Finance
3 18 (8.65%) Investment Banking
4 13 (6.25%) Retail Banking
5 5 (2.40%) Public Sector
5 5 (2.40%) Retail
5 5 (2.40%) Telecoms
6 3 (1.44%) Manufacturing
6 3 (1.44%) Pharmaceutical
7 1 (0.48%) Electronics
7 1 (0.48%) Games
7 1 (0.48%) Marketing
Job Titles
1 69 (33.17%) Developer
2 45 (21.63%) Data Engineer
3 35 (16.83%) Architect
4 22 (10.58%) Scala Developer
5 20 (9.62%) Big Data Engineer
5 20 (9.62%) Hadoop Developer
6 17 (8.17%) Administrator
6 17 (8.17%) Big Data Developer
6 17 (8.17%) Data Architect
7 16 (7.69%) Java Developer
8 10 (4.81%) Java Engineer
9 9 (4.33%) Hadoop Architect
10 8 (3.85%) Java Hadoop Developer
10 8 (3.85%) Python Developer
11 7 (3.37%) DevOps Engineer
11 7 (3.37%) Hadoop Administrator
11 7 (3.37%) Systems Architect
12 6 (2.88%) Cloud Architect
12 6 (2.88%) Hadoop Engineer
13 5 (2.40%) Business Intelligence Developer
Libraries, Frameworks & Software Standards
1 107 (51.44%) Kafka
2 22 (10.58%) PySpark
3 16 (7.69%) Spring
4 15 (7.21%) Web Services
5 11 (5.29%) Elastic Stack
6 9 (4.33%) Apache Avro
6 9 (4.33%) RabbitMQ
6 9 (4.33%) RESTful
7 8 (3.85%) Apache NiFi
7 8 (3.85%) JSON
7 8 (3.85%) Pandas
7 8 (3.85%) Spring Boot
7 8 (3.85%) XML
8 7 (3.37%) Flask
8 7 (3.37%) React
8 7 (3.37%) REST
8 7 (3.37%) Spark Streaming
9 5 (2.40%) HTML
10 4 (1.92%) Natural Language Processing
11 2 (0.96%) Highcharts JS
Miscellaneous
1 19 (9.13%) Analytical Skills
2 13 (6.25%) Algorithms
3 10 (4.81%) Greenfield Project
4 9 (4.33%) Linux Command Line
5 8 (3.85%) Cloud Native
5 8 (3.85%) Data Structures
6 5 (2.40%) PKI
7 3 (1.44%) Derivative
7 3 (1.44%) Replication
8 2 (0.96%) EMC VNX
8 2 (0.96%) Management Information System
8 2 (0.96%) NHS
9 1 (0.48%) Clustering
9 1 (0.48%) Enterprise Software
9 1 (0.48%) Equities
9 1 (0.48%) Mainframe
9 1 (0.48%) Public Cloud
9 1 (0.48%) Reference Data
Operating Systems
1 56 (26.92%) Linux
2 42 (20.19%) Unix
3 13 (6.25%) Red Hat Enterprise Linux
3 13 (6.25%) Ubuntu
4 12 (5.77%) Windows
5 2 (0.96%) Oracle Linux
5 2 (0.96%) SUSE Linux Enterprise Server
6 1 (0.48%) Android
6 1 (0.48%) Apple iOS
6 1 (0.48%) CentOS
Processes & Methodologies
1 64 (30.77%) Agile Software Development
2 44 (21.15%) ETL
3 43 (20.67%) DevOps
4 36 (17.31%) Analytics
5 35 (16.83%) Data Modelling
6 31 (14.90%) Continuous Integration
7 29 (13.94%) Data Development
7 29 (13.94%) Performance Tuning
8 24 (11.54%) Data Architecture
8 24 (11.54%) Data Ingestion
9 23 (11.06%) Business Intelligence
10 22 (10.58%) Problem-Solving
10 22 (10.58%) SDLC
11 21 (10.10%) Software Engineering
12 20 (9.62%) Data Science
12 20 (9.62%) TDD
12 20 (9.62%) Use Case
13 19 (9.13%) Visualisation
14 18 (8.65%) Data Analysis
15 17 (8.17%) Migration
Programming Languages
1 85 (40.87%) SQL
2 84 (40.38%) Scala
3 80 (38.46%) Java
4 79 (37.98%) Python
5 25 (12.02%) R
6 17 (8.17%) Shell Script
7 15 (7.21%) JavaScript
8 12 (5.77%) Go
8 12 (5.77%) PL/SQL
9 11 (5.29%) Bash Shell
10 8 (3.85%) Kotlin
11 6 (2.88%) C#
12 5 (2.40%) Ruby
13 4 (1.92%) Perl
14 3 (1.44%) Java 8
15 2 (0.96%) C
15 2 (0.96%) HiveQL
16 1 (0.48%) Korn
16 1 (0.48%) PHP
16 1 (0.48%) PowerShell
Qualifications
1 20 (9.62%) Security Cleared
2 12 (5.77%) SC Cleared
3 7 (3.37%) Degree
4 3 (1.44%) Computer Science Degree
5 2 (0.96%) DV Cleared
5 2 (0.96%) Microsoft Certification
6 1 (0.48%) BPSS Clearance
6 1 (0.48%) CISSP
6 1 (0.48%) ISTQB
6 1 (0.48%) MCSE
Quality Assurance & Compliance
1 10 (4.81%) Data Quality
2 1 (0.48%) Disclosure Scotland
System Software
1 18 (8.65%) Docker
2 15 (7.21%) Apache Flume
3 12 (5.77%) Apache ZooKeeper
4 7 (3.37%) vSphere
5 6 (2.88%) Active Directory
6 2 (0.96%) VMware ESXi
Systems Management
1 48 (23.08%) Hadoop YARN
2 21 (10.10%) Kubernetes
2 21 (10.10%) Oozie
3 17 (8.17%) Ansible
4 11 (5.29%) Puppet
4 11 (5.29%) Terraform
5 6 (2.88%) Grafana
6 5 (2.40%) logstash
6 5 (2.40%) vCenter Server
7 4 (1.92%) Opscode Chef
8 3 (1.44%) Ambari
8 3 (1.44%) Control-M
9 2 (0.96%) Kibana
9 2 (0.96%) Nagios
Vendors
1 43 (20.67%) Cloudera
2 38 (18.27%) Oracle
3 28 (13.46%) Hortonworks
4 16 (7.69%) Tableau
5 15 (7.21%) Microsoft
6 14 (6.73%) Teradata
7 12 (5.77%) Splunk
8 10 (4.81%) Google
9 8 (3.85%) Talend
10 7 (3.37%) Foundry
10 7 (3.37%) Palantir
10 7 (3.37%) Red Hat
11 6 (2.88%) Databricks
12 5 (2.40%) Ab Initio
12 5 (2.40%) Informatica
13 4 (1.92%) New Relic
14 3 (1.44%) Atlassian
14 3 (1.44%) MicroStrategy
14 3 (1.44%) Sybase
15 2 (0.96%) Huawei