Job Vacancy Trend Daily Rate Trend Daily Rate Histogram Hourly Rate Trend Job Locations Skill Sets
Period
to 19 June 2019

The following table provides summary statistics for contract job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 19 June 2019 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
UK
6 months to
19 Jun 2019		 Same period 2018 Same period 2017
Rank 575 555 571
Rank change year-on-year -20 +16 +188
Contract jobs citing HDFS 284 350 264
As % of all contract IT jobs advertised in the UK 0.31% 0.35% 0.29%
As % of the System Software category 2.03% 2.57% 2.58%
Number of daily rates quoted 168 238 188
UK median daily rate £575 £550 £525
Median daily rate % change year-on-year +4.55% +4.76% -
10th Percentile £443 £425 £376
90th Percentile £700 £658 £675
UK excluding London median daily rate £525 £513 £401
% change year-on-year +2.44% +27.97% -11.00%
Number of hourly rates quoted 0 3 2
UK median hourly rate - £22.00 £50.00
Median hourly rate % change year-on-year - -56.00% -
UK excluding London median hourly rate - £22.25 £50.00
% change year-on-year - -55.50% -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for system software skills.

All System Software Skills
UK
Contract vacancies with a requirement for system software skills 14,006 13,608 10,228
As % of all contract IT jobs advertised in the UK 15.44% 13.65% 11.24%
Number of daily rates quoted 9,272 8,784 6,553
UK median daily rate £475 £450 £420
Median daily rate % change year-on-year +5.56% +7.14% +5.00%
10th Percentile £181 £185 £163
90th Percentile £638 £625 £588
UK excluding London median daily rate £425 £400 £375
% change year-on-year +6.25% +6.67% +2.74%
Number of hourly rates quoted 608 807 655
UK median hourly rate £16.98 £16.00 £16.00
Median hourly rate % change year-on-year +6.09% - -
10th Percentile £11.00 £11.00 £10.13
90th Percentile £47.50 £45.32 £45.55
UK excluding London median hourly rate £15.76 £15.50 £15.00
% change year-on-year +1.68% +3.33% -6.25%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a percentage of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing HDFS.

Contractor daily rate trend for HDFS in the UK

HDFS
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing HDFS over the 6 months to 19 June 2019.

Contractor daily rate histogram for HDFS in the UK

HDFS
Contractor Hourly Rate Trend

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing HDFS.

Contractor hourly rate trend for HDFS in the UK

HDFS
Top 14 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the UK over the 6 months to 19 June 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year		 Matching
Contract
IT Job Ads		 Median
Daily Rate
Past 6 Months		 Median Daily Rate
% Change
on Same Period
Last Year		 Live
Job
Vacancies
England -40 243 £600 +9.09% 5
London -18 146 £600 +9.09% 3
UK excluding London +11 133 £525 +2.44% 3
South East -37 59 £525 -
Scotland +58 38 £475 -5.00% 1
North of England +46 16 £600 -
South West -20 12 £560 +14.87%
North East - 7 £575 -
North West +21 6 £650 +8.33%
East of England +9 5 £585 +30.00% 1
Yorkshire +19 3 - -
Northern Ireland - 2 - -
East Midlands +21 1 £450 -25.00% 1
Midlands +11 1 £450 -25.00% 1

For the 6 months to 19 June 2019, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for HDFS.

1 253 (89.08%) Hadoop
2 204 (71.83%) Big Data
3 203 (71.48%) Apache Spark
4 162 (57.04%) HBase
5 156 (54.93%) Apache Hive
6 130 (45.77%) Kafka
7 119 (41.90%) Java
8 110 (38.73%) Agile Software Development
9 98 (34.51%) Linux
10 92 (32.39%) Python
11 85 (29.93%) Impala
12 83 (29.23%) Finance
13 82 (28.87%) SQL
14 81 (28.52%) Unix
15 79 (27.82%) Amazon AWS
16 78 (27.46%) Scala
17 70 (24.65%) Hadoop YARN
18 63 (22.18%) DevOps
19 58 (20.42%) Jenkins
20 56 (19.72%) NoSQL
21 55 (19.37%) MapReduce
22 54 (19.01%) Continuous Integration
23 52 (18.31%) Parquet
24 50 (17.61%) Apache
25 49 (17.25%) Analytics
26 48 (16.90%) TDD
26 48 (16.90%) Banking
27 44 (15.49%) Spring
27 44 (15.49%) Shell Script
28 42 (14.79%) Microsoft Azure

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 203 (71.48%) Apache Spark
2 50 (17.61%) Apache
3 29 (10.21%) Apache Pig
4 22 (7.75%) Apache Solr
5 13 (4.58%) Elasticsearch
6 11 (3.87%) Confluence
7 7 (2.46%) Jupyter
8 5 (1.76%) Cloud Foundry
8 5 (1.76%) SAS
9 3 (1.06%) Apache Airflow
10 1 (0.35%) MS Exchange
10 1 (0.35%) OpenStack
10 1 (0.35%) SharePoint
10 1 (0.35%) Tornado
Business Applications
1 1 (0.35%) Eloqua
Cloud Services
1 79 (27.82%) Amazon AWS
2 42 (14.79%) Microsoft Azure
3 28 (9.86%) GitHub
4 26 (9.15%) Google Cloud Platform
5 24 (8.45%) Amazon EMR
6 23 (8.10%) Amazon S3
7 22 (7.75%) Serverless
8 18 (6.34%) Amazon Kinesis
9 17 (5.99%) Cloud Computing
10 11 (3.87%) Amazon EC2
11 9 (3.17%) Amazon CloudWatch
12 7 (2.46%) AWS Lambda
13 5 (1.76%) Amazon ECS
13 5 (1.76%) Azure Machine Learning
13 5 (1.76%) Data Lake Analytics
14 4 (1.41%) Amazon Glacier
14 4 (1.41%) IaaS
14 4 (1.41%) Virtual Private Cloud
15 3 (1.06%) OpenShift
15 3 (1.06%) Power Platform
Communications & Networking
1 22 (7.75%) Kerberos
2 7 (2.46%) TCP/IP
3 2 (0.70%) 2.5G
3 2 (0.70%) 2G
3 2 (0.70%) 3G
3 2 (0.70%) 4G
3 2 (0.70%) 5G
3 2 (0.70%) DSL
3 2 (0.70%) InfiniBand
3 2 (0.70%) MQTT
4 1 (0.35%) DMZ
4 1 (0.35%) SSL
Database & Business Intelligence
1 253 (89.08%) Hadoop
2 204 (71.83%) Big Data
3 162 (57.04%) HBase
4 156 (54.93%) Apache Hive
5 85 (29.93%) Impala
6 56 (19.72%) NoSQL
7 52 (18.31%) Parquet
8 39 (13.73%) Apache Cassandra
9 35 (12.32%) Sqoop
10 30 (10.56%) MongoDB
10 30 (10.56%) Relational Database
11 29 (10.21%) Data Lake
12 28 (9.86%) Data Warehouse
13 22 (7.75%) SQL Server
14 19 (6.69%) Spark SQL
15 18 (6.34%) PostgreSQL
16 14 (4.93%) MySQL
16 14 (4.93%) RDBMS
17 12 (4.23%) Amazon Redshift
18 10 (3.52%) Apache Kudu
Development Applications
1 58 (20.42%) Jenkins
2 36 (12.68%) Git (software)
3 29 (10.21%) JIRA
4 20 (7.04%) Gradle
5 12 (4.23%) Cucumber
6 11 (3.87%) Atlassian Bamboo
6 11 (3.87%) Maven
6 11 (3.87%) Subversion
7 10 (3.52%) JUnit
8 9 (3.17%) TeamCity
9 8 (2.82%) Travis CI
10 7 (2.46%) Eclipse
10 7 (2.46%) FindBugs
10 7 (2.46%) IntelliJ
10 7 (2.46%) Selenium
11 5 (1.76%) Sonatype Nexus
12 4 (1.41%) SoapUI
12 4 (1.41%) Visual Studio
13 3 (1.06%) FishEye
13 3 (1.06%) jMock
General
1 83 (29.23%) Finance
2 48 (16.90%) Banking
3 9 (3.17%) Investment Banking
3 9 (3.17%) Retail Banking
4 7 (2.46%) Telecoms
5 6 (2.11%) Retail
6 5 (1.76%) Front Office
6 5 (1.76%) General Ledger
6 5 (1.76%) Legal
7 3 (1.06%) Marketing
7 3 (1.06%) Pharmaceutical
8 2 (0.70%) Games
9 1 (0.35%) Advertising
9 1 (0.35%) Aerospace
9 1 (0.35%) Manufacturing
Job Titles
1 109 (38.38%) Developer
2 51 (17.96%) Java Developer
3 48 (16.90%) Big Data Developer
4 44 (15.49%) Data Engineer
5 42 (14.79%) Administrator
6 40 (14.08%) Hadoop Administrator
7 38 (13.38%) Architect
7 38 (13.38%) Hadoop Developer
8 28 (9.86%) Big Data Engineer
9 20 (7.04%) DevOps Engineer
10 19 (6.69%) Java Hadoop Developer
11 18 (6.34%) Hadoop Engineer
11 18 (6.34%) Scala Developer
12 17 (5.99%) Data Architect
13 12 (4.23%) Big Data Architect
14 11 (3.87%) AWS Developer
14 11 (3.87%) Python Developer
14 11 (3.87%) Solutions Architect
15 10 (3.52%) Cloud Engineer
16 9 (3.17%) AWS Engineer
Libraries, Frameworks & Software Standards
1 130 (45.77%) Kafka
2 44 (15.49%) Spring
3 41 (14.44%) Apache Avro
4 33 (11.62%) REST
5 25 (8.80%) Web Services
6 22 (7.75%) RabbitMQ
7 21 (7.39%) JSON
8 20 (7.04%) Middleware
9 17 (5.99%) PySpark
10 16 (5.63%) Flask
10 16 (5.63%) J2EE
10 16 (5.63%) Natural Language Processing
11 12 (4.23%) Apache Camel
11 12 (4.23%) RESTful
12 11 (3.87%) Elastic Stack
12 11 (3.87%) MLlib
12 11 (3.87%) NumPy
12 11 (3.87%) Spring Boot
13 9 (3.17%) AngularJS
13 9 (3.17%) scikit-learn
Miscellaneous
1 26 (9.15%) Linux Command Line
2 20 (7.04%) Algorithms
2 20 (7.04%) Analytical Skills
3 18 (6.34%) Data Structures
3 18 (6.34%) Virtual Team
4 17 (5.99%) Greenfield Project
5 6 (2.11%) Cloud Native
5 6 (2.11%) Credit Risk
6 5 (1.76%) Hedge funds
6 5 (1.76%) PKI
7 3 (1.06%) Enterprise Software
7 3 (1.06%) Management Information System
8 2 (0.70%) EMC VNX
8 2 (0.70%) Enterprise Cloud
8 2 (0.70%) Market Risk
8 2 (0.70%) NHS
8 2 (0.70%) Replication
8 2 (0.70%) Self-Motivation
9 1 (0.35%) Data Centre
9 1 (0.35%) Hybrid Cloud
Operating Systems
1 98 (34.51%) Linux
2 81 (28.52%) Unix
3 9 (3.17%) Red Hat Enterprise Linux
4 8 (2.82%) Windows
5 7 (2.46%) Ubuntu
6 3 (1.06%) SUSE Linux Enterprise Server
Processes & Methodologies
1 110 (38.73%) Agile Software Development
2 63 (22.18%) DevOps
3 55 (19.37%) MapReduce
4 54 (19.01%) Continuous Integration
5 49 (17.25%) Analytics
6 48 (16.90%) TDD
7 39 (13.73%) ETL
8 38 (13.38%) Data Modelling
9 36 (12.68%) Data Science
10 35 (12.32%) BDD
11 33 (11.62%) Business Intelligence
12 30 (10.56%) Performance Tuning
12 30 (10.56%) SDLC
13 25 (8.80%) Problem-Solving
13 25 (8.80%) Scrum
14 24 (8.45%) Microservices
15 22 (7.75%) Visualisation
16 21 (7.39%) Machine Learning
17 20 (7.04%) Infrastructure as Code
18 19 (6.69%) Software Engineering
Programming Languages
1 119 (41.90%) Java
2 92 (32.39%) Python
3 82 (28.87%) SQL
4 78 (27.46%) Scala
5 44 (15.49%) Shell Script
6 34 (11.97%) R
7 19 (6.69%) JavaScript
8 11 (3.87%) Bash Shell
9 9 (3.17%) Go
10 8 (2.82%) C++
10 8 (2.82%) Java 8
10 8 (2.82%) Ruby
11 6 (2.11%) PL/SQL
12 5 (1.76%) C#
13 4 (1.41%) C
14 3 (1.06%) Korn
14 3 (1.06%) MATLAB
14 3 (1.06%) Perl
15 2 (0.70%) Groovy
15 2 (0.70%) PowerShell
Qualifications
1 21 (7.39%) Security Cleared
2 19 (6.69%) Degree
3 14 (4.93%) SC Cleared
4 5 (1.76%) Computer Science Degree
5 3 (1.06%) AWS Certification
5 3 (1.06%) DV Cleared
6 2 (0.70%) ISTQB
6 2 (0.70%) TOGAF Certification
7 1 (0.35%) BPSS Clearance
7 1 (0.35%) CISSP
7 1 (0.35%) ISEB
7 1 (0.35%) MCSE
7 1 (0.35%) Microsoft Certification
7 1 (0.35%) PhD
7 1 (0.35%) SAS Certification
Quality Assurance & Compliance
1 15 (5.28%) Data Quality
2 4 (1.41%) GDPR
2 4 (1.41%) QA
3 2 (0.70%) ISO/IEC 27001
4 1 (0.35%) PSD2
System Software
1 38 (13.38%) Docker
2 35 (12.32%) Apache Flume
3 32 (11.27%) Apache ZooKeeper
4 11 (3.87%) vCloud
4 11 (3.87%) VMware Infrastructure
5 7 (2.46%) Active Directory
6 6 (2.11%) vSphere
7 1 (0.35%) VMware ESXi
Systems Management
1 70 (24.65%) Hadoop YARN
2 33 (11.62%) Ansible
3 31 (10.92%) Kubernetes
4 30 (10.56%) Oozie
5 29 (10.21%) Puppet
6 25 (8.80%) Terraform
7 8 (2.82%) Opscode Chef
8 5 (1.76%) vCenter Server
9 4 (1.41%) Ambari
9 4 (1.41%) logstash
9 4 (1.41%) Tivoli
10 3 (1.06%) Grafana
10 3 (1.06%) Kibana
11 2 (0.70%) Nagios
11 2 (0.70%) Prometheus
11 2 (0.70%) ZABBIX
12 1 (0.35%) Consul
12 1 (0.35%) Control-M
12 1 (0.35%) HAProxy
12 1 (0.35%) Webmin
Vendors
1 35 (12.32%) Hortonworks
2 34 (11.97%) Oracle
3 32 (11.27%) Cloudera
4 25 (8.80%) Google
5 13 (4.58%) Teradata
6 12 (4.23%) VMware
7 11 (3.87%) Databricks
7 11 (3.87%) Tableau
8 10 (3.52%) Microsoft
9 9 (3.17%) Splunk
10 8 (2.82%) Ab Initio
11 7 (2.46%) New Relic
12 6 (2.11%) Atlassian
12 6 (2.11%) Datameer
12 6 (2.11%) Informatica
12 6 (2.11%) MapR
13 5 (1.76%) Netezza
13 5 (1.76%) Sybase
13 5 (1.76%) Talend
14 4 (1.41%) Red Hat