Period
to 21 January 2019

The following table provides summary statistics for contract job vacancies advertised in England with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 21 January 2019 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
UK > England
6 months to
21 Jan 2019
Same period 2018 Same period 2017
Rank 573 456 543
Rank change year-on-year -117 +87 +153
Contract jobs citing HDFS 280 432 293
As % of all contract IT jobs advertised in England 0.31% 0.48% 0.33%
As % of the System Software category 1.98% 3.66% 2.69%
Number of daily rates quoted 171 309 179
England median daily rate £550 £550 £550
10th Percentile £475 £400 £410
90th Percentile £725 £688 £713
UK median daily rate £550 £550 £555
% change year-on-year - -0.90% -7.50%
Number of hourly rates quoted 0 0 0
England median hourly rate - - -
UK median hourly rate - - -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in England with a requirement for system software skills.

All System Software Skills
England
Contract vacancies with a requirement for system software skills 14,109 11,799 10,910
As % of all contract IT jobs advertised in England 15.58% 12.98% 12.23%
Number of daily rates quoted 9,343 7,750 6,904
England median daily rate £475 £443 £413
Median daily rate % change year-on-year +7.34% +7.27% +8.55%
10th Percentile £188 £163 £170
90th Percentile £638 £625 £588
UK median daily rate £463 £425 £401
% change year-on-year +8.82% +6.12% +6.80%
Number of hourly rates quoted 715 629 738
England median hourly rate £16.00 £16.50 £16.15
Median hourly rate % change year-on-year -3.03% +2.17% +4.19%
10th Percentile £10.50 £10.40 £10.39
90th Percentile £44.27 £46.80 £43.75
UK median hourly rate £16.00 £16.08 £16.00
% change year-on-year -0.50% +0.50% +3.23%

HDFS
Job Vacancy Trend in England

Job postings citing HDFS as a percentage of all IT jobs advertised in England.

Job vacancy trend for HDFS in England

HDFS
Contractor Daily Rate Trend in England

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing HDFS in England.

Contractor daily rate trend for HDFS in England

HDFS
Contractor Daily Rate Histogram in England

The daily rate distribution of IT jobs citing HDFS in England over the 6 months to 21 January 2019.

Contractor daily rate histogram for HDFS in England

HDFS
Contractor Hourly Rate Trend in England

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing HDFS in England.

Contractor hourly rate trend for HDFS in England

HDFS
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the England region over the 6 months to 21 January 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London -93 163 £575 +4.07% 17
South East -28 75 £538 -3.15% 3
North of England 0 32 £525 - 3
North West +7 14 £550 -4.35% 1
North East - 13 £525 - 2
East of England +6 6 - - 2
Yorkshire -12 5 £510 +7.37%
South West -40 2 - - 3
HDFS
UK

For the 6 months to 21 January 2019, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region with a requirement for HDFS.

1 252 (90.00%) Hadoop
2 236 (84.29%) Big Data
3 166 (59.29%) Apache Spark
4 161 (57.50%) Agile Software Development
5 160 (57.14%) Java
6 158 (56.43%) Apache Hive
7 154 (55.00%) HBase
8 150 (53.57%) Kafka
9 129 (46.07%) Hadoop YARN
10 115 (41.07%) Sqoop
11 112 (40.00%) Linux
12 103 (36.79%) Python
12 103 (36.79%) Unix
13 93 (33.21%) Jenkins
14 92 (32.86%) SQL
15 87 (31.07%) Continuous Integration
16 86 (30.71%) Impala
17 83 (29.64%) Amazon AWS
18 74 (26.43%) TDD
19 71 (25.36%) Git (software)
19 71 (25.36%) Apache
20 70 (25.00%) Cloudera
20 70 (25.00%) BDD
21 68 (24.29%) Shell Script
21 68 (24.29%) Oozie
22 67 (23.93%) Oracle
23 65 (23.21%) SQL Server
23 65 (23.21%) DevOps
24 64 (22.86%) Apache ZooKeeper
24 64 (22.86%) Business Intelligence

HDFS
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 166 (59.29%) Apache Spark
2 71 (25.36%) Apache
3 33 (11.79%) Confluence
4 32 (11.43%) Apache Pig
5 31 (11.07%) Apache Solr
6 23 (8.21%) Elasticsearch
7 9 (3.21%) Cloud Foundry
7 9 (3.21%) SAS
8 6 (2.14%) Jupyter
9 3 (1.07%) GemFire
10 1 (0.36%) Apache Airflow
10 1 (0.36%) OpenStack
Applications
1 30 (10.71%) Microsoft Excel
2 28 (10.00%) MS Visio
3 2 (0.71%) Spreadsheet
Business Applications
1 1 (0.36%) Eloqua
1 1 (0.36%) Summit
Cloud Services
1 83 (29.64%) Amazon AWS
2 47 (16.79%) Microsoft Azure
3 46 (16.43%) GitHub
4 40 (14.29%) Amazon S3
5 30 (10.71%) Amazon EC2
6 28 (10.00%) Virtual Private Cloud
7 27 (9.64%) Google Cloud Platform
8 15 (5.36%) Cloud Computing
9 14 (5.00%) IaaS
10 12 (4.29%) PaaS
11 10 (3.57%) Amazon EMR
12 6 (2.14%) Amazon Kinesis
13 4 (1.43%) Serverless
14 3 (1.07%) IBM Cloud
15 2 (0.71%) AWS Elastic Beanstalk
16 1 (0.36%) Amazon CloudWatch
16 1 (0.36%) Amazon SQS
16 1 (0.36%) AWS Lambda
16 1 (0.36%) SaaS
Communications & Networking
1 31 (11.07%) Kerberos
2 11 (3.93%) VPN
3 9 (3.21%) Firewall
4 8 (2.86%) HTTPS
4 8 (2.86%) SSL
5 7 (2.50%) HTTP
6 5 (1.79%) LAN
6 5 (1.79%) WAN
7 2 (0.71%) Network Security
8 1 (0.36%) Internet
8 1 (0.36%) SSH
Database & Business Intelligence
1 252 (90.00%) Hadoop
2 236 (84.29%) Big Data
3 158 (56.43%) Apache Hive
4 154 (55.00%) HBase
5 115 (41.07%) Sqoop
6 86 (30.71%) Impala
7 65 (23.21%) SQL Server
8 55 (19.64%) Parquet
9 54 (19.29%) NoSQL
10 45 (16.07%) Apache Cassandra
11 36 (12.86%) MongoDB
12 30 (10.71%) Data Lake
12 30 (10.71%) Relational Database
13 26 (9.29%) Data Warehouse
14 23 (8.21%) MySQL
15 19 (6.79%) Apache Kudu
16 15 (5.36%) Spark SQL
17 14 (5.00%) Power BI
18 13 (4.64%) Mahout
19 12 (4.29%) PostgreSQL
Development Applications
1 93 (33.21%) Jenkins
2 71 (25.36%) Git (software)
3 36 (12.86%) JIRA
4 32 (11.43%) Cucumber
5 31 (11.07%) Gradle
6 15 (5.36%) JUnit
6 15 (5.36%) Maven
7 12 (4.29%) FishEye
8 11 (3.93%) JBehave
9 10 (3.57%) Atlassian Bamboo
9 10 (3.57%) jMock
9 10 (3.57%) Subversion
10 9 (3.21%) Selenium
11 6 (2.14%) Gerrit
11 6 (2.14%) IntelliJ
12 4 (1.43%) Ant
12 4 (1.43%) CruiseControl
12 4 (1.43%) GitLab
12 4 (1.43%) Sonatype Nexus
12 4 (1.43%) Visual Studio
General
1 41 (14.64%) Finance
2 19 (6.79%) Banking
3 16 (5.71%) Front Office
4 14 (5.00%) Legal
5 5 (1.79%) Games
5 5 (1.79%) Retail
6 3 (1.07%) Back Office
6 3 (1.07%) Telecoms
7 2 (0.71%) Advertising
7 2 (0.71%) Aerospace
7 2 (0.71%) Marketing
8 1 (0.36%) Investment Banking
8 1 (0.36%) Retail Banking
Job Titles
1 108 (38.57%) Developer
2 69 (24.64%) Java Developer
3 55 (19.64%) Data Engineer
4 53 (18.93%) Big Data Engineer
5 36 (12.86%) Big Data Developer
6 30 (10.71%) Architect
7 28 (10.00%) Hadoop Developer
7 28 (10.00%) Platform Engineer
8 27 (9.64%) Apache Spark Developer
9 22 (7.86%) DevOps Engineer
10 15 (5.36%) Hadoop Engineer
11 14 (5.00%) Data Architect
11 14 (5.00%) Java Hadoop Developer
12 13 (4.64%) Big Data Architect
13 12 (4.29%) Consultant
14 10 (3.57%) Big Data Consultant
14 10 (3.57%) Data Consultant
15 9 (3.21%) Hadoop Administrator
15 9 (3.21%) Hadoop Architect
15 9 (3.21%) Java Engineer
Libraries, Frameworks & Software Standards
1 150 (53.57%) Kafka
2 52 (18.57%) Spring
3 48 (17.14%) RabbitMQ
4 47 (16.79%) JSON
5 42 (15.00%) Apache Avro
5 42 (15.00%) REST
6 34 (12.14%) J2EE
6 34 (12.14%) Middleware
7 24 (8.57%) Spark Streaming
8 21 (7.50%) Apache NiFi
9 10 (3.57%) XML
10 9 (3.21%) LDAP
10 9 (3.21%) PySpark
11 8 (2.86%) MLlib
11 8 (2.86%) OAuth
11 8 (2.86%) Oracle GoldenGate
11 8 (2.86%) RESTful
11 8 (2.86%) Web Services
12 7 (2.50%) Anaconda
13 6 (2.14%) Spring Boot
Miscellaneous
1 35 (12.50%) Virtual Team
2 31 (11.07%) Linux Command Line
3 30 (10.71%) Wiki
4 22 (7.86%) Analytical Skills
5 20 (7.14%) Greenfield Project
6 14 (5.00%) Algorithms
7 12 (4.29%) Data Structures
8 11 (3.93%) Data Centre
9 9 (3.21%) Fintech
10 5 (1.79%) Management Information System
11 2 (0.71%) Cyber Defence
11 2 (0.71%) Cyberthreat
11 2 (0.71%) Private Cloud
11 2 (0.71%) User Experience
12 1 (0.36%) Analytical Mindset
12 1 (0.36%) Enterprise Software
12 1 (0.36%) Hybrid Cloud
12 1 (0.36%) Public Cloud
12 1 (0.36%) Self-Motivation
12 1 (0.36%) Taxonomies
Operating Systems
1 112 (40.00%) Linux
2 103 (36.79%) Unix
3 30 (10.71%) Windows
4 7 (2.50%) Red Hat Enterprise Linux
5 2 (0.71%) CoreOS
6 1 (0.36%) CentOS
6 1 (0.36%) DC/OS
6 1 (0.36%) Ubuntu
Processes & Methodologies
1 161 (57.50%) Agile Software Development
2 87 (31.07%) Continuous Integration
3 74 (26.43%) TDD
4 70 (25.00%) BDD
5 65 (23.21%) DevOps
6 64 (22.86%) Business Intelligence
7 63 (22.50%) ETL
8 51 (18.21%) MapReduce
8 51 (18.21%) Scrum
8 51 (18.21%) Test Automation
9 48 (17.14%) Analytics
9 48 (17.14%) Data Analysis
10 40 (14.29%) Data Modelling
11 37 (13.21%) Benchmarking
11 37 (13.21%) Visualisation
12 35 (12.50%) Data Extraction
13 32 (11.43%) Data Engineering
14 31 (11.07%) Build Automation
14 31 (11.07%) Unit Testing
15 30 (10.71%) Open Source
Programming Languages
1 160 (57.14%) Java
2 103 (36.79%) Python
3 92 (32.86%) SQL
4 68 (24.29%) Shell Script
5 53 (18.93%) Scala
6 48 (17.14%) Bash Shell
7 28 (10.00%) Korn
7 28 (10.00%) PowerShell
8 24 (8.57%) Java 8
9 13 (4.64%) R
10 10 (3.57%) C
10 10 (3.57%) JavaScript
11 9 (3.21%) C++
12 8 (2.86%) Go
13 7 (2.50%) Groovy
13 7 (2.50%) Ruby
14 3 (1.07%) MATLAB
14 3 (1.07%) Perl
15 2 (0.71%) Cypher
15 2 (0.71%) VBA
Qualifications
1 38 (13.57%) Degree
2 16 (5.71%) Security Cleared
3 15 (5.36%) SC Cleared
4 7 (2.50%) Computer Science Degree
5 5 (1.79%) ISTQB
6 3 (1.07%) PhD
7 2 (0.71%) BPSS Clearance
8 1 (0.36%) AWS Certification
8 1 (0.36%) SAS Certification
8 1 (0.36%) TOGAF Certification
Quality Assurance & Compliance
1 16 (5.71%) Data Quality
2 9 (3.21%) GDPR
3 8 (2.86%) QA
4 7 (2.50%) ISO/IEC 27001
5 5 (1.79%) SLA
6 1 (0.36%) Dodd-Frank
6 1 (0.36%) EMIR
6 1 (0.36%) MiFID
System Software
1 64 (22.86%) Apache ZooKeeper
2 49 (17.50%) Apache Flume
3 27 (9.64%) Docker
4 5 (1.79%) Oracle RAC
5 2 (0.71%) etcd
5 2 (0.71%) flannel
6 1 (0.36%) Cronacle
Systems Management
1 129 (46.07%) Hadoop YARN
2 68 (24.29%) Oozie
3 62 (22.14%) Ansible
4 34 (12.14%) Puppet
5 32 (11.43%) Opscode Chef
6 26 (9.29%) Ambari
7 22 (7.86%) Kubernetes
8 9 (3.21%) Terraform
9 6 (2.14%) Single Sign-On
10 5 (1.79%) Kibana
10 5 (1.79%) logstash
10 5 (1.79%) Tivoli
11 4 (1.43%) Grafana
12 2 (0.71%) Mesos
12 2 (0.71%) Prometheus
12 2 (0.71%) rkt
12 2 (0.71%) ZABBIX
13 1 (0.36%) Apache Brooklyn
13 1 (0.36%) Control-M
13 1 (0.36%) Ganglia
Vendors
1 70 (25.00%) Cloudera
2 67 (23.93%) Oracle
3 42 (15.00%) Hortonworks
4 29 (10.36%) Google
4 29 (10.36%) Tableau
5 16 (5.71%) Talend
6 12 (4.29%) Atlassian
7 10 (3.57%) Sybase
8 8 (2.86%) MapR
8 8 (2.86%) Splunk
9 6 (2.14%) Cognos
9 6 (2.14%) Informatica
9 6 (2.14%) Teradata
10 5 (1.79%) Cisco
10 5 (1.79%) IBM
10 5 (1.79%) Netezza
10 5 (1.79%) SAP
11 4 (1.43%) Datameer
12 3 (1.07%) Alteryx
12 3 (1.07%) Microsoft