Period
to 17 August 2019

The following table provides summary statistics for contract job vacancies advertised in England with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 17 August 2019 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
UK > England
6 months to
17 Aug 2019
Same period 2018 Same period 2017
Rank 606 542 485
Rank change year-on-year -64 -57 +244
Contract jobs citing HDFS 204 342 359
As % of all contract IT jobs advertised in England 0.25% 0.36% 0.41%
As % of the System Software category 1.57% 2.52% 3.41%
Number of daily rates quoted 125 236 250
England median daily rate £575 £531 £525
Median daily rate % change year-on-year +8.24% +1.19% -4.55%
10th Percentile £465 £444 £350
90th Percentile £688 £675 £700
UK median daily rate £570 £525 £525
% change year-on-year +8.57% - -4.55%
Number of hourly rates quoted 0 3 2
England median hourly rate - £22.00 £50.00
Median hourly rate % change year-on-year - -56.00% -
UK median hourly rate - £22.00 £50.00
% change year-on-year - -56.00% -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in England with a requirement for system software skills.

All System Software Skills
England
Contract vacancies with a requirement for system software skills 13,013 13,547 10,532
As % of all contract IT jobs advertised in England 15.65% 14.39% 11.93%
Number of daily rates quoted 8,716 8,824 6,801
England median daily rate £475 £450 £425
Median daily rate % change year-on-year +5.56% +5.88% +4.94%
10th Percentile £181 £185 £175
90th Percentile £638 £625 £594
UK median daily rate £470 £450 £425
% change year-on-year +4.44% +5.88% +6.25%
Number of hourly rates quoted 538 806 598
England median hourly rate £16.54 £16.50 £16.00
Median hourly rate % change year-on-year +0.24% +3.13% +1.59%
10th Percentile £11.17 £11.09 £10.34
90th Percentile £47.83 £45.19 £45.55
UK median hourly rate £17.00 £16.36 £16.00
% change year-on-year +3.93% +2.23% -3.03%

HDFS
Job Vacancy Trend in England

Job postings citing HDFS as a percentage of all IT jobs advertised in England.

Job vacancy trend for HDFS in England

HDFS
Contractor Daily Rate Trend in England

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing HDFS in England.

Contractor daily rate trend for HDFS in England

HDFS
Contractor Daily Rate Histogram in England

The daily rate distribution of IT jobs citing HDFS in England over the 6 months to 17 August 2019.

Contractor daily rate histogram for HDFS in England

HDFS
Contractor Hourly Rate Trend in England

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing HDFS in England.

Contractor hourly rate trend for HDFS in England

HDFS
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the England region over the 6 months to 17 August 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London -47 124 £600 +9.09% 14
South East -10 52 £523 +4.50% 1
South West -13 11 £560 +7.95%
North of England +29 7 £600 +20.00% 1
East of England +18 5 £585 -0.43% 1
North West +12 5 £600 +20.00%
Yorkshire +15 2 £600 +41.18% 1
Midlands +28 1 £450 +5.88% 1
East Midlands +23 1 £450 -
HDFS
UK

For the 6 months to 17 August 2019, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region with a requirement for HDFS.

1 180 (88.24%) Hadoop
2 156 (76.47%) Apache Spark
3 126 (61.76%) Big Data
4 112 (54.90%) HBase
5 104 (50.98%) Apache Hive
6 83 (40.69%) Kafka
7 82 (40.20%) Java
8 80 (39.22%) Agile Software Development
9 75 (36.76%) Python
10 71 (34.80%) Linux
11 55 (26.96%) Finance
12 54 (26.47%) Amazon AWS
13 53 (25.98%) SQL
14 52 (25.49%) Hadoop YARN
15 49 (24.02%) Impala
16 48 (23.53%) Unix
17 47 (23.04%) DevOps
18 44 (21.57%) Jenkins
19 41 (20.10%) Scala
19 41 (20.10%) Continuous Integration
20 39 (19.12%) Analytics
21 37 (18.14%) Kubernetes
22 36 (17.65%) TDD
22 36 (17.65%) NoSQL
23 35 (17.16%) Docker
24 34 (16.67%) Apache
25 33 (16.18%) Ansible
25 33 (16.18%) Cloudera
25 33 (16.18%) Data Lake
26 31 (15.20%) Shell Script

HDFS
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 156 (76.47%) Apache Spark
2 34 (16.67%) Apache
3 17 (8.33%) Apache Pig
4 13 (6.37%) Apache Solr
5 11 (5.39%) Elasticsearch
6 6 (2.94%) Jupyter
7 5 (2.45%) Confluence
8 4 (1.96%) Apache Airflow
9 1 (0.49%) Tornado
Cloud Services
1 54 (26.47%) Amazon AWS
2 30 (14.71%) Microsoft Azure
3 27 (13.24%) Serverless
4 19 (9.31%) Google Cloud Platform
5 18 (8.82%) Amazon EMR
5 18 (8.82%) GitHub
6 17 (8.33%) Amazon S3
7 14 (6.86%) Amazon Kinesis
8 11 (5.39%) Amazon CloudWatch
9 10 (4.90%) Amazon ECS
9 10 (4.90%) AWS Lambda
10 8 (3.92%) Amazon Glacier
11 7 (3.43%) Amazon EC2
12 5 (2.45%) Azure Machine Learning
12 5 (2.45%) Data Lake Analytics
13 4 (1.96%) AWS CloudFormation
13 4 (1.96%) AWS CloudTrail
14 3 (1.47%) Cloud Computing
14 3 (1.47%) Virtual Private Cloud
15 2 (0.98%) PaaS
Communications & Networking
1 20 (9.80%) Kerberos
2 7 (3.43%) TCP/IP
3 2 (0.98%) 2.5G
3 2 (0.98%) 2G
3 2 (0.98%) 3G
3 2 (0.98%) 4G
3 2 (0.98%) 5G
3 2 (0.98%) DSL
3 2 (0.98%) SSL
4 1 (0.49%) DMZ
4 1 (0.49%) InfiniBand
Database & Business Intelligence
1 180 (88.24%) Hadoop
2 126 (61.76%) Big Data
3 112 (54.90%) HBase
4 104 (50.98%) Apache Hive
5 49 (24.02%) Impala
6 36 (17.65%) NoSQL
7 33 (16.18%) Data Lake
8 31 (15.20%) Parquet
9 24 (11.76%) Sqoop
10 22 (10.78%) Data Warehouse
11 17 (8.33%) SQL Server
12 15 (7.35%) Relational Database
13 13 (6.37%) Cloudera CDH
13 13 (6.37%) PostgreSQL
14 11 (5.39%) Apache Cassandra
15 8 (3.92%) Apache Kudu
16 7 (3.43%) HDInsight
17 6 (2.94%) Amazon Redshift
17 6 (2.94%) MySQL
18 5 (2.45%) RDBMS
Development Applications
1 44 (21.57%) Jenkins
2 25 (12.25%) Git (software)
3 18 (8.82%) JIRA
4 13 (6.37%) Gradle
5 11 (5.39%) Atlassian Bamboo
5 11 (5.39%) Travis CI
6 8 (3.92%) TeamCity
7 6 (2.94%) Jasmine
8 5 (2.45%) Sonatype Nexus
9 4 (1.96%) Cucumber
10 3 (1.47%) JUnit
10 3 (1.47%) Maven
10 3 (1.47%) Selenium
11 2 (0.98%) Eclipse
11 2 (0.98%) FindBugs
11 2 (0.98%) Gatling
11 2 (0.98%) IntelliJ
11 2 (0.98%) JMeter
11 2 (0.98%) Subversion
11 2 (0.98%) WebDriver
General
1 55 (26.96%) Finance
2 28 (13.73%) Banking
3 11 (5.39%) Retail Banking
4 6 (2.94%) Telecoms
5 5 (2.45%) Retail
6 2 (0.98%) Games
6 2 (0.98%) Investment Banking
6 2 (0.98%) Marketing
7 1 (0.49%) Advertising
7 1 (0.49%) Legal
Job Titles
1 62 (30.39%) Developer
2 39 (19.12%) Architect
3 33 (16.18%) Data Engineer
4 32 (15.69%) Administrator
4 32 (15.69%) Java Developer
5 30 (14.71%) Hadoop Administrator
6 28 (13.73%) Hadoop Developer
7 22 (10.78%) Big Data Developer
8 16 (7.84%) Big Data Engineer
8 16 (7.84%) Data Architect
8 16 (7.84%) Java Hadoop Developer
9 12 (5.88%) AWS Engineer
9 12 (5.88%) Cloud Engineer
10 9 (4.41%) DevOps Engineer
10 9 (4.41%) Hadoop Engineer
10 9 (4.41%) Platform Engineer
10 9 (4.41%) Solutions Architect
11 8 (3.92%) AWS Developer
11 8 (3.92%) Cloud Architect
11 8 (3.92%) Hadoop Architect
Libraries, Frameworks & Software Standards
1 83 (40.69%) Kafka
2 25 (12.25%) Spring
3 21 (10.29%) PySpark
4 20 (9.80%) Apache Avro
5 15 (7.35%) RabbitMQ
6 14 (6.86%) Web Services
7 13 (6.37%) Middleware
7 13 (6.37%) REST
8 12 (5.88%) Elastic Stack
8 12 (5.88%) RESTful
9 10 (4.90%) JSON
10 8 (3.92%) J2EE
11 7 (3.43%) Pandas
11 7 (3.43%) Spark Streaming
12 6 (2.94%) Apache Camel
12 6 (2.94%) Drools
12 6 (2.94%) Flask
13 5 (2.45%) Apache NiFi
13 5 (2.45%) Natural Language Processing
13 5 (2.45%) React
Miscellaneous
1 22 (10.78%) Linux Command Line
2 16 (7.84%) Greenfield Project
3 13 (6.37%) Cloud Native
4 11 (5.39%) Algorithms
4 11 (5.39%) Virtual Team
5 7 (3.43%) Data Structures
6 5 (2.45%) Analytical Skills
6 5 (2.45%) PKI
7 3 (1.47%) Enterprise Software
7 3 (1.47%) Replication
8 2 (0.98%) Market Risk
8 2 (0.98%) NHS
9 1 (0.49%) Credit Risk
9 1 (0.49%) Derivative
9 1 (0.49%) EMC VNX
9 1 (0.49%) Enterprise Cloud
9 1 (0.49%) Mainframe
9 1 (0.49%) Public Cloud
Operating Systems
1 71 (34.80%) Linux
2 48 (23.53%) Unix
3 12 (5.88%) Red Hat Enterprise Linux
4 8 (3.92%) Windows
5 7 (3.43%) Ubuntu
6 4 (1.96%) SUSE Linux Enterprise Server
7 1 (0.49%) Android
7 1 (0.49%) Apple iOS
Processes & Methodologies
1 80 (39.22%) Agile Software Development
2 47 (23.04%) DevOps
3 41 (20.10%) Continuous Integration
4 39 (19.12%) Analytics
5 36 (17.65%) TDD
6 26 (12.75%) ETL
7 24 (11.76%) Data Modelling
7 24 (11.76%) Data Science
8 22 (10.78%) MapReduce
8 22 (10.78%) Performance Tuning
9 16 (7.84%) BDD
9 16 (7.84%) Computer Science
9 16 (7.84%) OOP
9 16 (7.84%) SDLC
10 15 (7.35%) Configuration Management
11 14 (6.86%) Data Analytics
12 13 (6.37%) Business Intelligence
12 13 (6.37%) Data Ingestion
12 13 (6.37%) Infrastructure as Code
13 12 (5.88%) Software Engineering
Programming Languages
1 82 (40.20%) Java
2 75 (36.76%) Python
3 53 (25.98%) SQL
4 41 (20.10%) Scala
5 31 (15.20%) Shell Script
6 21 (10.29%) R
7 10 (4.90%) JavaScript
8 9 (4.41%) Bash Shell
9 8 (3.92%) PL/SQL
10 7 (3.43%) Go
10 7 (3.43%) Ruby
11 5 (2.45%) C#
11 5 (2.45%) Java 8
12 4 (1.96%) Perl
13 3 (1.47%) Korn
14 2 (0.98%) Groovy
14 2 (0.98%) PowerShell
Qualifications
1 16 (7.84%) Security Cleared
2 11 (5.39%) Degree
3 9 (4.41%) SC Cleared
4 3 (1.47%) DV Cleared
5 2 (0.98%) AWS Certification
5 2 (0.98%) Computer Science Degree
6 1 (0.49%) CISSP
6 1 (0.49%) ISTQB
6 1 (0.49%) MCSE
6 1 (0.49%) Microsoft Certification
6 1 (0.49%) TOGAF Certification
Quality Assurance & Compliance
1 2 (0.98%) Data Quality
1 2 (0.98%) GDPR
2 1 (0.49%) QA
System Software
1 35 (17.16%) Docker
2 26 (12.75%) Apache ZooKeeper
3 18 (8.82%) Apache Flume
4 7 (3.43%) vSphere
5 6 (2.94%) Active Directory
6 2 (0.98%) VMware ESXi
Systems Management
1 52 (25.49%) Hadoop YARN
2 37 (18.14%) Kubernetes
3 33 (16.18%) Ansible
4 28 (13.73%) Terraform
5 27 (13.24%) Oozie
6 22 (10.78%) Puppet
7 5 (2.45%) Grafana
7 5 (2.45%) Opscode Chef
7 5 (2.45%) vCenter Server
8 3 (1.47%) logstash
9 2 (0.98%) Ambari
9 2 (0.98%) Kibana
10 1 (0.49%) Consul
10 1 (0.49%) Control-M
10 1 (0.49%) HAProxy
10 1 (0.49%) Nagios
10 1 (0.49%) Webmin
Vendors
1 33 (16.18%) Cloudera
2 26 (12.75%) Hortonworks
2 26 (12.75%) Oracle
3 18 (8.82%) Google
4 14 (6.86%) Splunk
5 11 (5.39%) Databricks
6 8 (3.92%) Microsoft
6 8 (3.92%) Teradata
7 7 (3.43%) Informatica
7 7 (3.43%) New Relic
7 7 (3.43%) Red Hat
8 6 (2.94%) Ab Initio
8 6 (2.94%) MapR
8 6 (2.94%) Talend
9 5 (2.45%) Tableau
10 3 (1.47%) Atlassian
10 3 (1.47%) MicroStrategy
11 2 (0.98%) SAP
11 2 (0.98%) Sybase
11 2 (0.98%) VMware