Period
to 21 November 2019

The following table provides summary statistics for contract job vacancies advertised in England with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 21 November 2019 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
UK > England
6 months to
21 Nov 2019
Same period 2018 Same period 2017
Rank 625 551 488
Rank change year-on-year -74 -63 +139
Contract jobs citing HDFS 144 330 425
As % of all contract IT jobs advertised in England 0.19% 0.35% 0.45%
As % of the System Software category 1.23% 2.26% 3.51%
Number of daily rates quoted 75 223 307
England median daily rate £550 £525 £550
Median daily rate % change year-on-year +4.76% -4.55% -
10th Percentile £429 £442 £387
90th Percentile £700 £700 £700
UK median daily rate £525 £525 £550
% change year-on-year - -4.55% -
Number of hourly rates quoted 0 3 1
England median hourly rate - £22.00 £50.00
Median hourly rate % change year-on-year - -56.00% -
UK median hourly rate - £22.00 £50.00
% change year-on-year - -56.00% -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in England with a requirement for system software skills.

All System Software Skills
England
Contract vacancies with a requirement for system software skills 11,750 14,630 12,112
As % of all contract IT jobs advertised in England 15.48% 15.37% 12.77%
Number of daily rates quoted 7,794 9,630 7,939
England median daily rate £450 £455 £425
Median daily rate % change year-on-year -1.10% +7.06% +3.66%
10th Percentile £168 £188 £161
90th Percentile £638 £635 £613
UK median daily rate £450 £450 £425
% change year-on-year - +5.88% +6.25%
Number of hourly rates quoted 603 791 662
England median hourly rate £16.50 £16.00 £16.50
Median hourly rate % change year-on-year +3.13% -3.03% +3.13%
10th Percentile £11.17 £10.50 £10.40
90th Percentile £47.50 £43.75 £47.25
UK median hourly rate £17.00 £16.00 £16.50
% change year-on-year +6.25% -3.03% +3.13%

HDFS
Job Vacancy Trend in England

Job postings citing HDFS as a percentage of all IT jobs advertised in England.

Job vacancy trend for HDFS in England

HDFS
Contractor Daily Rate Trend in England

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing HDFS in England.

Contractor daily rate trend for HDFS in England

HDFS
Contractor Daily Rate Histogram in England

The daily rate distribution of IT jobs citing HDFS in England over the 6 months to 21 November 2019.

Contractor daily rate histogram for HDFS in England

HDFS
Contractor Hourly Rate Trend in England

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing HDFS in England.

Contractor hourly rate trend for HDFS in England

HDFS
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the England region over the 6 months to 21 November 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London -39 85 £550 - 8
South East +3 38 £528 +3.02% 1
North of England +26 11 £600 +17.65% 5
South West +1 6 £610 -
Yorkshire +37 5 £500 -1.96% 1
North West +10 5 £600 +20.00% 3
Midlands +49 2 £513 +20.59% 2
East Midlands +28 2 £513 - 1
East of England +22 1 £488 -17.89% 3
North East +20 1 £800 +52.38% 1
HDFS
UK

For the 6 months to 21 November 2019, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region with a requirement for HDFS.

1 126 (87.50%) Hadoop
2 97 (67.36%) Apache Spark
3 81 (56.25%) Apache Hive
4 74 (51.39%) Big Data
5 69 (47.92%) Python
6 65 (45.14%) HBase
7 63 (43.75%) SQL
8 58 (40.28%) Kafka
9 52 (36.11%) Java
10 51 (35.42%) Linux
10 51 (35.42%) Agile Software Development
11 47 (32.64%) Scala
12 45 (31.25%) Hadoop YARN
13 42 (29.17%) Impala
14 35 (24.31%) DevOps
15 34 (23.61%) AWS
15 34 (23.61%) Unix
16 33 (22.92%) ETL
17 31 (21.53%) Cloudera
18 30 (20.83%) Analytics
19 28 (19.44%) NoSQL
20 27 (18.75%) Continuous Integration
20 27 (18.75%) Finance
20 27 (18.75%) Data Modelling
21 25 (17.36%) Azure
22 23 (15.97%) Hortonworks
22 23 (15.97%) Oracle
23 22 (15.28%) R
24 21 (14.58%) Data Lake
25 20 (13.89%) PySpark

HDFS
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 97 (67.36%) Apache Spark
2 19 (13.19%) Apache
3 12 (8.33%) Elasticsearch
4 11 (7.64%) Jupyter
5 7 (4.86%) Apache Pig
6 6 (4.17%) Confluence
7 5 (3.47%) Apache Solr
8 4 (2.78%) SAS
9 3 (2.08%) Apache Airflow
10 2 (1.39%) Cloud Foundry
10 2 (1.39%) Tomcat
10 2 (1.39%) Tornado
10 2 (1.39%) WebLogic
11 1 (0.69%) GemFire
11 1 (0.69%) Luigi
11 1 (0.69%) OpenStack
Applications
1 6 (4.17%) Microsoft Excel
2 1 (0.69%) Microsoft Project
Business Applications
1 2 (1.39%) Oracle Applications
2 1 (0.69%) SAP MDM
Cloud Services
1 34 (23.61%) AWS
2 25 (17.36%) Azure
3 15 (10.42%) Amazon Kinesis
4 12 (8.33%) Amazon EMR
4 12 (8.33%) AWS Lambda
4 12 (8.33%) Serverless
5 11 (7.64%) Amazon S3
5 11 (7.64%) GCP
6 8 (5.56%) Power Platform
7 7 (4.86%) GitHub
8 5 (3.47%) Amazon ECS
9 4 (2.78%) Amazon CloudWatch
9 4 (2.78%) Amazon Glacier
9 4 (2.78%) AWS CloudTrail
10 2 (1.39%) Amazon EC2
10 2 (1.39%) Amazon ELB
10 2 (1.39%) AWS CloudFormation
10 2 (1.39%) AWS Step Functions
10 2 (1.39%) OpenShift
10 2 (1.39%) Virtual Private Cloud
Communications & Networking
1 8 (5.56%) Kerberos
2 3 (2.08%) SSL
3 1 (0.69%) HTTP
3 1 (0.69%) InfiniBand
3 1 (0.69%) Internet
3 1 (0.69%) VPN
Database & Business Intelligence
1 126 (87.50%) Hadoop
2 81 (56.25%) Apache Hive
3 74 (51.39%) Big Data
4 65 (45.14%) HBase
5 42 (29.17%) Impala
6 28 (19.44%) NoSQL
7 21 (14.58%) Data Lake
8 20 (13.89%) Sqoop
9 19 (13.19%) Data Warehouse
10 16 (11.11%) SQL Server
11 14 (9.72%) Spark SQL
12 12 (8.33%) Relational Database
13 11 (7.64%) Parquet
14 10 (6.94%) Apache Cassandra
14 10 (6.94%) MongoDB
15 9 (6.25%) Cloudera CDH
16 8 (5.56%) Oracle Database
16 8 (5.56%) Power BI
17 6 (4.17%) Flink
17 6 (4.17%) Neo4j
Development Applications
1 17 (11.81%) Jenkins
2 16 (11.11%) JIRA
3 12 (8.33%) Git (software)
4 8 (5.56%) Travis CI
5 6 (4.17%) Jasmine
6 5 (3.47%) Cucumber
7 4 (2.78%) Atlassian Bamboo
7 4 (2.78%) Selenium
8 3 (2.08%) kappa
8 3 (2.08%) Maven
8 3 (2.08%) Sonatype Nexus
9 2 (1.39%) Bitbucket
9 2 (1.39%) Gatling
9 2 (1.39%) IBM UrbanCode
9 2 (1.39%) JMeter
9 2 (1.39%) JUnit
9 2 (1.39%) WebDriver
10 1 (0.69%) Eclipse
10 1 (0.69%) FishEye
10 1 (0.69%) IntelliJ
General
1 27 (18.75%) Finance
2 9 (6.25%) Retail Banking
3 6 (4.17%) Banking
4 5 (3.47%) Public Sector
5 4 (2.78%) Telecoms
6 3 (2.08%) Investment Banking
7 2 (1.39%) Manufacturing
7 2 (1.39%) Retail
8 1 (0.69%) Electronics
8 1 (0.69%) Games
8 1 (0.69%) Marketing
Job Titles
1 34 (23.61%) Developer
2 32 (22.22%) Data Engineer
3 30 (20.83%) Architect
4 18 (12.50%) Big Data Engineer
5 14 (9.72%) Administrator
5 14 (9.72%) Data Architect
6 12 (8.33%) Hadoop Developer
7 11 (7.64%) Hadoop Architect
8 9 (6.25%) Java Developer
9 8 (5.56%) Python Developer
10 7 (4.86%) Java Hadoop Developer
10 7 (4.86%) Systems Architect
11 6 (4.17%) Big Data Architect
11 6 (4.17%) Python Engineer
12 5 (3.47%) AWS Engineer
12 5 (3.47%) Cloud Architect
12 5 (3.47%) Hadoop Engineer
12 5 (3.47%) Scala Developer
12 5 (3.47%) Software Engineer
13 4 (2.78%) ETL Developer
Libraries, Frameworks & Software Standards
1 58 (40.28%) Kafka
2 20 (13.89%) PySpark
3 10 (6.94%) JSON
3 10 (6.94%) RabbitMQ
4 9 (6.25%) XML
5 8 (5.56%) Pandas
6 7 (4.86%) Flask
6 7 (4.86%) React
6 7 (4.86%) Spring
6 7 (4.86%) Web Services
7 6 (4.17%) Elastic Stack
7 6 (4.17%) RESTful
8 5 (3.47%) Apache NiFi
8 5 (3.47%) HTML
9 4 (2.78%) Natural Language Processing
10 3 (2.08%) REST
10 3 (2.08%) Spark Streaming
11 2 (1.39%) Keycloak
11 2 (1.39%) Oracle GoldenGate
11 2 (1.39%) TensorFlow
Miscellaneous
1 10 (6.94%) Algorithms
2 9 (6.25%) Linux Command Line
3 8 (5.56%) Analytical Skills
3 8 (5.56%) Greenfield Project
4 7 (4.86%) Cloud Native
5 5 (3.47%) PKI
6 4 (2.78%) Data Structures
7 3 (2.08%) Derivative
8 2 (1.39%) Management Information System
8 2 (1.39%) NHS
8 2 (1.39%) Replication
9 1 (0.69%) Clustering
9 1 (0.69%) EMC VNX
9 1 (0.69%) Equities
9 1 (0.69%) Public Cloud
9 1 (0.69%) Reference Data
9 1 (0.69%) Smart Energy
9 1 (0.69%) Virtual Team
Operating Systems
1 51 (35.42%) Linux
2 34 (23.61%) Unix
3 13 (9.03%) Ubuntu
4 12 (8.33%) Red Hat Enterprise Linux
5 11 (7.64%) Windows
6 2 (1.39%) Oracle Linux
6 2 (1.39%) SUSE Linux Enterprise Server
7 1 (0.69%) Android
7 1 (0.69%) Apple iOS
7 1 (0.69%) CentOS
Processes & Methodologies
1 51 (35.42%) Agile Software Development
2 35 (24.31%) DevOps
3 33 (22.92%) ETL
4 30 (20.83%) Analytics
5 27 (18.75%) Continuous Integration
5 27 (18.75%) Data Modelling
6 19 (13.19%) Performance Tuning
7 18 (12.50%) Data Analytics
7 18 (12.50%) TDD
7 18 (12.50%) Use Case
7 18 (12.50%) Visualisation
8 17 (11.81%) Data Engineering
8 17 (11.81%) Migration
8 17 (11.81%) SDLC
9 16 (11.11%) Business Intelligence
10 15 (10.42%) Computer Science
10 15 (10.42%) Data Visualisation
10 15 (10.42%) OOP
10 15 (10.42%) Software Engineering
11 14 (9.72%) Data Architecture
Programming Languages
1 69 (47.92%) Python
2 63 (43.75%) SQL
3 52 (36.11%) Java
4 47 (32.64%) Scala
5 22 (15.28%) R
6 13 (9.03%) JavaScript
7 9 (6.25%) PL/SQL
8 8 (5.56%) Bash Shell
8 8 (5.56%) Go
8 8 (5.56%) Kotlin
8 8 (5.56%) Shell Script
9 6 (4.17%) C#
10 4 (2.78%) Perl
11 2 (1.39%) C
11 2 (1.39%) HiveQL
11 2 (1.39%) Java 8
12 1 (0.69%) Korn
12 1 (0.69%) PHP
12 1 (0.69%) PowerShell
12 1 (0.69%) Ruby
Qualifications
1 20 (13.89%) Security Cleared
2 12 (8.33%) SC Cleared
3 3 (2.08%) Computer Science Degree
3 3 (2.08%) Degree
4 2 (1.39%) DV Cleared
4 2 (1.39%) ISTQB
4 2 (1.39%) Microsoft Certification
5 1 (0.69%) BPSS Clearance
5 1 (0.69%) CISSP
5 1 (0.69%) MCSE
Quality Assurance & Compliance
1 3 (2.08%) Data Quality
2 2 (1.39%) QA
3 1 (0.69%) Disclosure Scotland
System Software
1 13 (9.03%) Apache ZooKeeper
2 12 (8.33%) Docker
3 8 (5.56%) Apache Flume
4 7 (4.86%) vSphere
5 6 (4.17%) Active Directory
6 2 (1.39%) VMware ESXi
Systems Management
1 45 (31.25%) Hadoop YARN
2 16 (11.11%) Kubernetes
3 15 (10.42%) Oozie
4 13 (9.03%) Ansible
5 7 (4.86%) Puppet
5 7 (4.86%) Terraform
6 6 (4.17%) Grafana
7 5 (3.47%) Opscode Chef
7 5 (3.47%) vCenter Server
8 4 (2.78%) logstash
9 3 (2.08%) Ambari
10 1 (0.69%) Kibana
10 1 (0.69%) Nagios
Vendors
1 31 (21.53%) Cloudera
2 23 (15.97%) Hortonworks
2 23 (15.97%) Oracle
3 16 (11.11%) Tableau
4 15 (10.42%) Microsoft
5 9 (6.25%) Teradata
6 8 (5.56%) Google
6 8 (5.56%) Splunk
7 7 (4.86%) Foundry
7 7 (4.86%) Palantir
7 7 (4.86%) Red Hat
8 6 (4.17%) Talend
9 4 (2.78%) Ab Initio
10 3 (2.08%) Atlassian
10 3 (2.08%) Databricks
10 3 (2.08%) Informatica
11 2 (1.39%) MicroStrategy
11 2 (1.39%) NVIDIA
11 2 (1.39%) SAP
11 2 (1.39%) Sybase