Job Vacancy Trend Daily Rate Trend Daily Rate Histogram Hourly Rate Trend Job Locations Skill Sets
Period
to 17 January 2020

The following table provides summary statistics for contract job vacancies with a requirement for HDFS skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited HDFS over the 6 months to 17 January 2020 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Hadoop Distributed File System (HDFS)
UK
6 months to
17 Jan 2020		 Same period 2019 Same period 2018
Rank 565 606 506
Rank change year-on-year +41 -100 +72
Contract jobs citing HDFS 189 282 439
As % of all contract IT jobs advertised in the UK 0.26% 0.28% 0.44%
As % of the System Software category 1.67% 1.85% 3.43%
Number of daily rates quoted 110 174 315
UK median daily rate £531 £550 £550
Median daily rate % change year-on-year -3.55% - -
10th Percentile £437 £475 £400
90th Percentile £688 £721 £695
UK excluding London median daily rate £525 £538 £550
% change year-on-year -2.33% -2.27% +15.18%
Number of hourly rates quoted 0 0 0
UK median hourly rate - - -
UK excluding London median hourly rate - - -

HDFS is in the System Software category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for system software skills.

All System Software Skills
UK
Contract vacancies with a requirement for system software skills 11,340 15,230 12,807
As % of all contract IT jobs advertised in the UK 15.57% 15.32% 12.77%
Number of daily rates quoted 7,440 9,973 8,326
UK median daily rate £450 £463 £425
Median daily rate % change year-on-year -2.70% +8.82% +6.12%
10th Percentile £163 £180 £160
90th Percentile £638 £638 £615
UK excluding London median daily rate £413 £413 £375
% change year-on-year - +10.00% +7.14%
Number of hourly rates quoted 676 821 697
UK median hourly rate £17.00 £16.00 £16.41
Median hourly rate % change year-on-year +6.25% -2.47% +2.53%
10th Percentile £11.26 £10.50 £10.40
90th Percentile £48.23 £44.75 £45.00
UK excluding London median hourly rate £16.44 £15.50 £15.00
% change year-on-year +6.03% +3.33% -

HDFS
Job Vacancy Trend

Job postings citing HDFS as a percentage of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing HDFS.

Contractor daily rate trend for HDFS in the UK

HDFS
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing HDFS over the 6 months to 17 January 2020.

Contractor daily rate histogram for HDFS in the UK

HDFS
Contractor Hourly Rate Trend

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing HDFS.

Contractor hourly rate trend for HDFS in the UK

HDFS
Top 14 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing HDFS within the UK over the 6 months to 17 January 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year		 Matching
Contract
IT Job Ads		 Median
Daily Rate
Past 6 Months		 Median Daily Rate
% Change
on Same Period
Last Year		 Live
Job
Vacancies
England +11 151 £550 - 23
UK excluding London +98 112 £525 -2.33% 18
London +33 76 £550 -4.35% 8
South East +40 43 £531 -1.30% 5
Scotland +52 31 £563 -6.25% 1
North of England +12 15 £530 +0.95% 1
South West +27 14 £541 - 3
Yorkshire +50 11 £500 -1.96% 1
Wales - 6 £500 - 2
North West +13 3 £750 +36.36%
East Midlands - 2 £513 - 2
Midlands - 2 £513 - 3
East of England +30 1 £488 - 3
North East +8 1 £800 +52.38%

For the 6 months to 17 January 2020, IT contractor jobs citing HDFS also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for HDFS.

1 165 (87.30%) Hadoop
2 136 (71.96%) Apache Spark
3 125 (66.14%) Big Data
4 121 (64.02%) Apache Hive
5 100 (52.91%) HBase
6 99 (52.38%) Kafka
7 96 (50.79%) SQL
8 88 (46.56%) Scala
9 78 (41.27%) Java
9 78 (41.27%) Python
10 61 (32.28%) Hadoop YARN
11 57 (30.16%) Agile Software Development
12 56 (29.63%) Impala
13 52 (27.51%) Data Modelling
14 51 (26.98%) ETL
15 46 (24.34%) Linux
15 46 (24.34%) NoSQL
16 43 (22.75%) Finance
16 43 (22.75%) Software Engineering
17 42 (22.22%) Cloudera
17 42 (22.22%) Analytics
18 39 (20.63%) AWS
19 37 (19.58%) DevOps
20 36 (19.05%) Spark SQL
21 33 (17.46%) Jenkins
21 33 (17.46%) Continuous Integration
21 33 (17.46%) Hortonworks
22 31 (16.40%) Apache
23 30 (15.87%) Oracle
23 30 (15.87%) Data Architecture

HDFS
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 136 (71.96%) Apache Spark
2 31 (16.40%) Apache
3 14 (7.41%) Elasticsearch
4 11 (5.82%) SAS
5 9 (4.76%) OpenStack
6 8 (4.23%) Apache Pig
6 8 (4.23%) Jupyter
6 8 (4.23%) Tomcat
6 8 (4.23%) WebLogic
7 5 (2.65%) Apache Airflow
8 3 (1.59%) Apache Solr
8 3 (1.59%) Confluence
9 2 (1.06%) Cloud Foundry
10 1 (0.53%) GemFire
10 1 (0.53%) Luigi
10 1 (0.53%) Tornado
Applications
1 6 (3.17%) Microsoft Excel
2 1 (0.53%) Chatbot
2 1 (0.53%) Microsoft Project
Business Applications
1 8 (4.23%) Oracle Applications
2 1 (0.53%) SAP MDM
Cloud Services
1 39 (20.63%) AWS
2 27 (14.29%) Azure
3 22 (11.64%) Amazon Kinesis
4 18 (9.52%) GCP
5 9 (4.76%) Amazon EMR
5 9 (4.76%) Amazon S3
5 9 (4.76%) AWS Lambda
5 9 (4.76%) Serverless
6 8 (4.23%) Power Platform
7 6 (3.17%) GitHub
8 5 (2.65%) Amazon ECS
8 5 (2.65%) PaaS
9 4 (2.12%) Amazon Glacier
10 3 (1.59%) IaaS
11 1 (0.53%) Amazon CloudWatch
11 1 (0.53%) AWS CloudFormation
11 1 (0.53%) AWS CloudTrail
11 1 (0.53%) AWS Step Functions
Communications & Networking
1 13 (6.88%) Kerberos
2 5 (2.65%) SSL
2 5 (2.65%) VPN
3 3 (1.59%) HTTP
4 2 (1.06%) Firewall
4 2 (1.06%) HTTPS
5 1 (0.53%) Internet
Database & Business Intelligence
1 165 (87.30%) Hadoop
2 125 (66.14%) Big Data
3 121 (64.02%) Apache Hive
4 100 (52.91%) HBase
5 56 (29.63%) Impala
6 46 (24.34%) NoSQL
7 36 (19.05%) Spark SQL
8 27 (14.29%) Apache Cassandra
8 27 (14.29%) MongoDB
9 23 (12.17%) SQL Server
10 22 (11.64%) Data Warehouse
11 20 (10.58%) Sqoop
12 16 (8.47%) Data Lake
13 14 (7.41%) Flink
13 14 (7.41%) Oracle Database
14 8 (4.23%) Power BI
14 8 (4.23%) Relational Database
15 7 (3.70%) DB2
16 6 (3.17%) Neo4j
16 6 (3.17%) Oracle ODI
Development Applications
1 33 (17.46%) Jenkins
2 18 (9.52%) Git (software)
3 11 (5.82%) IBM UrbanCode
3 11 (5.82%) Maven
3 11 (5.82%) Sonatype Nexus
4 9 (4.76%) JIRA
5 8 (4.23%) Cucumber
6 7 (3.70%) Selenium
7 6 (3.17%) Jasmine
8 5 (2.65%) Travis CI
9 3 (1.59%) JUnit
10 2 (1.06%) Oracle APEX
11 1 (0.53%) Atlassian Bamboo
11 1 (0.53%) Bitbucket Server
11 1 (0.53%) Eclipse
11 1 (0.53%) ER Studio
11 1 (0.53%) ERwin
11 1 (0.53%) FishEye
11 1 (0.53%) IntelliJ
11 1 (0.53%) kappa
General
1 43 (22.75%) Finance
2 29 (15.34%) Banking
3 14 (7.41%) Investment Banking
4 12 (6.35%) Public Sector
5 8 (4.23%) Retail Banking
6 2 (1.06%) Manufacturing
6 2 (1.06%) Retail
6 2 (1.06%) Telecoms
7 1 (0.53%) Electronics
7 1 (0.53%) Games
7 1 (0.53%) Marketing
Job Titles
1 57 (30.16%) Data Engineer
2 54 (28.57%) Developer
3 29 (15.34%) Big Data Engineer
4 27 (14.29%) Architect
5 23 (12.17%) Data Architect
6 19 (10.05%) Scala Developer
7 15 (7.94%) Big Data Architect
8 14 (7.41%) Administrator
9 13 (6.88%) Java Engineer
10 11 (5.82%) Hadoop Developer
11 10 (5.29%) Hadoop Engineer
12 9 (4.76%) Apache Spark Developer
12 9 (4.76%) Hadoop Architect
13 7 (3.70%) DevOps Engineer
13 7 (3.70%) Java Developer
13 7 (3.70%) Oracle Developer
13 7 (3.70%) Software Engineer
14 6 (3.17%) Java Hadoop Developer
15 5 (2.65%) Data Administrator
15 5 (2.65%) ETL Developer
Libraries, Frameworks & Software Standards
1 99 (52.38%) Kafka
2 17 (8.99%) PySpark
3 12 (6.35%) React
4 11 (5.82%) HTML
5 10 (5.29%) RabbitMQ
5 10 (5.29%) Spring
6 9 (4.76%) JSON
6 9 (4.76%) OAuth
6 9 (4.76%) Spark Streaming
6 9 (4.76%) XML
7 8 (4.23%) XSLT
8 7 (3.70%) RESTful
9 6 (3.17%) Apache NiFi
10 5 (2.65%) Elastic Stack
10 5 (2.65%) Node.js
10 5 (2.65%) OAuth2
10 5 (2.65%) pytest
11 4 (2.12%) LDAP
11 4 (2.12%) SAML
11 4 (2.12%) Spring Boot
Miscellaneous
1 27 (14.29%) Analytical Skills
2 9 (4.76%) Management Information System
2 9 (4.76%) Public Cloud
3 7 (3.70%) Algorithms
3 7 (3.70%) Greenfield Project
3 7 (3.70%) User Experience
4 6 (3.17%) Cloud Native
4 6 (3.17%) Data Structures
4 6 (3.17%) Linux Command Line
5 5 (2.65%) Derivative
6 3 (1.59%) Data Centre
6 3 (1.59%) Replication
7 1 (0.53%) Clustering
7 1 (0.53%) Equities
7 1 (0.53%) Reference Data
7 1 (0.53%) Smart Energy
7 1 (0.53%) Virtual Team
Operating Systems
1 46 (24.34%) Linux
2 28 (14.81%) Unix
3 9 (4.76%) Windows
4 7 (3.70%) Red Hat Enterprise Linux
4 7 (3.70%) Ubuntu
5 4 (2.12%) Oracle Linux
6 2 (1.06%) SUSE Linux Enterprise Server
7 1 (0.53%) Android
7 1 (0.53%) Apple iOS
7 1 (0.53%) CentOS
Processes & Methodologies
1 57 (30.16%) Agile Software Development
2 52 (27.51%) Data Modelling
3 51 (26.98%) ETL
4 43 (22.75%) Software Engineering
5 42 (22.22%) Analytics
6 37 (19.58%) DevOps
7 33 (17.46%) Continuous Integration
8 30 (15.87%) Data Architecture
9 29 (15.34%) Business Intelligence
10 28 (14.81%) Data Ingestion
11 26 (13.76%) Data Development
11 26 (13.76%) Use Case
12 24 (12.70%) Data Engineering
12 24 (12.70%) Data Science
12 24 (12.70%) TDD
13 23 (12.17%) Migration
14 22 (11.64%) Data Analytics
14 22 (11.64%) Scrum
15 21 (11.11%) Performance Tuning
15 21 (11.11%) Problem-Solving
Programming Languages
1 96 (50.79%) SQL
2 88 (46.56%) Scala
3 78 (41.27%) Java
3 78 (41.27%) Python
4 26 (13.76%) R
5 21 (11.11%) Go
5 21 (11.11%) JavaScript
6 20 (10.58%) Bash Shell
7 16 (8.47%) Kotlin
7 16 (8.47%) PL/SQL
8 12 (6.35%) Perl
9 10 (5.29%) Ruby
10 9 (4.76%) C
11 8 (4.23%) Shell Script
12 6 (3.17%) C#
13 2 (1.06%) HiveQL
14 1 (0.53%) Java 8
14 1 (0.53%) PHP
Qualifications
1 19 (10.05%) Security Cleared
2 16 (8.47%) SC Cleared
3 12 (6.35%) Degree
4 10 (5.29%) Computer Science Degree
5 3 (1.59%) BPSS Clearance
6 2 (1.06%) ISTQB
7 1 (0.53%) Microsoft Certification
Quality Assurance & Compliance
1 9 (4.76%) Data Quality
2 3 (1.59%) QA
3 1 (0.53%) Disclosure Scotland
System Software
1 8 (4.23%) Apache ZooKeeper
2 7 (3.70%) Docker
3 6 (3.17%) Apache Flume
4 2 (1.06%) Active Directory
Systems Management
1 61 (32.28%) Hadoop YARN
2 20 (10.58%) Kubernetes
3 17 (8.99%) Oozie
4 10 (5.29%) Ansible
5 7 (3.70%) Ambari
6 6 (3.17%) Grafana
6 6 (3.17%) Kibana
6 6 (3.17%) Puppet
7 5 (2.65%) Terraform
8 4 (2.12%) Opscode Chef
8 4 (2.12%) Single Sign-On
9 3 (1.59%) logstash
10 2 (1.06%) Control-M
Vendors
1 42 (22.22%) Cloudera
2 33 (17.46%) Hortonworks
3 30 (15.87%) Oracle
4 23 (12.17%) Tableau
5 11 (5.82%) Microsoft
5 11 (5.82%) Teradata
6 10 (5.29%) Talend
7 7 (3.70%) Ab Initio
7 7 (3.70%) Foundry
7 7 (3.70%) Google
7 7 (3.70%) Palantir
8 6 (3.17%) Splunk
9 5 (2.65%) Sybase
10 4 (2.12%) MapR
11 3 (1.59%) Informatica
12 2 (1.06%) Red Hat
12 2 (1.06%) SAP
12 2 (1.06%) StreamSets
13 1 (0.53%) Atlassian
13 1 (0.53%) Confluent