Period
to 5 August 2020

The following table provides summary statistics for contract job vacancies with a requirement for Hadoop skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Hadoop over the 6 months to 5 August 2020 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Hadoop
UK
6 months to
5 Aug 2020
Same period 2019 Same period 2018
Rank 178 153 125
Rank change year-on-year -25 -28 -8
Contract jobs citing Hadoop 521 1,660 2,196
As % of all contract jobs advertised in the UK 1.46% 1.82% 2.14%
As % of the Database & Business Intelligence category 6.77% 8.26% 9.37%
Number of daily rates quoted 371 1,154 1,570
Median daily rate £540 £550 £550
Median daily rate % change year-on-year -1.82% - +5.77%
10th Percentile £393 £413 £400
90th Percentile £713 £700 £688
UK excluding London median daily rate £513 £500 £500
% change year-on-year +2.50% - +11.11%
Number of hourly rates quoted 11 17 7
Median hourly rate £35.00 £49.13 £47.50
Median hourly rate % change year-on-year -28.76% +3.43% +35.71%
UK excluding London median hourly rate £35.00 £52.00 £47.50
% change year-on-year -32.69% +9.47% +35.71%

Hadoop is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Contract vacancies with a requirement for database or business intelligence skills 7,701 20,097 23,428
As % of all contract IT jobs advertised in the UK 21.62% 22.06% 22.80%
Number of daily rates quoted 4,980 13,760 16,339
Median daily rate £500 £475 £463
Median daily rate % change year-on-year +5.26% +2.70% +2.78%
10th Percentile £325 £325 £313
90th Percentile £688 £650 £636
UK excluding London median daily rate £450 £435 £415
% change year-on-year +3.45% +4.82% +3.75%
Number of hourly rates quoted 161 260 239
Median hourly rate £35.00 £35.00 £32.00
Median hourly rate % change year-on-year - +9.38% +3.23%
10th Percentile £15.00 £14.07 £13.64
90th Percentile £53.75 £60.00 £55.00
Median hourly rate £35.00 £35.00 £33.50
% change year-on-year - +4.48% -1.47%

Hadoop
Job Vacancy Trend

Job postings citing Hadoop as a proportion of all IT jobs advertised.

Job vacancy trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Trend

3-month moving average daily rate quoted in jobs citing Hadoop.

Daily rate trend for Hadoop in the UK

Hadoop
Daily Rate Histogram

Daily rate distribution for jobs citing Hadoop over the 6 months to 5 August 2020.

Daily rate histogram for Hadoop in the UK

Hadoop
Contractor Hourly Rate Trend

3-month moving average hourly rates quoted in jobs citing Hadoop.

Hourly rate trend for Hadoop in the UK

Hadoop
Top 15 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Hadoop within the UK over the 6 months to 5 August 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -31 481 £550 - 118
London -40 253 £550 -4.35% 68
UK excluding London -4 238 £513 +2.50% 71
North of England +24 96 £525 +5.00% 16
North West +60 76 £525 - 6
South East +42 50 £500 - 11
Work from Home -12 43 £540 -1.82% 21
South West +47 30 £650 +23.81% 3
Midlands +71 28 £432 -17.67% 17
West Midlands +69 24 £432 -24.83% 1
Wales +56 17 £500 -4.76%
Yorkshire -17 17 £475 -5.00% 9
East of England +41 14 £450 -23.40% 3
Scotland -20 8 £475 +5.56% 21
East Midlands +38 3 - - 16

For the 6 months to 5 August 2020, IT contractor jobs citing Hadoop also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for Hadoop.

1 288 (55.28%) Python
2 284 (54.51%) Big Data
3 258 (49.52%) Apache Spark
4 192 (36.85%) AWS
5 186 (35.70%) DevOps
6 179 (34.36%) SQL
7 161 (30.90%) Cloudera
8 156 (29.94%) Java
9 152 (29.17%) Analytics
10 149 (28.60%) Agile Software Development
11 146 (28.02%) Azure
12 139 (26.68%) Apache Hive
13 126 (24.18%) Jenkins
14 123 (23.61%) Security Cleared
15 107 (20.54%) Kafka
15 107 (20.54%) Scala
15 107 (20.54%) Docker
16 106 (20.35%) ETL
17 98 (18.81%) SC Cleared
18 97 (18.62%) Kubernetes
19 92 (17.66%) Terraform
20 83 (15.93%) Ansible
21 82 (15.74%) Git (software)
22 81 (15.55%) Hortonworks
23 80 (15.36%) Data Modelling
23 80 (15.36%) Linux
24 75 (14.40%) Data Science
25 73 (14.01%) Scrum
26 72 (13.82%) Oracle
27 69 (13.24%) Finance

Hadoop
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 258 (49.52%) Apache Spark
2 50 (9.60%) Apache
3 37 (7.10%) Confluence
3 37 (7.10%) SAS
4 18 (3.45%) Elasticsearch
5 14 (2.69%) Apache Airflow
5 14 (2.69%) Apache Pig
5 14 (2.69%) Mule
5 14 (2.69%) WebLogic
6 11 (2.11%) Jupyter
7 10 (1.92%) Tomcat
8 6 (1.15%) Cloud Foundry
8 6 (1.15%) Jetty
9 5 (0.96%) JBoss
9 5 (0.96%) MQSeries
9 5 (0.96%) Oracle Self-Service HR
9 5 (0.96%) Red Hat Fuse
10 2 (0.38%) OpenStack
11 1 (0.19%) Apache Solr
11 1 (0.19%) Luigi
Applications
1 9 (1.73%) Microsoft Excel
2 6 (1.15%) Spreadsheet
3 2 (0.38%) SPSS
4 1 (0.19%) Microsoft PowerPoint
4 1 (0.19%) Stata
Business Applications
1 8 (1.54%) SAP FI-CA
2 6 (1.15%) Sparx Enterprise Architect
3 2 (0.38%) SAP Oil and Gas
4 1 (0.19%) FINCAD Analytics
4 1 (0.19%) SAP CRM
4 1 (0.19%) SAP ERP
4 1 (0.19%) SAP IS-U
4 1 (0.19%) Shopify
Cloud Services
1 192 (36.85%) AWS
2 146 (28.02%) Azure
3 60 (11.52%) GCP
4 31 (5.95%) Power Platform
5 26 (4.99%) PaaS
6 15 (2.88%) Serverless
7 14 (2.69%) GitHub
8 13 (2.50%) Amazon S3
8 13 (2.50%) Cloud Computing
9 12 (2.30%) IaaS
10 11 (2.11%) AWS Lambda
11 9 (1.73%) Amazon EMR
11 9 (1.73%) Amazon Kinesis
12 7 (1.34%) AWS CloudFormation
12 7 (1.34%) Azure Data Factory
13 6 (1.15%) Slack
14 5 (0.96%) npm
15 4 (0.77%) Virtual Private Cloud
16 3 (0.58%) Amazon SQS
16 3 (0.58%) SaaS
Communications & Networking
1 10 (1.92%) Kerberos
2 9 (1.73%) SSL
3 5 (0.96%) SSH
4 4 (0.77%) 5G
4 4 (0.77%) HTTP
5 3 (0.58%) DNS
6 2 (0.38%) Internet
6 2 (0.38%) TCP/IP
7 1 (0.19%) Broadband
7 1 (0.19%) HTTPS
7 1 (0.19%) MQTT
7 1 (0.19%) Network Security
7 1 (0.19%) Skype
7 1 (0.19%) VPN
7 1 (0.19%) Wi-Fi
Database & Business Intelligence
1 284 (54.51%) Big Data
2 139 (26.68%) Apache Hive
3 68 (13.05%) Impala
4 57 (10.94%) Data Warehouse
5 56 (10.75%) MongoDB
6 53 (10.17%) HBase
7 52 (9.98%) NoSQL
8 47 (9.02%) Data Lake
8 47 (9.02%) Relational Database
9 41 (7.87%) SQL Server
10 35 (6.72%) Apache Cassandra
11 31 (5.95%) Power BI
12 29 (5.57%) MySQL
13 26 (4.99%) Amazon Redshift
14 25 (4.80%) Oracle Database
15 24 (4.61%) BigQuery
16 20 (3.84%) SQL Server Integration Services
17 17 (3.26%) Data Mining
17 17 (3.26%) Pentaho
18 16 (3.07%) PostgreSQL
Development Applications
1 126 (24.18%) Jenkins
2 82 (15.74%) Git (software)
3 44 (8.45%) JIRA
4 37 (7.10%) Maven
5 30 (5.76%) Bitbucket
6 29 (5.57%) GitLab
7 20 (3.84%) Artifactory
7 20 (3.84%) TeamCity
8 19 (3.65%) Selenium
9 15 (2.88%) Cucumber
10 12 (2.30%) Sonatype Nexus
11 11 (2.11%) Visual Studio
12 9 (1.73%) JUnit
12 9 (1.73%) Team Foundation Server
13 8 (1.54%) Postman
13 8 (1.54%) SoapUI
13 8 (1.54%) Toad
13 8 (1.54%) Visual Studio Team System
14 7 (1.34%) Atlassian Bamboo
14 7 (1.34%) Subversion
General
1 69 (13.24%) Finance
2 45 (8.64%) Banking
3 28 (5.37%) Retail
3 28 (5.37%) Telecoms
4 25 (4.80%) Public Sector
5 22 (4.22%) Marketing
6 19 (3.65%) Front Office
7 18 (3.45%) Aerospace
8 15 (2.88%) Investment Banking
9 11 (2.11%) Legal
10 7 (1.34%) Manufacturing
11 5 (0.96%) Law
11 5 (0.96%) Pharmaceutical
12 3 (0.58%) Games
12 3 (0.58%) Retail Banking
13 2 (0.38%) Electronics
13 2 (0.38%) Financial Institution
14 1 (0.19%) Billing
14 1 (0.19%) Humanities
14 1 (0.19%) Social Science
Job Titles
1 146 (28.02%) Data Engineer
2 102 (19.58%) Developer
3 67 (12.86%) Architect
4 63 (12.09%) DevOps Engineer
5 55 (10.56%) Big Data Engineer
6 47 (9.02%) Python Engineer
7 36 (6.91%) Data Architect
7 36 (6.91%) Hadoop Engineer
8 34 (6.53%) Administrator
8 34 (6.53%) Cloud Engineer
9 30 (5.76%) Java Developer
10 25 (4.80%) Data Scientist
11 23 (4.41%) Analyst
11 23 (4.41%) Azure Engineer
11 23 (4.41%) Python Developer
12 22 (4.22%) AWS Engineer
13 19 (3.65%) Cloud Architect
14 18 (3.45%) Big Data Developer
15 17 (3.26%) Cloud Administrator
15 17 (3.26%) Systems Administrator
Libraries, Frameworks & Software Standards
1 107 (20.54%) Kafka
2 64 (12.28%) PySpark
3 41 (7.87%) Web Services
4 38 (7.29%) TensorFlow
5 32 (6.14%) Spring
6 30 (5.76%) scikit-learn
7 25 (4.80%) JSON
7 25 (4.80%) XML
8 21 (4.03%) REST
9 20 (3.84%) Apache NiFi
9 20 (3.84%) Spring Boot
10 17 (3.26%) Node.js
11 16 (3.07%) .NET
11 16 (3.07%) React
12 15 (2.88%) AngularJS
12 15 (2.88%) Natural Language Processing
12 15 (2.88%) RabbitMQ
13 14 (2.69%) H2O
13 14 (2.69%) Spark Streaming
14 11 (2.11%) HTML
Miscellaneous
1 49 (9.40%) Analytical Skills
2 33 (6.33%) Algorithms
3 19 (3.65%) Greenfield Project
4 15 (2.88%) PKI
5 13 (2.50%) Cloud Native
6 12 (2.30%) Credit Risk
7 11 (2.11%) Derivative
7 11 (2.11%) Self-Motivation
8 10 (1.92%) Public Cloud
9 9 (1.73%) Data Structures
9 9 (1.73%) FMCG
9 9 (1.73%) Management Information System
9 9 (1.73%) Reference Data
10 8 (1.54%) Fixed Income
10 8 (1.54%) User Experience
11 7 (1.34%) Analytical Mindset
11 7 (1.34%) Enterprise Software
11 7 (1.34%) Hedge funds
12 5 (0.96%) Equities
13 4 (0.77%) PMI
Operating Systems
1 80 (15.36%) Linux
2 32 (6.14%) Unix
3 27 (5.18%) Windows
4 22 (4.22%) CentOS
5 21 (4.03%) Red Hat Enterprise Linux
6 5 (0.96%) Android
6 5 (0.96%) Apple iOS
6 5 (0.96%) Debian
6 5 (0.96%) Mac OS X
6 5 (0.96%) Ubuntu
7 1 (0.19%) DC/OS
7 1 (0.19%) Windows Server
Processes & Methodologies
1 186 (35.70%) DevOps
2 152 (29.17%) Analytics
3 149 (28.60%) Agile Software Development
4 106 (20.35%) ETL
5 80 (15.36%) Data Modelling
6 75 (14.40%) Data Science
7 73 (14.01%) Scrum
8 68 (13.05%) Business Intelligence
9 62 (11.90%) Data Engineering
10 60 (11.52%) Machine Learning
11 56 (10.75%) Data Analytics
12 50 (9.60%) Visualisation
13 48 (9.21%) Microservices
14 47 (9.02%) Software Engineering
15 45 (8.64%) TDD
16 41 (7.87%) Continuous Integration
17 37 (7.10%) Data Visualisation
18 36 (6.91%) Test Automation
19 33 (6.33%) Computer Science
20 32 (6.14%) Data Management
Programming Languages
1 288 (55.28%) Python
2 179 (34.36%) SQL
3 156 (29.94%) Java
4 107 (20.54%) Scala
5 42 (8.06%) R
6 39 (7.49%) Bash Shell
7 33 (6.33%) PL/SQL
7 33 (6.33%) Shell Script
8 26 (4.99%) JavaScript
9 24 (4.61%) Ruby
10 20 (3.84%) C#
10 20 (3.84%) Perl
11 18 (3.45%) PowerShell
12 17 (3.26%) C++
13 14 (2.69%) T-SQL
14 12 (2.30%) Java 8
15 11 (2.11%) Groovy
16 8 (1.54%) C
17 7 (1.34%) Go
18 6 (1.15%) VBA
Qualifications
1 123 (23.61%) Security Cleared
2 98 (18.81%) SC Cleared
3 37 (7.10%) Degree
4 24 (4.61%) DV Cleared
5 23 (4.41%) CCA
6 16 (3.07%) Numerate Degree
7 10 (1.92%) Computer Science Degree
8 7 (1.34%) Master's Degree
9 5 (0.96%) AWS Certification
10 4 (0.77%) BPSS Clearance
10 4 (0.77%) ISEB
10 4 (0.77%) ISTQB
10 4 (0.77%) ISTQB Foundation Certification
10 4 (0.77%) PMI Certification
11 2 (0.38%) PhD
12 1 (0.19%) CSPO
12 1 (0.19%) DBS Check
12 1 (0.19%) MBA
Quality Assurance & Compliance
1 12 (2.30%) Data Quality
2 4 (0.77%) SLA
3 3 (0.58%) QA
4 1 (0.19%) CDISC
4 1 (0.19%) COBIT
4 1 (0.19%) GDPR
4 1 (0.19%) IFRS
4 1 (0.19%) ISO/IEC 27002 (supersedes ISO/IEC 17799)
4 1 (0.19%) MiFID
System Software
1 107 (20.54%) Docker
2 63 (12.09%) HDFS
3 22 (4.22%) Apache ZooKeeper
4 20 (3.84%) Active Directory
5 13 (2.50%) Apache Flume
6 11 (2.11%) Virtual Machines
7 3 (0.58%) NFS
8 1 (0.19%) VMware Infrastructure
Systems Management
1 97 (18.62%) Kubernetes
2 92 (17.66%) Terraform
3 83 (15.93%) Ansible
4 40 (7.68%) Puppet
5 17 (3.26%) Oozie
6 12 (2.30%) Hadoop YARN
6 12 (2.30%) Packer
7 5 (0.96%) Opscode Chef
7 5 (0.96%) Salt
8 4 (0.77%) logstash
9 3 (0.58%) Ambari
9 3 (0.58%) Grafana
9 3 (0.58%) Kubeflow
10 2 (0.38%) Netcool
10 2 (0.38%) Rancher
11 1 (0.19%) Control-M
11 1 (0.19%) Kibana
11 1 (0.19%) Marathon
11 1 (0.19%) Mesos
11 1 (0.19%) SCCM
Vendors
1 161 (30.90%) Cloudera
2 81 (15.55%) Hortonworks
3 72 (13.82%) Oracle
4 44 (8.45%) Microsoft
4 44 (8.45%) Tableau
5 30 (5.76%) Teradata
6 26 (4.99%) Databricks
7 24 (4.61%) Informatica
8 22 (4.22%) Red Hat
9 16 (3.07%) Google
9 16 (3.07%) SAP
10 15 (2.88%) VMware
11 13 (2.50%) Splunk
12 10 (1.92%) MapR
13 9 (1.73%) IBM
14 8 (1.54%) Sybase
15 7 (1.34%) MuleSoft
16 6 (1.15%) AppDynamics
16 6 (1.15%) Sparx
17 5 (0.96%) StreamSets