Period
to 25 July 2021

The following table provides summary statistics for contract job vacancies with a requirement for Hadoop skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Hadoop over the 6 months to 25 July 2021 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Hadoop
UK
6 months to
25 Jul 2021
Same period 2020 Same period 2019
Rank 189 187 148
Rank change year-on-year -2 -39 -28
Contract jobs citing Hadoop 999 537 1,733
As % of all contract jobs advertised in the UK 1.41% 1.43% 1.89%
As % of the Database & Business Intelligence category 6.63% 6.49% 8.33%
Number of daily rates quoted 570 383 1,185
10th Percentile £413 £400 £413
25th Percentile £475 £450 £475
Median daily rate (50th Percentile) £553 £550 £550
Median % change year-on-year +0.45% - -
75th Percentile £638 £650 £630
90th Percentile £718 £725 £700
UK excluding London median daily rate £508 £521 £500
% change year-on-year -2.64% +4.25% -
Number of hourly rates quoted 13 0 11
10th Percentile £19.55 " £32.50
25th Percentile £24.30 " £44.86
Median hourly rate £26.00 - £49.86
75th Percentile £27.73 " £51.15
90th Percentile £45.20 " £70.00
UK excluding London median hourly rate £25.75 - £69.00

Hadoop is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Contract vacancies with a requirement for database or business intelligence skills 15,072 8,278 20,804
As % of all contract IT jobs advertised in the UK 21.30% 22.06% 22.65%
Number of daily rates quoted 10,344 5,730 14,380
10th Percentile £338 £325 £325
25th Percentile £413 £406 £400
Median daily rate (50th Percentile) £500 £500 £475
Median % change year-on-year - +5.26% +2.70%
75th Percentile £611 £600 £575
90th Percentile £700 £688 £650
UK excluding London median daily rate £450 £450 £433
% change year-on-year - +3.93% +4.40%
Number of hourly rates quoted 217 114 257
10th Percentile £11.71 £12.63 £13.45
25th Percentile £19.00 £17.77 £17.63
Median hourly rate £27.00 £22.50 £30.00
Median % change year-on-year +20.00% -25.00% -
75th Percentile £45.18 £43.12 £50.00
90th Percentile £62.00 £63.79 £60.00
UK excluding London median hourly rate £26.00 £21.57 £30.00
% change year-on-year +20.54% -28.10% +4.35%

Hadoop
Job Vacancy Trend

Job postings citing Hadoop as a proportion of all IT jobs advertised.

Job vacancy trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Trend

3-month moving average daily rate quoted in jobs citing Hadoop.

Daily rate trend for Hadoop in the UK

Hadoop
Daily Rate Histogram

Daily rate distribution for jobs citing Hadoop over the 6 months to 25 July 2021.

Daily rate histogram for Hadoop in the UK

Hadoop
Contractor Hourly Rate Trend

3-month moving average hourly rates quoted in jobs citing Hadoop.

Hourly rate trend for Hadoop in the UK

Hadoop
Hourly Rate Histogram

Hourly rate distribution of jobs citing Hadoop over the 6 months to 25 July 2021.

Hourly rate histogram for Hadoop in the UK

Hadoop
Top 17 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Hadoop within the UK over the 6 months to 25 July 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -6 819 £550 - 160
London -15 494 £580 +5.45% 97
UK excluding London -16 310 £508 -2.64% 75
Work from Home -70 269 £575 +6.48% 26
South East -7 82 £543 +8.50% 19
Midlands +14 77 £490 +12.64% 11
West Midlands +16 69 £475 +9.20% 7
Scotland -18 49 £475 -9.52% 5
North of England -56 48 £565 +2.73% 23
South West -17 24 £508 -21.92% 5
Yorkshire -12 17 £542 +14.05% 10
East of England -15 17 £575 +34.50% 4
North West -74 16 £600 +5.49% 9
North East - 16 £608 - 4
Wales -17 13 £538 +10.26% 2
East Midlands -9 6 £530 - 4
Northern Ireland - 1 £475 - 6

For the 6 months to 25 July 2021, IT contractor jobs citing Hadoop also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for Hadoop.

1 509 (50.95%) Apache Spark
2 471 (47.15%) Big Data
3 443 (44.34%) Python
4 412 (41.24%) SQL
5 366 (36.64%) AWS
6 365 (36.54%) Java
7 344 (34.43%) Apache Hive
8 324 (32.43%) Agile Software Development
9 283 (28.33%) Kafka
10 274 (27.43%) DevOps
11 269 (26.93%) Scala
12 267 (26.73%) Analytics
13 256 (25.63%) Azure
14 224 (22.42%) ETL
15 215 (21.52%) Finance
16 191 (19.12%) Social Skills
17 189 (18.92%) Cloudera
18 183 (18.32%) Security Cleared
19 177 (17.72%) Data Modelling
20 172 (17.22%) HDFS
21 168 (16.82%) Docker
22 167 (16.72%) Kubernetes
22 167 (16.72%) Jenkins
23 163 (16.32%) Oracle
24 159 (15.92%) Scrum
25 158 (15.82%) Linux
26 149 (14.91%) HBase
27 147 (14.71%) Git (software)
28 146 (14.61%) Unix
28 146 (14.61%) Data Warehouse

Hadoop
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 509 (50.95%) Apache Spark
2 100 (10.01%) Apache
3 82 (8.21%) Confluence
4 66 (6.61%) Apache Airflow
5 56 (5.61%) Elasticsearch
6 55 (5.51%) SAS
7 23 (2.30%) Luigi
8 13 (1.30%) Apache Pig
9 12 (1.20%) Tomcat
10 9 (0.90%) WebSphere
11 8 (0.80%) Apache Solr
12 7 (0.70%) nginx
12 7 (0.70%) OpenStack
13 6 (0.60%) Apache Pulsar
13 6 (0.60%) JBoss
13 6 (0.60%) SharePoint
13 6 (0.60%) WebSphere Application Server
14 3 (0.30%) Cloud Foundry
14 3 (0.30%) Jetty
15 2 (0.20%) Jupyter
Applications
1 15 (1.50%) Spreadsheet
2 14 (1.40%) Microsoft Excel
3 6 (0.60%) SPSS
4 3 (0.30%) Microsoft Office
Business Applications
1 2 (0.20%) Oracle BPM
2 1 (0.10%) Distributed Ledger
2 1 (0.10%) Oracle Cash Management
2 1 (0.10%) SAP FI-CA
2 1 (0.10%) SAP MDG
2 1 (0.10%) SAP MDM
2 1 (0.10%) SAP PSCD
2 1 (0.10%) Temenos T24
2 1 (0.10%) Traffica
Cloud Services
1 366 (36.64%) AWS
2 256 (25.63%) Azure
3 123 (12.31%) GCP
4 72 (7.21%) Power Platform
5 62 (6.21%) Serverless
6 59 (5.91%) Amazon S3
7 51 (5.11%) GitHub
8 49 (4.90%) Amazon EMR
9 45 (4.50%) Azure Cosmos DB
9 45 (4.50%) PaaS
10 37 (3.70%) Azure Data Factory
10 37 (3.70%) IaaS
11 36 (3.60%) Amazon EC2
12 32 (3.20%) AWS Lambda
13 31 (3.10%) Snowflake
14 27 (2.70%) Route 53
15 25 (2.50%) Amazon ECS
15 25 (2.50%) Azure Functions
16 24 (2.40%) OpenShift
17 21 (2.10%) Amazon EKS
Communications & Networking
1 18 (1.80%) Firewall
1 18 (1.80%) Kerberos
2 14 (1.40%) DNS
3 8 (0.80%) WAN
4 7 (0.70%) NAS
4 7 (0.70%) SAN
4 7 (0.70%) SSL
5 6 (0.60%) Cisco Nexus
5 6 (0.60%) FCoE
5 6 (0.60%) Fibre Channel
5 6 (0.60%) HTTP
5 6 (0.60%) iSCSI
5 6 (0.60%) LAN
5 6 (0.60%) Multicast
6 5 (0.50%) FTP
6 5 (0.50%) Internet
7 4 (0.40%) HTTPS
7 4 (0.40%) VSAN
8 2 (0.20%) Ethernet
8 2 (0.20%) VLAN
Database & Business Intelligence
1 471 (47.15%) Big Data
2 344 (34.43%) Apache Hive
3 149 (14.91%) HBase
4 146 (14.61%) Data Warehouse
5 134 (13.41%) Relational Database
6 125 (12.51%) Data Lake
7 119 (11.91%) PostgreSQL
8 108 (10.81%) NoSQL
9 100 (10.01%) Impala
10 98 (9.81%) MongoDB
11 83 (8.31%) SQL Server
12 77 (7.71%) MySQL
13 73 (7.31%) Tableau
14 72 (7.21%) Power BI
15 63 (6.31%) Apache Cassandra
16 62 (6.21%) Pentaho
17 59 (5.91%) Metadata
18 47 (4.70%) Spark SQL
19 44 (4.40%) Amazon Redshift
19 44 (4.40%) RDBMS
Development Applications
1 167 (16.72%) Jenkins
2 147 (14.71%) Git (software)
3 104 (10.41%) JIRA
4 45 (4.50%) GitLab
5 43 (4.30%) Bitbucket
6 40 (4.00%) Sonatype Nexus
7 31 (3.10%) Maven
8 24 (2.40%) Artifactory
9 21 (2.10%) TeamCity
10 17 (1.70%) IBM UrbanCode
11 15 (1.50%) Cucumber
12 12 (1.20%) Selenium
12 12 (1.20%) Subversion
13 11 (1.10%) Mockito
14 10 (1.00%) Gradle
14 10 (1.00%) JUnit
15 9 (0.90%) ERwin
16 7 (0.70%) Protractor
17 6 (0.60%) Visual Studio
17 6 (0.60%) WebDriver
General
1 215 (21.52%) Finance
2 191 (19.12%) Social Skills
3 137 (13.71%) Banking
4 93 (9.31%) Analytical Skills
5 58 (5.81%) Public Sector
6 40 (4.00%) Presentation Skills
7 28 (2.80%) Documentation Skills
8 25 (2.50%) Investment Banking
9 20 (2.00%) Marketing
10 19 (1.90%) Retail
11 12 (1.20%) Front Office
12 11 (1.10%) Telecoms
13 9 (0.90%) Games
13 9 (0.90%) Legal
14 8 (0.80%) Financial Institution
15 7 (0.70%) Manufacturing
16 6 (0.60%) Electronics
17 5 (0.50%) French Language
17 5 (0.50%) Influencing Skills
18 4 (0.40%) Retail Banking
Job Titles
1 265 (26.53%) Data Engineer
2 215 (21.52%) Developer
3 126 (12.61%) Architect
4 72 (7.21%) DevOps Engineer
5 67 (6.71%) AWS Engineer
6 62 (6.21%) Data Architect
7 61 (6.11%) Big Data Engineer
8 50 (5.01%) Scala Developer
9 48 (4.80%) Consultant
10 45 (4.50%) Hadoop Developer
11 43 (4.30%) Hadoop Engineer
12 42 (4.20%) Java Developer
13 39 (3.90%) Data Scientist
14 38 (3.80%) Cloud Architect
15 37 (3.70%) Azure Engineer
16 35 (3.50%) AWS DevOps
16 35 (3.50%) Big Data Developer
16 35 (3.50%) Solutions Architect
17 33 (3.30%) Apache Spark Developer
17 33 (3.30%) Cloud Engineer
Libraries, Frameworks & Software Standards
1 283 (28.33%) Kafka
2 95 (9.51%) PySpark
3 67 (6.71%) Spring
4 49 (4.90%) Spark Streaming
5 42 (4.20%) JSON
6 38 (3.80%) Apache NiFi
6 38 (3.80%) Web Services
7 36 (3.60%) Spring Boot
8 34 (3.40%) RESTful
9 23 (2.30%) Akka
10 20 (2.00%) Apache Avro
10 20 (2.00%) OAuth
10 20 (2.00%) REST
10 20 (2.00%) TensorFlow
11 19 (1.90%) .NET
12 17 (1.70%) AngularJS
12 17 (1.70%) HTML
13 15 (1.50%) Pandas
13 15 (1.50%) SAML
13 15 (1.50%) XML
Miscellaneous
1 77 (7.71%) Data Structures
2 29 (2.90%) Linux Command Line
3 22 (2.20%) Algorithms
3 22 (2.20%) Data Protection Act
3 22 (2.20%) Public Cloud
4 19 (1.90%) Self-Motivation
4 19 (1.90%) Social Media
5 16 (1.60%) Data Centre
5 16 (1.60%) Management Information System
6 13 (1.30%) Greenfield Project
6 13 (1.30%) Hybrid Cloud
6 13 (1.30%) PKI
6 13 (1.30%) Virtual Team
7 12 (1.20%) Cloud Native
7 12 (1.20%) Mainframe
8 10 (1.00%) Video Conferencing
9 9 (0.90%) Analytical Mindset
10 8 (0.80%) Enterprise Software
10 8 (0.80%) iPad
10 8 (0.80%) Replication
Operating Systems
1 158 (15.82%) Linux
2 146 (14.61%) Unix
3 51 (5.11%) Windows
4 11 (1.10%) CentOS
5 9 (0.90%) Red Hat Enterprise Linux
6 8 (0.80%) Ubuntu
7 7 (0.70%) Solaris
8 5 (0.50%) Debian
8 5 (0.50%) VMS
9 3 (0.30%) CoreOS
10 2 (0.20%) Windows 10
10 2 (0.20%) Windows Server
11 1 (0.10%) AIX
11 1 (0.10%) Android
11 1 (0.10%) Apple iOS
11 1 (0.10%) Mac OS X
11 1 (0.10%) MVS
11 1 (0.10%) SUSE
Processes & Methodologies
1 324 (32.43%) Agile Software Development
2 274 (27.43%) DevOps
3 267 (26.73%) Analytics
4 224 (22.42%) ETL
5 177 (17.72%) Data Modelling
6 159 (15.92%) Scrum
7 146 (14.61%) Business Intelligence
8 139 (13.91%) Data Engineering
9 137 (13.71%) Data Pipeline
10 118 (11.81%) Data Science
11 106 (10.61%) Computer Science
12 102 (10.21%) Migration
13 100 (10.01%) Software Engineering
14 95 (9.51%) Machine Learning
15 88 (8.81%) Data Analytics
16 86 (8.61%) Microservices
17 78 (7.81%) Problem-Solving
18 69 (6.91%) Visualisation
19 68 (6.81%) Configuration Management
19 68 (6.81%) Data Architecture
Programming Languages
1 443 (44.34%) Python
2 412 (41.24%) SQL
3 365 (36.54%) Java
4 269 (26.93%) Scala
5 65 (6.51%) R
5 65 (6.51%) Shell Script
6 40 (4.00%) PL/SQL
7 31 (3.10%) JavaScript
8 28 (2.80%) HiveQL
9 25 (2.50%) Bash Shell
9 25 (2.50%) Perl
10 23 (2.30%) Groovy
11 22 (2.20%) C++
12 17 (1.70%) C#
12 17 (1.70%) PowerShell
13 16 (1.60%) Go
14 15 (1.50%) VBA
15 11 (1.10%) C
16 10 (1.00%) CQL
16 10 (1.00%) Ruby
Qualifications
1 183 (18.32%) Security Cleared
2 135 (13.51%) SC Cleared
3 94 (9.41%) Degree
4 47 (4.70%) Computer Science Degree
5 20 (2.00%) DV Cleared
6 12 (1.20%) SAS Certification
7 11 (1.10%) BPSS Clearance
8 7 (0.70%) Numerate Degree
8 7 (0.70%) PhD
9 6 (0.60%) CCA
10 4 (0.40%) Master's Degree
11 2 (0.20%) (ISC)2 CCSP
11 2 (0.20%) AWS Certification
11 2 (0.20%) Cisco Certification
11 2 (0.20%) CISM
11 2 (0.20%) CISSP
11 2 (0.20%) PMI Certification
12 1 (0.10%) CBAP
12 1 (0.10%) ISEB
12 1 (0.10%) ISTQB
Quality Assurance & Compliance
1 54 (5.41%) Data Quality
2 47 (4.70%) QA
3 13 (1.30%) SLA
4 10 (1.00%) GDPR
5 6 (0.60%) Disclosure Scotland
6 3 (0.30%) MiFID
7 2 (0.20%) COBIT
7 2 (0.20%) ISO/IEC 27001
7 2 (0.20%) NIST
7 2 (0.20%) Sarbanes-Oxley
8 1 (0.10%) Basel II
8 1 (0.10%) Dodd-Frank
8 1 (0.10%) EMIR
8 1 (0.10%) ISO 20022
8 1 (0.10%) PMO
8 1 (0.10%) PSD2
8 1 (0.10%) TARGET2
System Software
1 172 (17.22%) HDFS
2 168 (16.82%) Docker
3 26 (2.60%) Apache ZooKeeper
4 20 (2.00%) Active Directory
4 20 (2.00%) Apache Flume
5 15 (1.50%) VMware Infrastructure
6 9 (0.90%) Oracle RAC
7 8 (0.80%) VMware NSX
8 6 (0.60%) Hyper-V
8 6 (0.60%) KVM
8 6 (0.60%) NFS
8 6 (0.60%) SRDF
8 6 (0.60%) Xen
9 4 (0.40%) vSphere
10 2 (0.20%) VMware ESXi
11 1 (0.10%) Virtual Machines
Systems Management
1 167 (16.72%) Kubernetes
2 132 (13.21%) Terraform
3 121 (12.11%) Ansible
4 55 (5.51%) Oozie
5 47 (4.70%) Hadoop YARN
6 38 (3.80%) Puppet
7 33 (3.30%) Prometheus
8 30 (3.00%) Grafana
9 20 (2.00%) Ambari
10 17 (1.70%) Rancher
11 15 (1.50%) Control-M
12 10 (1.00%) Opscode Chef
13 6 (0.60%) Kibana
14 4 (0.40%) Active Directory Federation Services
14 4 (0.40%) logstash
15 3 (0.30%) Apache Slider
15 3 (0.30%) HAProxy
15 3 (0.30%) Host Intrusion Detection System
15 3 (0.30%) Network Intrusion Detection System
16 2 (0.20%) vCenter Server
Vendors
1 189 (18.92%) Cloudera
2 163 (16.32%) Oracle
3 91 (9.11%) Informatica
4 83 (8.31%) Google
5 67 (6.71%) Microsoft
6 58 (5.81%) Databricks
7 55 (5.51%) Hortonworks
8 47 (4.70%) Talend
9 46 (4.60%) Teradata
10 30 (3.00%) IBM
11 27 (2.70%) StreamSets
12 23 (2.30%) Alteryx
12 23 (2.30%) Splunk
13 20 (2.00%) SAP
14 17 (1.70%) VMware
15 16 (1.60%) Confluent
16 15 (1.50%) Ab Initio
16 15 (1.50%) Atlassian
16 15 (1.50%) Red Hat
17 11 (1.10%) MapR