Period
to 17 April 2021

The following table provides summary statistics for contract job vacancies with a requirement for Hadoop skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Hadoop over the 6 months to 17 April 2021 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Hadoop
UK
6 months to
17 Apr 2021
Same period 2020 Same period 2019
Rank 182 206 129
Rank change year-on-year +24 -77 -11
Contract jobs citing Hadoop 734 750 1,903
As % of all contract jobs advertised in the UK 1.38% 1.29% 1.98%
As % of the Database & Business Intelligence category 6.52% 6.11% 8.83%
Number of daily rates quoted 429 527 1,289
10th Percentile £400 £404 £413
25th Percentile £463 £463 £480
Median daily rate (50th Percentile) £550 £550 £550
75th Percentile £625 £631 £638
90th Percentile £675 £713 £713
UK excluding London median daily rate £500 £520 £525
% change year-on-year -3.85% -0.95% +5.00%
Number of hourly rates quoted 4 0 4
10th Percentile £17.78 " "
25th Percentile £23.06 " £32.50
Median hourly rate £33.50 - £45.00
75th Percentile £43.75 " £60.88
90th Percentile £50.50 " £66.95
UK excluding London median hourly rate £32.25 - £69.00

Hadoop is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Contract vacancies with a requirement for database or business intelligence skills 11,260 12,281 21,556
As % of all contract IT jobs advertised in the UK 21.20% 21.17% 22.46%
Number of daily rates quoted 7,758 8,377 14,990
10th Percentile £320 £320 £320
25th Percentile £405 £400 £390
Median daily rate (50th Percentile) £500 £475 £475
Median % change year-on-year +5.26% - +2.15%
75th Percentile £588 £575 £575
90th Percentile £675 £675 £650
UK excluding London median daily rate £438 £450 £425
% change year-on-year -2.78% +5.88% +3.03%
Number of hourly rates quoted 192 183 239
10th Percentile £13.50 £14.36 £13.94
25th Percentile £19.19 £18.25 £18.07
Median hourly rate £26.00 £24.70 £30.00
Median % change year-on-year +5.26% -17.67% -
75th Percentile £45.00 £44.73 £47.34
90th Percentile £58.68 £57.90 £57.50
UK excluding London median hourly rate £23.47 £22.50 £30.00
% change year-on-year +4.31% -25.00% -13.04%

Hadoop
Job Vacancy Trend

Job postings citing Hadoop as a proportion of all IT jobs advertised.

Job vacancy trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Trend

3-month moving average daily rate quoted in jobs citing Hadoop.

Daily rate trend for Hadoop in the UK

Hadoop
Daily Rate Histogram

Daily rate distribution for jobs citing Hadoop over the 6 months to 17 April 2021.

Daily rate histogram for Hadoop in the UK

Hadoop
Contractor Hourly Rate Trend

3-month moving average hourly rates quoted in jobs citing Hadoop.

Hourly rate trend for Hadoop in the UK

Hadoop
Hourly Rate Histogram

Hourly rate distribution of jobs citing Hadoop over the 6 months to 17 April 2021.

Hourly rate histogram for Hadoop in the UK

Hadoop
Top 16 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Hadoop within the UK over the 6 months to 17 April 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +15 587 £550 - 156
London -4 360 £575 +4.55% 97
UK excluding London +3 227 £500 -3.85% 69
Work from Home -84 218 £540 +6.67% 29
South East +42 62 £475 -0.78% 15
Midlands +47 48 £491 +9.00% 11
West Midlands +41 45 £491 +7.51% 8
North of England -6 36 £555 +0.91% 14
Scotland +3 30 £400 -30.43% 10
South West +37 28 £516 -6.73% 6
North West -5 21 £580 +3.07% 9
Yorkshire +17 15 £555 +15.63% 5
Wales +11 13 £550 +12.82% 1
East of England +27 8 £450 +3.45% 11
Northern Ireland - 2 £450 - 1
East Midlands +22 1 £350 - 3

For the 6 months to 17 April 2021, IT contractor jobs citing Hadoop also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for Hadoop.

1 426 (58.04%) Apache Spark
2 363 (49.46%) Big Data
3 308 (41.96%) Python
4 282 (38.42%) Java
4 282 (38.42%) Apache Hive
5 275 (37.47%) SQL
6 264 (35.97%) Agile Software Development
7 247 (33.65%) Scala
8 224 (30.52%) AWS
9 188 (25.61%) Kafka
10 181 (24.66%) DevOps
11 163 (22.21%) HDFS
12 158 (21.53%) Cloudera
13 157 (21.39%) Security Cleared
14 156 (21.25%) ETL
15 152 (20.71%) Finance
15 152 (20.71%) Analytics
16 147 (20.03%) Azure
17 140 (19.07%) SC Cleared
18 135 (18.39%) Linux
19 134 (18.26%) Apache
20 131 (17.85%) Scrum
21 116 (15.80%) Banking
21 116 (15.80%) Jenkins
22 114 (15.53%) Ansible
22 114 (15.53%) Oracle
23 109 (14.85%) HBase
24 106 (14.44%) Data Engineering
24 106 (14.44%) Git (software)
25 104 (14.17%) Data Modelling

Hadoop
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 426 (58.04%) Apache Spark
2 134 (18.26%) Apache
3 66 (8.99%) Confluence
4 42 (5.72%) Apache Airflow
5 37 (5.04%) Elasticsearch
6 28 (3.81%) SAS
7 12 (1.63%) Tomcat
8 11 (1.50%) Apache Pig
9 7 (0.95%) JBoss
9 7 (0.95%) Jetty
9 7 (0.95%) WebSphere
10 6 (0.82%) Apache Solr
11 5 (0.68%) Apache Pulsar
11 5 (0.68%) Luigi
12 4 (0.54%) MQSeries
12 4 (0.54%) Oracle Self-Service HR
12 4 (0.54%) Red Hat Fuse
12 4 (0.54%) WebLogic
12 4 (0.54%) WebSphere Application Server
13 3 (0.41%) SharePoint
Applications
1 24 (3.27%) Spreadsheet
2 6 (0.82%) Microsoft Excel
3 5 (0.68%) SPSS
3 5 (0.68%) Stata
4 2 (0.27%) Chatbot
4 2 (0.27%) Microsoft Office
4 2 (0.27%) MS Visio
5 1 (0.14%) Microsoft PowerPoint
Business Applications
1 4 (0.54%) Sparx Enterprise Architect
2 2 (0.27%) Temenos T24
3 1 (0.14%) Oracle BPM
3 1 (0.14%) SAP CRM
3 1 (0.14%) SAP ERP
3 1 (0.14%) SAP IS-U
3 1 (0.14%) SAP MDG
3 1 (0.14%) SAP MDM
3 1 (0.14%) Traffica
Cloud Services
1 224 (30.52%) AWS
2 147 (20.03%) Azure
3 77 (10.49%) GCP
4 74 (10.08%) Amazon S3
5 42 (5.72%) Amazon EMR
6 38 (5.18%) Power Platform
7 34 (4.63%) GitHub
8 32 (4.36%) Serverless
9 29 (3.95%) AWS Lambda
10 26 (3.54%) PaaS
11 24 (3.27%) IaaS
12 23 (3.13%) Amazon EC2
13 13 (1.77%) Azure Data Factory
14 12 (1.63%) OpenShift
15 11 (1.50%) Amazon ECS
16 10 (1.36%) Amazon CloudWatch
17 9 (1.23%) AWS CloudFormation
17 9 (1.23%) Route 53
17 9 (1.23%) SaaS
18 8 (1.09%) Microsoft 365
Communications & Networking
1 14 (1.91%) Firewall
2 11 (1.50%) WAN
3 10 (1.36%) Kerberos
3 10 (1.36%) VSAN
4 7 (0.95%) Ethernet
4 7 (0.95%) VLAN
5 6 (0.82%) DNS
6 4 (0.54%) Cisco Nexus
6 4 (0.54%) FCoE
6 4 (0.54%) Fibre Channel
6 4 (0.54%) iSCSI
6 4 (0.54%) LAN
6 4 (0.54%) Multicast
6 4 (0.54%) NAS
6 4 (0.54%) Network Security
6 4 (0.54%) SAN
6 4 (0.54%) SSH
7 3 (0.41%) Remote Desktop
8 2 (0.27%) Internet
8 2 (0.27%) SSL
Database & Business Intelligence
1 363 (49.46%) Big Data
2 282 (38.42%) Apache Hive
3 109 (14.85%) HBase
4 103 (14.03%) Impala
5 94 (12.81%) Data Lake
6 90 (12.26%) Data Warehouse
7 89 (12.13%) NoSQL
8 85 (11.58%) Relational Database
9 69 (9.40%) MongoDB
10 65 (8.86%) PostgreSQL
11 59 (8.04%) MySQL
11 59 (8.04%) SQL Server
12 49 (6.68%) Pentaho
13 46 (6.27%) Tableau
14 41 (5.59%) Spark SQL
15 40 (5.45%) Apache Cassandra
16 39 (5.31%) Amazon Redshift
17 38 (5.18%) Power BI
18 28 (3.81%) BigQuery
19 23 (3.13%) Presto
Development Applications
1 116 (15.80%) Jenkins
2 106 (14.44%) Git (software)
3 68 (9.26%) JIRA
4 47 (6.40%) Sonatype Nexus
5 34 (4.63%) Maven
6 26 (3.54%) IBM UrbanCode
7 21 (2.86%) GitLab
8 19 (2.59%) Gradle
8 19 (2.59%) Selenium
9 17 (2.32%) RStudio
10 16 (2.18%) Bitbucket
11 14 (1.91%) Artifactory
12 13 (1.77%) TeamCity
13 12 (1.63%) Cucumber
14 8 (1.09%) Atlassian Bamboo
14 8 (1.09%) IntelliJ
14 8 (1.09%) Subversion
15 7 (0.95%) Hudson
15 7 (0.95%) Protractor
16 6 (0.82%) Eclipse
General
1 152 (20.71%) Finance
2 116 (15.80%) Banking
3 52 (7.08%) Public Sector
4 45 (6.13%) Investment Banking
5 34 (4.63%) Front Office
6 15 (2.04%) Marketing
7 11 (1.50%) Retail
8 10 (1.36%) Legal
9 8 (1.09%) Manufacturing
9 8 (1.09%) Retail Banking
9 8 (1.09%) Telecoms
10 6 (0.82%) Games
11 5 (0.68%) Electronics
11 5 (0.68%) Financial Institution
11 5 (0.68%) French Language
12 4 (0.54%) Automotive
13 3 (0.41%) Publishing
14 2 (0.27%) Pharmaceutical
15 1 (0.14%) Billing
15 1 (0.14%) German Language
Job Titles
1 209 (28.47%) Developer
2 179 (24.39%) Data Engineer
3 94 (12.81%) Architect
4 63 (8.58%) Scala Developer
5 57 (7.77%) Big Data Engineer
6 52 (7.08%) Java Developer
7 50 (6.81%) Hadoop Developer
8 46 (6.27%) Apache Spark Developer
9 45 (6.13%) AWS Engineer
10 43 (5.86%) Big Data Developer
11 42 (5.72%) Data Architect
12 37 (5.04%) DevOps Engineer
13 36 (4.90%) Analyst
14 33 (4.50%) Consultant
14 33 (4.50%) Solutions Architect
15 32 (4.36%) Cloud Engineer
16 26 (3.54%) Scala Engineer
17 23 (3.13%) AWS Architect
17 23 (3.13%) Hadoop Engineer
17 23 (3.13%) Software Engineer
Libraries, Frameworks & Software Standards
1 188 (25.61%) Kafka
2 58 (7.90%) PySpark
3 38 (5.18%) Spring
4 37 (5.04%) Apache NiFi
5 31 (4.22%) JSON
6 27 (3.68%) Spark Streaming
7 26 (3.54%) RESTful
7 26 (3.54%) XML
8 21 (2.86%) Web Services
9 19 (2.59%) RabbitMQ
10 14 (1.91%) Node.js
11 13 (1.77%) AngularJS
11 13 (1.77%) Pandas
12 11 (1.50%) HTML
12 11 (1.50%) Spring Boot
12 11 (1.50%) TensorFlow
13 9 (1.23%) .NET
13 9 (1.23%) Hibernate
13 9 (1.23%) HTML5
13 9 (1.23%) React
Miscellaneous
1 90 (12.26%) Analytical Skills
2 32 (4.36%) Data Structures
3 31 (4.22%) Linux Command Line
4 17 (2.32%) Algorithms
5 13 (1.77%) Analytical Mindset
5 13 (1.77%) Self-Motivation
6 9 (1.23%) Cloud Native
6 9 (1.23%) Cybercrime
6 9 (1.23%) Enterprise Software
6 9 (1.23%) Equities
6 9 (1.23%) Greenfield Project
7 6 (0.82%) Credit Risk
7 6 (0.82%) Mainframe
7 6 (0.82%) Management Information System
7 6 (0.82%) Public Cloud
7 6 (0.82%) Replication
7 6 (0.82%) User Experience
7 6 (0.82%) VSAM
8 5 (0.68%) Derivative
8 5 (0.68%) PKI
Operating Systems
1 135 (18.39%) Linux
2 73 (9.95%) Unix
3 39 (5.31%) Windows
4 22 (3.00%) CentOS
5 17 (2.32%) Ubuntu
6 10 (1.36%) Red Hat Enterprise Linux
7 5 (0.68%) Android
7 5 (0.68%) Apple iOS
7 5 (0.68%) Mac OS X
7 5 (0.68%) Solaris
8 4 (0.54%) Debian
9 3 (0.41%) Windows 10
9 3 (0.41%) Windows Server
10 2 (0.27%) AIX
10 2 (0.27%) SUSE
10 2 (0.27%) VME
11 1 (0.14%) MVS
Processes & Methodologies
1 264 (35.97%) Agile Software Development
2 181 (24.66%) DevOps
3 156 (21.25%) ETL
4 152 (20.71%) Analytics
5 131 (17.85%) Scrum
6 106 (14.44%) Data Engineering
7 104 (14.17%) Data Modelling
8 88 (11.99%) Data Science
9 87 (11.85%) TDD
10 86 (11.72%) Business Intelligence
11 83 (11.31%) Problem-Solving
12 82 (11.17%) Computer Science
13 73 (9.95%) BDD
14 65 (8.86%) Machine Learning
15 63 (8.58%) Software Engineering
16 59 (8.04%) Data Ingestion
16 59 (8.04%) Migration
17 58 (7.90%) Data Management
18 57 (7.77%) Test Automation
19 52 (7.08%) Configuration Management
Programming Languages
1 308 (41.96%) Python
2 282 (38.42%) Java
3 275 (37.47%) SQL
4 247 (33.65%) Scala
5 52 (7.08%) Shell Script
6 47 (6.40%) R
7 29 (3.95%) JavaScript
8 20 (2.72%) C++
9 19 (2.59%) Bash Shell
9 19 (2.59%) PL/SQL
10 17 (2.32%) C#
11 12 (1.63%) Haskell
11 12 (1.63%) Perl
12 11 (1.50%) C
12 11 (1.50%) Elm
13 9 (1.23%) Go
13 9 (1.23%) Groovy
13 9 (1.23%) T-SQL
14 8 (1.09%) Ruby
15 7 (0.95%) PowerShell
Qualifications
1 157 (21.39%) Security Cleared
2 140 (19.07%) SC Cleared
3 72 (9.81%) Degree
4 26 (3.54%) Numerate Degree
5 22 (3.00%) Computer Science Degree
6 17 (2.32%) DV Cleared
7 12 (1.63%) AWS Certification
8 8 (1.09%) SAS Certification
9 7 (0.95%) PhD
10 6 (0.82%) BPSS Clearance
10 6 (0.82%) Master's Degree
11 4 (0.54%) CCA
12 2 (0.27%) TOGAF Certification
13 1 (0.14%) CEH
13 1 (0.14%) PMI Certification
Quality Assurance & Compliance
1 33 (4.50%) QA
2 28 (3.81%) Data Quality
3 14 (1.91%) SLA
4 8 (1.09%) GDPR
5 1 (0.14%) Basel II
5 1 (0.14%) ISO 20022
5 1 (0.14%) PCI DSS
5 1 (0.14%) PMO
5 1 (0.14%) SEPA
5 1 (0.14%) TARGET2
System Software
1 163 (22.21%) HDFS
2 73 (9.95%) Docker
3 22 (3.00%) Apache Flume
4 20 (2.72%) Apache ZooKeeper
5 19 (2.59%) Active Directory
6 10 (1.36%) VMware NSX
6 10 (1.36%) vSphere
7 7 (0.95%) VMware Infrastructure
8 5 (0.68%) Oracle RAC
9 4 (0.54%) Hyper-V
9 4 (0.54%) KVM
9 4 (0.54%) NFS
9 4 (0.54%) SRDF
9 4 (0.54%) Xen
10 3 (0.41%) Virtual Machines
10 3 (0.41%) VMware ESXi
11 2 (0.27%) Terminal Services
12 1 (0.14%) Active Directory Lightweight Directory Services
Systems Management
1 114 (15.53%) Ansible
2 86 (11.72%) Kubernetes
3 54 (7.36%) Oozie
3 54 (7.36%) Terraform
4 48 (6.54%) Hadoop YARN
5 25 (3.41%) Grafana
6 23 (3.13%) Puppet
7 20 (2.72%) Prometheus
8 10 (1.36%) Ambari
9 8 (1.09%) Consul
9 8 (1.09%) Control-M
9 8 (1.09%) FreeIPA
10 7 (0.95%) Microsoft Intune
10 7 (0.95%) vCenter Server
11 6 (0.82%) Opscode Chef
12 5 (0.68%) Rancher
13 4 (0.54%) Kibana
13 4 (0.54%) Salt
14 3 (0.41%) logstash
15 2 (0.27%) SCCM
Vendors
1 158 (21.53%) Cloudera
2 114 (15.53%) Oracle
3 53 (7.22%) Hortonworks
4 50 (6.81%) Google
5 45 (6.13%) Microsoft
6 32 (4.36%) Informatica
7 31 (4.22%) Teradata
8 28 (3.81%) Talend
9 22 (3.00%) StreamSets
10 21 (2.86%) IBM
11 14 (1.91%) Alteryx
11 14 (1.91%) SAP
12 13 (1.77%) MapR
12 13 (1.77%) Red Hat
13 12 (1.63%) Cisco
14 11 (1.50%) Databricks
14 11 (1.50%) EMC
15 10 (1.36%) VMware
16 9 (1.23%) Confluent
16 9 (1.23%) Splunk