Period
to 25 October 2021

The following table provides summary statistics for contract job vacancies with a requirement for Hadoop skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Hadoop over the 6 months to 25 October 2021 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Hadoop
UK
6 months to
25 Oct 2021
Same period 2020 Same period 2019
Rank 199 190 165
Rank change year-on-year -9 -25 -36
Contract jobs citing Hadoop 1,155 462 1,352
As % of all contract jobs advertised in the UK 1.36% 1.45% 1.60%
As % of the Database & Business Intelligence category 6.39% 6.34% 7.29%
Number of daily rates quoted 672 296 947
10th Percentile £418 £369 £425
25th Percentile £475 £425 £475
Median daily rate (50th Percentile) £563 £525 £550
Median % change year-on-year +7.14% -4.55% +2.98%
75th Percentile £675 £596 £641
90th Percentile £738 £675 £721
UK excluding London median daily rate £525 £450 £525
% change year-on-year +16.67% -14.29% +5.00%
Number of hourly rates quoted 15 2 5
10th Percentile £20.66 £21.90 £44.34
25th Percentile £24.53 £22.50 £47.43
Median hourly rate £26.00 £24.50 £49.86
Median % change year-on-year +6.12% -50.86% +4.97%
75th Percentile £27.73 £26.00 £50.00
90th Percentile £58.80 £26.30 £51.37
UK excluding London median hourly rate £27.00 £24.50 -
% change year-on-year +10.20% - -

Hadoop is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Contract vacancies with a requirement for database or business intelligence skills 18,068 7,285 18,547
As % of all contract IT jobs advertised in the UK 21.30% 22.88% 21.98%
Number of daily rates quoted 12,695 4,988 12,805
10th Percentile £351 £313 £319
25th Percentile £425 £400 £400
Median daily rate (50th Percentile) £525 £488 £475
Median % change year-on-year +7.69% +2.63% -
75th Percentile £625 £588 £575
90th Percentile £713 £668 £656
UK excluding London median daily rate £463 £438 £438
% change year-on-year +5.71% - +2.94%
Number of hourly rates quoted 220 120 247
10th Percentile £16.20 £12.63 £11.72
25th Percentile £22.69 £17.65 £16.84
Median hourly rate £34.75 £25.52 £29.96
Median % change year-on-year +36.15% -14.81% +8.95%
75th Percentile £57.58 £42.00 £46.17
90th Percentile £65.50 £55.00 £60.00
UK excluding London median hourly rate £30.00 £23.43 £27.94
% change year-on-year +28.04% -16.14% +7.46%

Hadoop
Job Vacancy Trend

Job postings citing Hadoop as a proportion of all IT jobs advertised.

Job vacancy trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Trend

3-month moving average daily rate quoted in jobs citing Hadoop.

Daily rate trend for Hadoop in the UK

Hadoop
Daily Rate Histogram

Daily rate distribution for jobs citing Hadoop over the 6 months to 25 October 2021.

Daily rate histogram for Hadoop in the UK

Hadoop
Contractor Hourly Rate Trend

3-month moving average hourly rates quoted in jobs citing Hadoop.

Hourly rate trend for Hadoop in the UK

Hadoop
Hourly Rate Histogram

Hourly rate distribution of jobs citing Hadoop over the 6 months to 25 October 2021.

Hourly rate histogram for Hadoop in the UK

Hadoop
Top 17 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Hadoop within the UK over the 6 months to 25 October 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -18 948 £563 +6.64% 204
London -21 563 £575 +4.55% 117
UK excluding London -29 355 £525 +16.67% 94
Work from Home -51 316 £575 +6.48% 37
South East -20 100 £525 +23.53% 25
Midlands -17 81 £500 +20.48% 36
West Midlands -10 75 £500 +16.28% 26
North of England -104 60 £513 +2.50% 14
Scotland -52 41 £475 +17.28% 6
East of England -33 31 £545 +14.74% 5
North East - 28 £608 - 3
Wales -12 21 £625 - 1
South West -41 19 £600 -7.69% 4
Yorkshire -40 17 £505 +18.57% 1
North West -109 16 £511 +2.25% 10
East Midlands -39 6 £519 +50.00% 10
Northern Ireland - 3 £475 - 3

For the 6 months to 25 October 2021, IT contractor jobs citing Hadoop also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for Hadoop.

1 638 (55.24%) Apache Spark
2 554 (47.97%) Big Data
3 505 (43.72%) SQL
4 456 (39.48%) Python
5 404 (34.98%) Apache Hive
6 403 (34.89%) AWS
7 369 (31.95%) Java
8 362 (31.34%) Kafka
9 331 (28.66%) Agile Software Development
10 325 (28.14%) Analytics
11 299 (25.89%) DevOps
12 297 (25.71%) ETL
13 290 (25.11%) Scala
14 271 (23.46%) Azure
15 247 (21.39%) Finance
16 236 (20.43%) Social Skills
17 228 (19.74%) Data Modelling
18 224 (19.39%) HBase
19 213 (18.44%) Data Warehouse
20 203 (17.58%) Docker
21 201 (17.40%) HDFS
22 188 (16.28%) Jenkins
23 182 (15.76%) Cloudera
23 182 (15.76%) Security Cleared
24 181 (15.67%) Git (software)
25 180 (15.58%) Kubernetes
26 179 (15.50%) Data Engineering
27 178 (15.41%) Linux
28 166 (14.37%) Business Intelligence
29 161 (13.94%) Data Lake

Hadoop
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 638 (55.24%) Apache Spark
2 105 (9.09%) Apache
3 74 (6.41%) Elasticsearch
4 73 (6.32%) Confluence
5 71 (6.15%) SAS
6 55 (4.76%) Apache Airflow
7 28 (2.42%) Luigi
8 18 (1.56%) nginx
9 17 (1.47%) Apache Pig
10 15 (1.30%) OpenStack
11 10 (0.87%) Tomcat
12 8 (0.69%) Jupyter
13 5 (0.43%) SharePoint
14 4 (0.35%) JBoss
15 2 (0.17%) Apache Solr
15 2 (0.17%) CMS
15 2 (0.17%) PowerExchange
15 2 (0.17%) R Markdown
15 2 (0.17%) WebSphere
15 2 (0.17%) WebSphere Application Server
Applications
1 33 (2.86%) Microsoft Excel
2 18 (1.56%) Spreadsheet
3 5 (0.43%) SPSS
4 2 (0.17%) Microsoft Office
5 1 (0.087%) SPSS Modeler
Business Applications
1 2 (0.17%) Infor M3
2 1 (0.087%) Distributed Ledger
2 1 (0.087%) Oracle BPM
2 1 (0.087%) Oracle Cash Management
2 1 (0.087%) SAP FI-CA
2 1 (0.087%) SAP Oil and Gas
2 1 (0.087%) SAP PSCD
2 1 (0.087%) Workday HCM
Cloud Services
1 403 (34.89%) AWS
2 271 (23.46%) Azure
3 128 (11.08%) GCP
4 88 (7.62%) Amazon EMR
5 67 (5.80%) Power Platform
5 67 (5.80%) Serverless
6 65 (5.63%) Amazon S3
7 60 (5.19%) Azure Cosmos DB
8 57 (4.94%) Amazon EC2
9 56 (4.85%) GitHub
10 53 (4.59%) Snowflake
11 49 (4.24%) Azure Data Factory
11 49 (4.24%) Route 53
12 43 (3.72%) Amazon EKS
12 43 (3.72%) PaaS
13 40 (3.46%) Amazon ECS
14 38 (3.29%) Azure Functions
15 35 (3.03%) IaaS
16 25 (2.16%) AWS Lambda
17 22 (1.90%) Azure Synapse Analytics
Communications & Networking
1 32 (2.77%) DNS
2 23 (1.99%) Kerberos
3 19 (1.65%) HTTP
4 16 (1.39%) WAN
5 15 (1.30%) Firewall
5 15 (1.30%) FTP
6 12 (1.04%) SSL
7 7 (0.61%) Network Security
8 5 (0.43%) HTTPS
8 5 (0.43%) VPN
9 4 (0.35%) Internet
9 4 (0.35%) LAN
9 4 (0.35%) Multicast
9 4 (0.35%) SAN
10 3 (0.26%) 3GPP
10 3 (0.26%) 4G
10 3 (0.26%) Intelligent Network
10 3 (0.26%) LTE
10 3 (0.26%) MPLS
10 3 (0.26%) Wireless
Database & Business Intelligence
1 554 (47.97%) Big Data
2 404 (34.98%) Apache Hive
3 224 (19.39%) HBase
4 213 (18.44%) Data Warehouse
5 161 (13.94%) Data Lake
5 161 (13.94%) Relational Database
6 147 (12.73%) NoSQL
7 142 (12.29%) PostgreSQL
8 125 (10.82%) MongoDB
9 108 (9.35%) Tableau
10 104 (9.00%) Impala
10 104 (9.00%) MySQL
11 100 (8.66%) Apache Cassandra
12 80 (6.93%) Metadata
13 74 (6.41%) SQL Server
14 67 (5.80%) Pentaho
15 65 (5.63%) Power BI
16 52 (4.50%) Flink
16 52 (4.50%) RDBMS
17 48 (4.16%) Amazon RDS
Development Applications
1 188 (16.28%) Jenkins
2 181 (15.67%) Git (software)
3 112 (9.70%) JIRA
4 57 (4.94%) Bitbucket
5 56 (4.85%) GitLab
6 32 (2.77%) Maven
7 31 (2.68%) Artifactory
8 22 (1.90%) Sonatype Nexus
9 20 (1.73%) ERwin
9 20 (1.73%) TeamCity
10 15 (1.30%) Subversion
11 14 (1.21%) Mockito
12 13 (1.13%) JUnit
13 11 (0.95%) ER Studio
14 9 (0.78%) Gradle
15 8 (0.69%) WebDriver
16 7 (0.61%) Cucumber
17 6 (0.52%) Eclipse
17 6 (0.52%) IntelliJ
17 6 (0.52%) Visual Studio
General
1 247 (21.39%) Finance
2 236 (20.43%) Social Skills
3 142 (12.29%) Banking
4 102 (8.83%) Analytical Skills
5 62 (5.37%) Public Sector
6 56 (4.85%) Presentation Skills
7 39 (3.38%) Retail
8 31 (2.68%) Investment Banking
9 30 (2.60%) Telecoms
10 23 (1.99%) Documentation Skills
11 21 (1.82%) Marketing
12 17 (1.47%) Legal
13 12 (1.04%) Financial Institution
14 10 (0.87%) Games
14 10 (0.87%) Influencing Skills
14 10 (0.87%) Manufacturing
15 9 (0.78%) Publishing
16 8 (0.69%) Law
17 6 (0.52%) Aerospace
18 5 (0.43%) Front Office
Job Titles
1 328 (28.40%) Data Engineer
2 239 (20.69%) Developer
3 135 (11.69%) Architect
4 105 (9.09%) DevOps Engineer
5 70 (6.06%) AWS Engineer
6 66 (5.71%) Data Architect
7 62 (5.37%) Big Data Engineer
8 48 (4.16%) AWS DevOps
9 47 (4.07%) Hadoop Developer
9 47 (4.07%) Scala Developer
10 45 (3.90%) Hadoop Engineer
11 44 (3.81%) AWS DevOps Engineer
12 38 (3.29%) Consultant
13 37 (3.20%) Analyst
13 37 (3.20%) Azure Engineer
13 37 (3.20%) Python Engineer
14 35 (3.03%) Java Developer
15 33 (2.86%) Data Manager
15 33 (2.86%) Solutions Architect
16 30 (2.60%) Senior Developer
Libraries, Frameworks & Software Standards
1 362 (31.34%) Kafka
2 101 (8.74%) PySpark
3 72 (6.23%) Spark Streaming
4 60 (5.19%) Spring
5 55 (4.76%) JSON
6 51 (4.42%) Apache NiFi
7 41 (3.55%) REST
8 37 (3.20%) Spring Boot
9 33 (2.86%) XML
10 32 (2.77%) Web Services
11 31 (2.68%) RESTful
12 29 (2.51%) ARM Templates
13 23 (1.99%) Apache Avro
14 20 (1.73%) .NET
15 19 (1.65%) Akka
15 19 (1.65%) OAuth
15 19 (1.65%) RabbitMQ
16 17 (1.47%) JDBC
17 15 (1.30%) HTML
17 15 (1.30%) Regular Expression
Miscellaneous
1 69 (5.97%) Data Structures
2 54 (4.68%) Data Protection Act
3 25 (2.16%) Public Cloud
4 24 (2.08%) Algorithms
4 24 (2.08%) PKI
5 22 (1.90%) Hybrid Cloud
6 21 (1.82%) Data Centre
7 20 (1.73%) Greenfield Project
8 19 (1.65%) Management Information System
9 18 (1.56%) Social Media
10 17 (1.47%) Legacy Systems
11 16 (1.39%) Cloud Native
12 15 (1.30%) Virtual Team
13 14 (1.21%) Linux Command Line
14 13 (1.13%) Self-Motivation
15 12 (1.04%) Mainframe
15 12 (1.04%) Market Risk
16 10 (0.87%) Hedge funds
17 9 (0.78%) Reference Data
18 8 (0.69%) Derivative
Operating Systems
1 178 (15.41%) Linux
2 137 (11.86%) Unix
3 46 (3.98%) Windows
4 17 (1.47%) Red Hat Enterprise Linux
5 12 (1.04%) Debian
6 6 (0.52%) VMS
7 5 (0.43%) CoreOS
8 3 (0.26%) CentOS
9 2 (0.17%) Solaris
9 2 (0.17%) Ubuntu
9 2 (0.17%) Windows Server
9 2 (0.17%) Windows Server 2012
9 2 (0.17%) Windows Server 2016
9 2 (0.17%) Windows Server 2019
Processes & Methodologies
1 331 (28.66%) Agile Software Development
2 325 (28.14%) Analytics
3 299 (25.89%) DevOps
4 297 (25.71%) ETL
5 228 (19.74%) Data Modelling
6 179 (15.50%) Data Engineering
7 166 (14.37%) Business Intelligence
8 153 (13.25%) Data Pipeline
9 144 (12.47%) Scrum
10 121 (10.48%) Data Science
10 121 (10.48%) Migration
11 119 (10.30%) Data Analytics
12 113 (9.78%) Machine Learning
13 108 (9.35%) Microservices
14 97 (8.40%) Computer Science
15 94 (8.14%) Data Management
16 93 (8.05%) Software Engineering
17 90 (7.79%) Visualisation
18 87 (7.53%) Continuous Integration
18 87 (7.53%) Data Architecture
Programming Languages
1 505 (43.72%) SQL
2 456 (39.48%) Python
3 369 (31.95%) Java
4 290 (25.11%) Scala
5 77 (6.67%) R
6 66 (5.71%) Shell Script
7 64 (5.54%) PL/SQL
8 41 (3.55%) HiveQL
9 38 (3.29%) Bash Shell
10 34 (2.94%) Perl
11 32 (2.77%) JavaScript
12 28 (2.42%) C++
13 25 (2.16%) Groovy
14 24 (2.08%) PowerShell
15 23 (1.99%) VBA
16 22 (1.90%) C#
17 17 (1.47%) C
18 16 (1.39%) Go
19 15 (1.30%) Java 8
20 13 (1.13%) CQL
Qualifications
1 182 (15.76%) Security Cleared
2 134 (11.60%) SC Cleared
3 97 (8.40%) Degree
4 54 (4.68%) Computer Science Degree
5 27 (2.34%) DV Cleared
6 20 (1.73%) BPSS Clearance
7 7 (0.61%) AWS Certification
8 6 (0.52%) SAS Certification
9 5 (0.43%) ISTQB
10 3 (0.26%) ISTQB Advanced Certification
11 2 (0.17%) (ISC)2 CCSP
11 2 (0.17%) CCA
11 2 (0.17%) Cisco Certification
11 2 (0.17%) CISM
11 2 (0.17%) CISSP
11 2 (0.17%) ISEB
11 2 (0.17%) TOGAF Certification
12 1 (0.087%) CBAP
12 1 (0.087%) MBA
12 1 (0.087%) PMI Certification
Quality Assurance & Compliance
1 74 (6.41%) Data Quality
2 45 (3.90%) QA
3 22 (1.90%) GDPR
4 7 (0.61%) SLA
5 6 (0.52%) Disclosure Scotland
5 6 (0.52%) Sarbanes-Oxley
6 5 (0.43%) MiFID
7 3 (0.26%) IFRS
8 2 (0.17%) COBIT
8 2 (0.17%) ISO/IEC 27001
8 2 (0.17%) NCSC
8 2 (0.17%) NIST
9 1 (0.087%) Dodd-Frank
9 1 (0.087%) EMIR
9 1 (0.087%) ISO 20022
9 1 (0.087%) PSD2
9 1 (0.087%) TARGET2
System Software
1 203 (17.58%) Docker
2 201 (17.40%) HDFS
3 43 (3.72%) Apache ZooKeeper
4 15 (1.30%) Active Directory
4 15 (1.30%) Apache Flume
5 10 (0.87%) VMware Infrastructure
6 6 (0.52%) Oracle RAC
6 6 (0.52%) vCloud
6 6 (0.52%) VMware NSX
7 4 (0.35%) Active Directory Lightweight Directory Services
7 4 (0.35%) Virtual Machines
8 3 (0.26%) Hyper-V
8 3 (0.26%) SRDF
9 2 (0.17%) KVM
9 2 (0.17%) NFS
9 2 (0.17%) Squid
9 2 (0.17%) VMware ESXi
9 2 (0.17%) vSphere
9 2 (0.17%) Xen
Systems Management
1 180 (15.58%) Kubernetes
2 149 (12.90%) Terraform
3 125 (10.82%) Ansible
4 61 (5.28%) Prometheus
5 45 (3.90%) Hadoop YARN
6 42 (3.64%) Oozie
7 37 (3.20%) Puppet
8 31 (2.68%) Grafana
9 15 (1.30%) Rancher
10 14 (1.21%) Ambari
11 12 (1.04%) Control-M
11 12 (1.04%) Kibana
12 11 (0.95%) HAProxy
13 8 (0.69%) Consul
13 8 (0.69%) Opscode Chef
14 6 (0.52%) Fluentd
14 6 (0.52%) ZABBIX
15 5 (0.43%) Host Intrusion Detection System
15 5 (0.43%) Network Intrusion Detection System
16 4 (0.35%) logstash
Vendors
1 182 (15.76%) Cloudera
2 158 (13.68%) Oracle
3 136 (11.77%) Informatica
4 89 (7.71%) Databricks
5 84 (7.27%) Google
6 68 (5.89%) Microsoft
7 67 (5.80%) Hortonworks
8 50 (4.33%) Splunk
9 49 (4.24%) Teradata
10 47 (4.07%) Talend
11 44 (3.81%) Ab Initio
12 28 (2.42%) Red Hat
13 24 (2.08%) SAP
14 23 (1.99%) Alteryx
15 20 (1.73%) StreamSets
16 18 (1.56%) MapR
16 18 (1.56%) VMware
17 16 (1.39%) MicroStrategy
18 15 (1.30%) Atlassian
18 15 (1.30%) IBM