Period
to 21 October 2020

The following table provides summary statistics for contract job vacancies with a requirement for Hadoop skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Hadoop over the 6 months to 21 October 2020 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Hadoop
UK
6 months to
21 Oct 2020
Same period 2019 Same period 2018
Rank 183 167 129
Rank change year-on-year -16 -38 -12
Contract jobs citing Hadoop 478 1,403 2,161
As % of all contract jobs advertised in the UK 1.51% 1.65% 2.11%
As % of the Database & Business Intelligence category 6.76% 7.62% 9.33%
Number of daily rates quoted 354 998 1,477
Median daily rate £531 £550 £533
Median daily rate % change year-on-year -3.41% +3.29% +1.43%
10th Percentile £362 £425 £406
90th Percentile £675 £719 £694
UK excluding London median daily rate £450 £525 £500
% change year-on-year -14.29% +5.00% +5.26%
Number of hourly rates quoted 13 14 6
Median hourly rate £35.00 £49.13 £56.25
Median hourly rate % change year-on-year -28.76% -12.66% +60.71%
UK excluding London median hourly rate £35.00 £35.00 £47.50
% change year-on-year - -26.32% +35.71%

Hadoop is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Contract vacancies with a requirement for database or business intelligence skills 7,071 18,406 23,173
As % of all contract IT jobs advertised in the UK 22.31% 21.66% 22.64%
Number of daily rates quoted 4,526 12,560 16,166
Median daily rate £488 £475 £475
Median daily rate % change year-on-year +2.68% - +5.56%
10th Percentile £313 £325 £313
90th Percentile £675 £651 £638
UK excluding London median daily rate £440 £438 £425
% change year-on-year +0.57% +2.94% +6.25%
Number of hourly rates quoted 153 257 227
Median hourly rate £35.00 £35.00 £30.00
Median hourly rate % change year-on-year - +16.67% -
10th Percentile £14.00 £12.58 £12.37
90th Percentile £51.75 £60.00 £56.55
Median hourly rate £29.50 £31.50 £33.00
% change year-on-year -6.35% -4.55% +1.54%

Hadoop
Job Vacancy Trend

Job postings citing Hadoop as a proportion of all IT jobs advertised.

Job vacancy trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Trend

3-month moving average daily rate quoted in jobs citing Hadoop.

Daily rate trend for Hadoop in the UK

Hadoop
Daily Rate Histogram

Daily rate distribution for jobs citing Hadoop over the 6 months to 21 October 2020.

Daily rate histogram for Hadoop in the UK

Hadoop
Contractor Hourly Rate Trend

3-month moving average hourly rates quoted in jobs citing Hadoop.

Hourly rate trend for Hadoop in the UK

Hadoop
Hourly Rate Histogram

Hourly rate distribution of jobs citing Hadoop over the 6 months to 21 October 2020.

Hourly rate histogram for Hadoop in the UK

Hadoop
Top 15 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Hadoop within the UK over the 6 months to 21 October 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -14 425 £550 - 138
London -34 241 £550 -4.35% 73
UK excluding London +51 183 £450 -14.29% 81
North of England +55 88 £500 - 22
North West +52 79 £500 -4.76% 14
Work from Home -56 79 £540 -1.82% 17
South East +68 30 £425 -17.07% 17
Midlands +71 24 £418 -20.48% 13
West Midlands +75 18 £425 -15.00% 7
South West +81 16 £650 +13.04% 5
Scotland +10 15 £405 -19.00% 14
Wales +46 9 £500 -25.93% 1
Yorkshire +31 8 £450 - 8
East of England +51 6 £475 -20.83% 8
East Midlands +34 6 £323 -38.57% 6

For the 6 months to 21 October 2020, IT contractor jobs citing Hadoop also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for Hadoop.

1 256 (53.56%) Big Data
2 245 (51.26%) Python
3 238 (49.79%) Apache Spark
4 166 (34.73%) Apache Hive
5 161 (33.68%) SQL
6 160 (33.47%) AWS
7 158 (33.05%) DevOps
8 147 (30.75%) Security Cleared
9 141 (29.50%) Cloudera
10 139 (29.08%) Java
11 129 (26.99%) SC Cleared
12 125 (26.15%) Jenkins
13 124 (25.94%) Azure
14 120 (25.10%) ETL
15 113 (23.64%) Kafka
16 111 (23.22%) Analytics
17 107 (22.38%) Agile Software Development
18 95 (19.87%) Kubernetes
19 94 (19.67%) Docker
20 92 (19.25%) Oracle
21 91 (19.04%) HDFS
22 90 (18.83%) Impala
23 88 (18.41%) Git (software)
24 87 (18.20%) Ansible
25 82 (17.15%) Scala
26 71 (14.85%) Terraform
27 69 (14.44%) Linux
28 68 (14.23%) Finance
28 68 (14.23%) Relational Database
28 68 (14.23%) Data Science

Hadoop
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 238 (49.79%) Apache Spark
2 49 (10.25%) Apache
3 37 (7.74%) Confluence
4 26 (5.44%) Elasticsearch
5 23 (4.81%) SAS
6 19 (3.97%) Apache Pig
7 17 (3.56%) Mule
8 16 (3.35%) WebLogic
9 14 (2.93%) Apache Airflow
10 8 (1.67%) Tomcat
11 6 (1.26%) Cloud Foundry
12 5 (1.05%) JBoss
12 5 (1.05%) Jetty
12 5 (1.05%) Jupyter
12 5 (1.05%) MQSeries
12 5 (1.05%) Oracle Self-Service HR
12 5 (1.05%) Red Hat Fuse
13 2 (0.42%) Luigi
13 2 (0.42%) OpenStack
13 2 (0.42%) Oracle SOA Suite
Applications
1 22 (4.60%) Spreadsheet
2 13 (2.72%) Microsoft Excel
3 3 (0.63%) MS Visio
4 2 (0.42%) Chatbot
4 2 (0.42%) SPSS
4 2 (0.42%) Stata
5 1 (0.21%) Microsoft PowerPoint
Business Applications
1 11 (2.30%) SAP FI-CA
2 6 (1.26%) Sparx Enterprise Architect
3 2 (0.42%) Oracle BPM
4 1 (0.21%) FINCAD Analytics
Cloud Services
1 160 (33.47%) AWS
2 124 (25.94%) Azure
3 45 (9.41%) GCP
4 32 (6.69%) Amazon S3
5 17 (3.56%) Serverless
6 13 (2.72%) Azure Data Factory
7 12 (2.51%) Amazon EMR
7 12 (2.51%) Power Platform
7 12 (2.51%) Virtual Private Cloud
8 9 (1.88%) Amazon EC2
8 9 (1.88%) AWS Lambda
9 8 (1.67%) Amazon EKS
10 7 (1.46%) PaaS
11 6 (1.26%) Amazon Kinesis
11 6 (1.26%) GitHub
11 6 (1.26%) Slack
12 5 (1.05%) npm
13 4 (0.84%) AWS CloudFormation
13 4 (0.84%) Cloud Computing
13 4 (0.84%) SaaS
Communications & Networking
1 14 (2.93%) SSL
2 7 (1.46%) Kerberos
3 5 (1.05%) SSH
4 3 (0.63%) VPN
5 1 (0.21%) 5G
5 1 (0.21%) Internet
5 1 (0.21%) Network Security
5 1 (0.21%) Wi-Fi
Database & Business Intelligence
1 256 (53.56%) Big Data
2 166 (34.73%) Apache Hive
3 90 (18.83%) Impala
4 68 (14.23%) Relational Database
5 49 (10.25%) MySQL
6 48 (10.04%) MongoDB
6 48 (10.04%) NoSQL
7 42 (8.79%) HBase
8 32 (6.69%) SQL Server
9 29 (6.07%) Data Warehouse
10 27 (5.65%) Apache Cassandra
11 26 (5.44%) Data Lake
11 26 (5.44%) Pentaho
12 23 (4.81%) PostgreSQL
13 19 (3.97%) Oracle Database
14 15 (3.14%) DataStage
14 15 (3.14%) Spark SQL
15 12 (2.51%) Data Mining
15 12 (2.51%) Power BI
15 12 (2.51%) SQL Server Integration Services
Development Applications
1 125 (26.15%) Jenkins
2 88 (18.41%) Git (software)
3 37 (7.74%) JIRA
4 27 (5.65%) Selenium
5 23 (4.81%) Cucumber
6 19 (3.97%) GitLab
7 16 (3.35%) Maven
7 16 (3.35%) TeamCity
8 11 (2.30%) Bitbucket
8 11 (2.30%) Postman
8 11 (2.30%) SoapUI
8 11 (2.30%) Toad
9 10 (2.09%) Sonatype Nexus
10 8 (1.67%) JUnit
11 6 (1.26%) Atlassian Bamboo
11 6 (1.26%) Vagrant
12 5 (1.05%) Hudson
12 5 (1.05%) Mercurial
12 5 (1.05%) Mockito
12 5 (1.05%) Poseidon for UML
General
1 68 (14.23%) Finance
2 51 (10.67%) Banking
3 29 (6.07%) Public Sector
4 20 (4.18%) Front Office
5 17 (3.56%) Marketing
6 15 (3.14%) Investment Banking
6 15 (3.14%) Retail
7 12 (2.51%) Aerospace
8 11 (2.30%) Telecoms
9 10 (2.09%) Manufacturing
10 8 (1.67%) Legal
11 6 (1.26%) Law
12 4 (0.84%) Electronics
12 4 (0.84%) Games
13 3 (0.63%) Pharmaceutical
14 2 (0.42%) Financial Institution
14 2 (0.42%) Military
15 1 (0.21%) Billing
Job Titles
1 125 (26.15%) Data Engineer
2 112 (23.43%) Developer
3 71 (14.85%) Architect
4 40 (8.37%) DevOps Engineer
5 37 (7.74%) Big Data Engineer
6 36 (7.53%) Administrator
7 30 (6.28%) Azure Engineer
7 30 (6.28%) Data Architect
7 30 (6.28%) Java Developer
8 29 (6.07%) Python Engineer
9 27 (5.65%) Cloud Engineer
10 26 (5.44%) Python Developer
11 24 (5.02%) Cloud Administrator
12 23 (4.81%) Hadoop Engineer
13 22 (4.60%) Cloud Architect
14 20 (4.18%) Analyst
14 20 (4.18%) DevOps Administrator
15 19 (3.97%) Data Scientist
16 16 (3.35%) Solutions Architect
17 15 (3.14%) AWS Engineer
Libraries, Frameworks & Software Standards
1 113 (23.64%) Kafka
2 38 (7.95%) PySpark
2 38 (7.95%) Spring
3 34 (7.11%) JSON
4 29 (6.07%) Web Services
5 24 (5.02%) XML
6 21 (4.39%) REST
7 19 (3.97%) RabbitMQ
7 19 (3.97%) Spring Boot
8 15 (3.14%) SOAP
9 14 (2.93%) React
9 14 (2.93%) Spark Streaming
10 13 (2.72%) Apache NiFi
11 12 (2.51%) .NET
11 12 (2.51%) scikit-learn
12 11 (2.30%) Node.js
13 10 (2.09%) Elastic Stack
14 9 (1.88%) TensorFlow
15 8 (1.67%) .NET Core
15 8 (1.67%) GraphQL
Miscellaneous
1 43 (9.00%) Analytical Skills
2 30 (6.28%) Algorithms
3 16 (3.35%) Greenfield Project
4 15 (3.14%) Public Cloud
5 12 (2.51%) Cloud Native
6 11 (2.30%) Data Structures
6 11 (2.30%) FMCG
7 10 (2.09%) Credit Risk
7 10 (2.09%) Derivative
8 9 (1.88%) Analytical Mindset
8 9 (1.88%) Self-Motivation
8 9 (1.88%) User Experience
9 8 (1.67%) PKI
9 8 (1.67%) Reference Data
10 7 (1.46%) Data Centre
10 7 (1.46%) Fixed Income
11 5 (1.05%) Virtual Team
12 4 (0.84%) Hedge funds
12 4 (0.84%) Video Conferencing
13 3 (0.63%) Taxonomies
Operating Systems
1 69 (14.44%) Linux
2 32 (6.69%) Unix
3 21 (4.39%) Windows
4 15 (3.14%) CentOS
4 15 (3.14%) Red Hat Enterprise Linux
5 5 (1.05%) Android
5 5 (1.05%) Apple iOS
5 5 (1.05%) Debian
5 5 (1.05%) Mac OS X
5 5 (1.05%) Ubuntu
6 1 (0.21%) Windows Server
Processes & Methodologies
1 158 (33.05%) DevOps
2 120 (25.10%) ETL
3 111 (23.22%) Analytics
4 107 (22.38%) Agile Software Development
5 68 (14.23%) Data Science
6 67 (14.02%) Microservices
7 65 (13.60%) Data Modelling
8 54 (11.30%) Visualisation
9 53 (11.09%) Business Intelligence
10 52 (10.88%) Machine Learning
11 47 (9.83%) Data Engineering
12 45 (9.41%) Data Analytics
13 42 (8.79%) TDD
14 41 (8.58%) Data Integration
14 41 (8.58%) Problem-Solving
15 38 (7.95%) Scrum
16 37 (7.74%) Data Ingestion
17 36 (7.53%) Computer Science
17 36 (7.53%) Continuous Integration
18 34 (7.11%) Data Visualisation
Programming Languages
1 245 (51.26%) Python
2 161 (33.68%) SQL
3 139 (29.08%) Java
4 82 (17.15%) Scala
5 42 (8.79%) Bash Shell
6 37 (7.74%) R
7 29 (6.07%) Shell Script
8 27 (5.65%) PowerShell
9 22 (4.60%) JavaScript
10 20 (4.18%) Perl
11 19 (3.97%) PL/SQL
12 18 (3.77%) C#
13 15 (3.14%) Groovy
14 14 (2.93%) C++
15 12 (2.51%) Ruby
16 10 (2.09%) Java 8
16 10 (2.09%) Kotlin
17 8 (1.67%) T-SQL
18 6 (1.26%) Clojure
19 5 (1.05%) TypeScript
Qualifications
1 147 (30.75%) Security Cleared
2 129 (26.99%) SC Cleared
3 40 (8.37%) Degree
4 19 (3.97%) Numerate Degree
5 16 (3.35%) DV Cleared
6 15 (3.14%) CCA
7 13 (2.72%) TOGAF Certification
8 11 (2.30%) Computer Science Degree
9 8 (1.67%) Master's Degree
10 6 (1.26%) AWS Certification
11 4 (0.84%) ISEB
11 4 (0.84%) ISTQB
11 4 (0.84%) ISTQB Foundation Certification
12 3 (0.63%) BPSS Clearance
12 3 (0.63%) PhD
13 1 (0.21%) CSPO
Quality Assurance & Compliance
1 13 (2.72%) Data Quality
2 9 (1.88%) SLA
3 4 (0.84%) QA
4 2 (0.42%) GDPR
4 2 (0.42%) MiFID
4 2 (0.42%) Sarbanes-Oxley
5 1 (0.21%) CDISC
5 1 (0.21%) IFRS
5 1 (0.21%) ISO 20022
5 1 (0.21%) PCI DSS
5 1 (0.21%) SEPA
5 1 (0.21%) Solvency II
5 1 (0.21%) TARGET2
System Software
1 94 (19.67%) Docker
2 91 (19.04%) HDFS
3 27 (5.65%) Apache ZooKeeper
4 19 (3.97%) Active Directory
5 9 (1.88%) Apache Flume
6 3 (0.63%) Active Directory Lightweight Directory Services
6 3 (0.63%) NFS
7 1 (0.21%) VMware Infrastructure
Systems Management
1 95 (19.87%) Kubernetes
2 87 (18.20%) Ansible
3 71 (14.85%) Terraform
4 36 (7.53%) Puppet
5 11 (2.30%) Packer
6 8 (1.67%) Hadoop YARN
7 7 (1.46%) Oozie
8 6 (1.26%) Opscode Chef
9 5 (1.05%) Salt
10 3 (0.63%) Ambari
10 3 (0.63%) logstash
11 2 (0.42%) Grafana
11 2 (0.42%) Kibana
11 2 (0.42%) Kubeflow
11 2 (0.42%) Rancher
11 2 (0.42%) ZABBIX
12 1 (0.21%) Istio
12 1 (0.21%) Knative
12 1 (0.21%) Prometheus
12 1 (0.21%) SCCM
Vendors
1 141 (29.50%) Cloudera
2 92 (19.25%) Oracle
3 52 (10.88%) Hortonworks
4 38 (7.95%) Tableau
5 31 (6.49%) Teradata
6 27 (5.65%) Microsoft
7 26 (5.44%) Databricks
8 19 (3.97%) Informatica
9 18 (3.77%) SAP
10 16 (3.35%) Google
11 15 (3.14%) Red Hat
12 12 (2.51%) Ab Initio
12 12 (2.51%) Splunk
13 11 (2.30%) VMware
14 10 (2.09%) StreamSets
15 9 (1.88%) MuleSoft
16 8 (1.67%) IBM
16 8 (1.67%) TIBCO
17 6 (1.26%) AppDynamics
17 6 (1.26%) Sparx