Period
to 2 June 2020

The following table provides summary statistics for contract job vacancies with a requirement for Hadoop skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Hadoop over the 6 months to 2 June 2020 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Hadoop
UK
6 months to
2 Jun 2020
Same period 2019 Same period 2018
Rank 178 131 120
Rank change year-on-year -47 -11 -1
Contract jobs citing Hadoop 655 1,794 2,185
As % of all contract jobs advertised in the UK 1.44% 1.98% 2.15%
As % of the Database & Business Intelligence category 6.79% 8.85% 9.55%
Number of daily rates quoted 474 1,246 1,569
Median daily rate £550 £550 £550
10th Percentile £410 £413 £400
90th Percentile £725 £713 £675
UK excluding London median daily rate £525 £513 £500
% change year-on-year +2.44% +2.50% +8.11%
Number of hourly rates quoted 9 18 10
Median hourly rate £35.00 £49.13 £46.25
Median hourly rate % change year-on-year -28.76% +6.23% -1.60%
UK excluding London median hourly rate £35.00 £52.00 £37.50
% change year-on-year -32.69% +38.67% -20.21%

Hadoop is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Contract vacancies with a requirement for database or business intelligence skills 9,651 20,262 22,891
As % of all contract IT jobs advertised in the UK 21.21% 22.35% 22.48%
Number of daily rates quoted 6,313 13,850 15,930
Median daily rate £490 £475 £473
Median daily rate % change year-on-year +3.16% +0.53% +8.00%
10th Percentile £325 £325 £313
90th Percentile £675 £650 £625
UK excluding London median daily rate £450 £430 £413
% change year-on-year +4.65% +4.24% +4.43%
Number of hourly rates quoted 194 249 239
Median hourly rate £35.00 £35.00 £34.50
Median hourly rate % change year-on-year - +1.45% +15.71%
10th Percentile £16.00 £14.22 £14.00
90th Percentile £53.00 £57.50 £53.55
Median hourly rate £35.00 £32.50 £35.00
% change year-on-year +7.69% -7.14% +20.69%

Hadoop
Job Vacancy Trend

Job postings citing Hadoop as a proportion of all IT jobs advertised.

Job vacancy trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Trend

3-month moving average daily rate quoted in jobs citing Hadoop.

Daily rate trend for Hadoop in the UK

Hadoop
Daily Rate Histogram

Daily rate distribution for jobs citing Hadoop over the 6 months to 2 June 2020.

Daily rate histogram for Hadoop in the UK

Hadoop
Contractor Hourly Rate Trend

3-month moving average hourly rates quoted in jobs citing Hadoop.

Hourly rate trend for Hadoop in the UK

Hadoop
Top 16 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Hadoop within the UK over the 6 months to 2 June 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -44 601 £550 - 108
London -47 336 £550 -4.35% 75
UK excluding London -20 303 £525 +2.44% 47
North of England -26 86 £625 +13.64% 9
South East -12 71 £500 - 10
North West +18 60 £650 +8.33% 5
South West +16 48 £560 +10.94% 5
Midlands +46 28 £396 -24.52% 2
Wales +46 27 £500 -4.76% 9
East of England +44 24 £450 -20.00% 7
Work from Home -10 23 £600 +4.35% 19
West Midlands +52 22 £437 -27.21%
Scotland -4 21 £475 +8.57% 5
Yorkshire -21 18 £475 -5.00% 4
East Midlands +26 5 £350 -27.27% 2
North East +17 5 £750 +53.85%

For the 6 months to 2 June 2020, IT contractor jobs citing Hadoop also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for Hadoop.

1 379 (57.86%) Big Data
2 349 (53.28%) Python
3 335 (51.15%) Apache Spark
4 242 (36.95%) SQL
5 202 (30.84%) AWS
6 194 (29.62%) Agile Software Development
7 193 (29.47%) Analytics
8 189 (28.85%) DevOps
9 187 (28.55%) Apache Hive
10 184 (28.09%) Java
11 173 (26.41%) Cloudera
12 162 (24.73%) Kafka
13 153 (23.36%) Scala
14 149 (22.75%) Azure
15 136 (20.76%) ETL
16 120 (18.32%) Data Modelling
17 117 (17.86%) Docker
17 117 (17.86%) Kubernetes
18 116 (17.71%) Jenkins
19 107 (16.34%) Security Cleared
20 105 (16.03%) Finance
21 100 (15.27%) Hortonworks
22 99 (15.11%) HBase
23 98 (14.96%) HDFS
24 97 (14.81%) NoSQL
25 96 (14.66%) Data Science
26 94 (14.35%) Business Intelligence
27 92 (14.05%) Linux
28 87 (13.28%) Terraform
29 85 (12.98%) Data Warehouse

Hadoop
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 335 (51.15%) Apache Spark
2 63 (9.62%) Apache
3 38 (5.80%) SAS
4 25 (3.82%) Confluence
5 20 (3.05%) Apache Pig
5 20 (3.05%) Elasticsearch
6 19 (2.90%) Jupyter
7 14 (2.14%) Apache Airflow
8 12 (1.83%) Tomcat
9 5 (0.76%) WebLogic
10 3 (0.46%) Apache Pulsar
10 3 (0.46%) Oracle Workflow
11 2 (0.31%) Jetty
11 2 (0.31%) Luigi
11 2 (0.31%) Tornado
12 1 (0.15%) Apache Solr
12 1 (0.15%) nginx
12 1 (0.15%) OpenStack
12 1 (0.15%) Oracle SOA Suite
12 1 (0.15%) WebSphere
Applications
1 7 (1.07%) Microsoft Excel
2 5 (0.76%) Microsoft Office
3 2 (0.31%) SPSS
4 1 (0.15%) Chatbot
4 1 (0.15%) Stata
Business Applications
1 4 (0.61%) SAP CRM
1 4 (0.61%) SAP ERP
1 4 (0.61%) SAP IS-U
2 3 (0.46%) SAP FI-CA
2 3 (0.46%) SAP PSCD
3 2 (0.31%) Hyperion Planning
3 2 (0.31%) SAP Oil and Gas
4 1 (0.15%) ENOVIA
4 1 (0.15%) FINCAD Analytics
4 1 (0.15%) Shopify
Cloud Services
1 202 (30.84%) AWS
2 149 (22.75%) Azure
3 84 (12.82%) GCP
4 36 (5.50%) Power Platform
5 27 (4.12%) PaaS
6 20 (3.05%) Cloud Computing
6 20 (3.05%) GitHub
7 15 (2.29%) Amazon S3
7 15 (2.29%) IaaS
8 14 (2.14%) Serverless
9 12 (1.83%) Amazon EMR
9 12 (1.83%) Amazon Kinesis
10 11 (1.68%) AWS Lambda
11 8 (1.22%) AWS CloudFormation
12 6 (0.92%) Amazon SageMaker
12 6 (0.92%) Virtual Private Cloud
13 5 (0.76%) Azure Data Factory
14 4 (0.61%) Amazon Glacier
14 4 (0.61%) OpenShift
15 3 (0.46%) Amazon EC2
Communications & Networking
1 22 (3.36%) Kerberos
2 7 (1.07%) HTTP
3 6 (0.92%) VPN
4 5 (0.76%) DNS
4 5 (0.76%) SSL
5 4 (0.61%) 5G
5 4 (0.61%) HTTPS
6 3 (0.46%) Firewall
6 3 (0.46%) SAN
7 2 (0.31%) Internet
7 2 (0.31%) TCP/IP
8 1 (0.15%) Broadband
8 1 (0.15%) MQTT
8 1 (0.15%) OpenLDAP
8 1 (0.15%) Skype
Database & Business Intelligence
1 379 (57.86%) Big Data
2 187 (28.55%) Apache Hive
3 99 (15.11%) HBase
4 97 (14.81%) NoSQL
5 85 (12.98%) Data Warehouse
6 79 (12.06%) Impala
7 68 (10.38%) Data Lake
7 68 (10.38%) Relational Database
8 52 (7.94%) SQL Server
9 50 (7.63%) Apache Cassandra
10 47 (7.18%) MongoDB
11 40 (6.11%) BigQuery
12 38 (5.80%) PostgreSQL
13 36 (5.50%) Power BI
14 31 (4.73%) Amazon Redshift
15 25 (3.82%) Sqoop
16 23 (3.51%) Spark SQL
17 22 (3.36%) Pentaho
17 22 (3.36%) RDBMS
18 21 (3.21%) Data Mining
Development Applications
1 116 (17.71%) Jenkins
2 82 (12.52%) Git (software)
3 44 (6.72%) JIRA
4 41 (6.26%) Maven
5 28 (4.27%) GitLab
6 26 (3.97%) Bitbucket
7 25 (3.82%) Artifactory
8 22 (3.36%) TeamCity
9 19 (2.90%) Sonatype Nexus
10 13 (1.98%) IBM UrbanCode
11 12 (1.83%) Visual Studio Team System
12 11 (1.68%) Team Foundation Server
12 11 (1.68%) Visual Studio
13 8 (1.22%) CircleCI
14 7 (1.07%) Subversion
15 6 (0.92%) Selenium
15 6 (0.92%) SonarQube
16 5 (0.76%) Cucumber
16 5 (0.76%) Mercurial
16 5 (0.76%) Octopus Deploy
General
1 105 (16.03%) Finance
2 54 (8.24%) Banking
3 35 (5.34%) Telecoms
4 31 (4.73%) Public Sector
4 31 (4.73%) Retail
5 19 (2.90%) Marketing
6 15 (2.29%) Investment Banking
7 13 (1.98%) Legal
8 12 (1.83%) Aerospace
9 11 (1.68%) Front Office
10 7 (1.07%) Retail Banking
11 6 (0.92%) Manufacturing
12 4 (0.61%) Games
12 4 (0.61%) Pharmaceutical
13 3 (0.46%) Electronics
13 3 (0.46%) Law
14 2 (0.31%) Billing
14 2 (0.31%) Financial Institution
15 1 (0.15%) Automotive
15 1 (0.15%) Humanities
Job Titles
1 188 (28.70%) Data Engineer
2 126 (19.24%) Developer
3 84 (12.82%) Architect
4 77 (11.76%) Big Data Engineer
5 68 (10.38%) DevOps Engineer
6 55 (8.40%) Python Engineer
7 53 (8.09%) Data Architect
8 43 (6.56%) Data Scientist
9 40 (6.11%) Java Developer
10 38 (5.80%) Hadoop Engineer
11 33 (5.04%) Analyst
12 32 (4.89%) Cloud Engineer
13 29 (4.43%) Python Developer
14 27 (4.12%) AWS Engineer
15 24 (3.66%) Administrator
16 23 (3.51%) Senior Data Scientist
17 21 (3.21%) Azure Engineer
17 21 (3.21%) Big Data Developer
17 21 (3.21%) Cloud Architect
18 20 (3.05%) Hadoop Developer
Libraries, Frameworks & Software Standards
1 162 (24.73%) Kafka
2 78 (11.91%) PySpark
3 58 (8.85%) TensorFlow
4 49 (7.48%) Web Services
5 43 (6.56%) scikit-learn
6 28 (4.27%) Apache NiFi
7 27 (4.12%) Spring
8 26 (3.97%) Natural Language Processing
9 24 (3.66%) XML
10 21 (3.21%) Spark Streaming
11 19 (2.90%) H2O
11 19 (2.90%) React
12 18 (2.75%) Pandas
13 16 (2.44%) Spring Boot
14 15 (2.29%) .NET
14 15 (2.29%) LDAP
14 15 (2.29%) Node.js
15 14 (2.14%) NumPy
15 14 (2.14%) RESTful
16 13 (1.98%) AngularJS
Miscellaneous
1 56 (8.55%) Analytical Skills
2 44 (6.72%) Algorithms
3 22 (3.36%) Greenfield Project
4 18 (2.75%) Public Cloud
5 16 (2.44%) Cloud Native
5 16 (2.44%) Management Information System
6 15 (2.29%) Data Structures
7 14 (2.14%) Credit Risk
8 12 (1.83%) PKI
8 12 (1.83%) Self-Motivation
9 11 (1.68%) Reference Data
10 10 (1.53%) Social Media
11 9 (1.37%) Data Centre
12 8 (1.22%) Enterprise Software
12 8 (1.22%) Hedge funds
13 6 (0.92%) Derivative
13 6 (0.92%) Fixed Income
14 5 (0.76%) Analytical Mindset
14 5 (0.76%) Linux Command Line
14 5 (0.76%) User Experience
Operating Systems
1 92 (14.05%) Linux
2 42 (6.41%) Unix
3 24 (3.66%) CentOS
4 22 (3.36%) Red Hat Enterprise Linux
4 22 (3.36%) Windows
5 5 (0.76%) Debian
5 5 (0.76%) Ubuntu
6 4 (0.61%) Windows Server
7 1 (0.15%) DC/OS
7 1 (0.15%) VME
Processes & Methodologies
1 194 (29.62%) Agile Software Development
2 193 (29.47%) Analytics
3 189 (28.85%) DevOps
4 136 (20.76%) ETL
5 120 (18.32%) Data Modelling
6 96 (14.66%) Data Science
7 94 (14.35%) Business Intelligence
8 83 (12.67%) Data Engineering
9 82 (12.52%) Machine Learning
10 77 (11.76%) Scrum
11 71 (10.84%) Data Analytics
12 69 (10.53%) Software Engineering
13 63 (9.62%) Visualisation
14 59 (9.01%) Continuous Integration
15 51 (7.79%) Data Management
16 50 (7.63%) Microservices
16 50 (7.63%) Migration
17 46 (7.02%) Open Source
18 45 (6.87%) Computer Science
18 45 (6.87%) Data Analysis
Programming Languages
1 349 (53.28%) Python
2 242 (36.95%) SQL
3 184 (28.09%) Java
4 153 (23.36%) Scala
5 44 (6.72%) Bash Shell
6 43 (6.56%) R
7 36 (5.50%) Shell Script
8 29 (4.43%) PL/SQL
9 27 (4.12%) JavaScript
10 26 (3.97%) Ruby
11 22 (3.36%) C#
12 18 (2.75%) C
12 18 (2.75%) Java 8
13 16 (2.44%) PowerShell
14 15 (2.29%) C++
15 13 (1.98%) Perl
16 12 (1.83%) Go
17 11 (1.68%) T-SQL
18 8 (1.22%) Groovy
19 7 (1.07%) Korn
Qualifications
1 107 (16.34%) Security Cleared
2 84 (12.82%) SC Cleared
3 40 (6.11%) Degree
4 25 (3.82%) Computer Science Degree
5 24 (3.66%) CCA
6 22 (3.36%) DV Cleared
7 6 (0.92%) Master's Degree
8 4 (0.61%) AWS Certification
8 4 (0.61%) PMI Certification
9 3 (0.46%) ISEB
9 3 (0.46%) ISTQB
9 3 (0.46%) ISTQB Foundation Certification
9 3 (0.46%) Numerate Degree
10 2 (0.31%) BPSS Clearance
10 2 (0.31%) DBS Check
10 2 (0.31%) PhD
11 1 (0.15%) CCIE
11 1 (0.15%) Cisco Certification
11 1 (0.15%) MBA
Quality Assurance & Compliance
1 27 (4.12%) Data Quality
2 4 (0.61%) SLA
3 3 (0.46%) QA
4 2 (0.31%) COBIT
4 2 (0.31%) GDPR
4 2 (0.31%) ISO/IEC 27002 (supersedes ISO/IEC 17799)
5 1 (0.15%) IFRS
5 1 (0.15%) ISO/IEC 27001
5 1 (0.15%) MiFID
System Software
1 117 (17.86%) Docker
2 98 (14.96%) HDFS
3 22 (3.36%) Apache Flume
4 21 (3.21%) Apache ZooKeeper
5 12 (1.83%) Active Directory
6 11 (1.68%) Virtual Machines
7 3 (0.46%) Virtual Desktop
7 3 (0.46%) Virtual Servers
Systems Management
1 117 (17.86%) Kubernetes
2 87 (13.28%) Terraform
3 72 (10.99%) Ansible
4 34 (5.19%) Hadoop YARN
4 34 (5.19%) Puppet
5 26 (3.97%) Oozie
6 8 (1.22%) Packer
7 7 (1.07%) Ambari
7 7 (1.07%) logstash
8 6 (0.92%) Kibana
8 6 (0.92%) Single Sign-On
9 4 (0.61%) Grafana
10 3 (0.46%) Kubeflow
10 3 (0.46%) Opscode Chef
11 2 (0.31%) Control-M
11 2 (0.31%) Netcool
12 1 (0.15%) HAProxy
12 1 (0.15%) Marathon
12 1 (0.15%) Tivoli
12 1 (0.15%) ZABBIX
Vendors
1 173 (26.41%) Cloudera
2 100 (15.27%) Hortonworks
3 72 (10.99%) Oracle
4 54 (8.24%) Microsoft
5 45 (6.87%) Tableau
6 32 (4.89%) Informatica
7 31 (4.73%) Google
8 25 (3.82%) SAP
9 24 (3.66%) Teradata
10 22 (3.36%) Databricks
11 21 (3.21%) MapR
12 17 (2.60%) Red Hat
13 13 (1.98%) Ab Initio
13 13 (1.98%) IBM
14 11 (1.68%) VMware
15 10 (1.53%) StreamSets
16 8 (1.22%) Sybase
17 7 (1.07%) Splunk
17 7 (1.07%) Talend
18 4 (0.61%) Trillium