Period
to 21 January 2020

The following table provides summary statistics for contract job vacancies with a requirement for Hadoop skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Hadoop over the 6 months to 21 January 2020 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Hadoop
UK
6 months to
21 Jan 2020
Same period 2019 Same period 2018
Rank 186 128 126
Rank change year-on-year -58 -2 -18
Contract jobs citing Hadoop 1,044 2,039 2,080
As % of all contract IT jobs advertised in the UK 1.43% 2.06% 2.11%
As % of the Database & Business Intelligence category 6.81% 9.18% 9.58%
Number of daily rates quoted 710 1,327 1,471
UK median daily rate £550 £550 £540
Median daily rate % change year-on-year - +1.85% +2.76%
10th Percentile £413 £413 £400
90th Percentile £719 £713 £675
UK excluding London median daily rate £525 £525 £488
% change year-on-year - +7.69% -2.50%
Number of hourly rates quoted 0 4 9
UK median hourly rate - £50.00 £35.00
Median hourly rate % change year-on-year - +42.86% -23.91%
UK excluding London median hourly rate - £35.00 £32.75
% change year-on-year - +6.87% -28.42%

Hadoop is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Contract vacancies with a requirement for database or business intelligence skills 15,340 22,200 21,719
As % of all contract IT jobs advertised in the UK 21.05% 22.46% 21.99%
Number of daily rates quoted 10,224 15,300 15,116
UK median daily rate £475 £475 £450
Median daily rate % change year-on-year - +5.56% +5.88%
10th Percentile £319 £319 £310
90th Percentile £663 £644 £625
UK excluding London median daily rate £438 £425 £400
% change year-on-year +2.94% +6.25% +3.23%
Number of hourly rates quoted 246 243 250
UK median hourly rate £31.25 £32.50 £30.63
Median hourly rate % change year-on-year -3.85% +6.12% +9.38%
10th Percentile £12.90 £13.34 £14.54
90th Percentile £54.38 £57.06 £55.72
UK excluding London median hourly rate £27.00 £32.75 £35.00
% change year-on-year -17.56% -6.43% +32.08%

Hadoop
Job Vacancy Trend

Job postings citing Hadoop as a percentage of all IT jobs advertised.

Job vacancy trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Hadoop.

Contractor daily rate trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing Hadoop over the 6 months to 21 January 2020.

Contractor daily rate histogram for Hadoop in the UK

Hadoop
Contractor Hourly Rate Trend

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing Hadoop.

Contractor hourly rate trend for Hadoop in the UK

Hadoop
Top 15 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Hadoop within the UK over the 6 months to 21 January 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -68 950 £550 - 304
London -36 562 £575 +1.10% 172
UK excluding London -36 464 £525 - 153
South East -79 131 £480 -8.57% 46
North of England +9 129 £550 +3.77% 21
North West +24 80 £550 -10.20% 11
Scotland +31 65 £525 +10.53% 14
South West -39 47 £554 +5.43% 27
East of England -1 42 £600 +9.09% 17
Yorkshire -7 40 £500 +5.26% 9
Midlands -6 34 £488 -1.42% 21
West Midlands +22 20 £525 +5.58% 16
Wales +21 16 £500 -0.99% 6
East Midlands -4 14 £475 -1.30% 5
North East +23 10 £663 +32.50% 1

For the 6 months to 21 January 2020, IT contractor jobs citing Hadoop also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for Hadoop.

1 628 (60.15%) Big Data
2 543 (52.01%) Apache Spark
3 506 (48.47%) Python
4 410 (39.27%) SQL
5 357 (34.20%) Java
6 324 (31.03%) Kafka
7 304 (29.12%) AWS
8 281 (26.92%) Agile Software Development
9 270 (25.86%) Analytics
10 266 (25.48%) Apache Hive
11 261 (25.00%) Scala
12 248 (23.75%) ETL
13 218 (20.88%) Finance
14 206 (19.73%) Azure
15 191 (18.30%) Data Science
16 184 (17.62%) Cloudera
17 182 (17.43%) DevOps
18 179 (17.15%) Linux
19 175 (16.76%) NoSQL
20 174 (16.67%) Data Modelling
21 165 (15.80%) Docker
22 161 (15.42%) HDFS
23 154 (14.75%) Jenkins
24 152 (14.56%) Data Warehouse
25 150 (14.37%) HBase
26 146 (13.98%) Machine Learning
26 146 (13.98%) Kubernetes
26 146 (13.98%) Oracle
27 139 (13.31%) Business Intelligence
28 125 (11.97%) Hortonworks

Hadoop
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 543 (52.01%) Apache Spark
2 80 (7.66%) Apache
3 77 (7.38%) SAS
4 52 (4.98%) Elasticsearch
5 32 (3.07%) Jupyter
6 29 (2.78%) Confluence
7 24 (2.30%) Apache Airflow
8 18 (1.72%) Tomcat
9 16 (1.53%) Apache Pig
9 16 (1.53%) JBoss
10 9 (0.86%) OpenStack
11 8 (0.77%) Jetty
11 8 (0.77%) Luigi
12 6 (0.57%) SharePoint
12 6 (0.57%) WebLogic
13 5 (0.48%) Apache Solr
14 3 (0.29%) nginx
15 2 (0.19%) CMS
15 2 (0.19%) Pachyderm
15 2 (0.19%) SharePoint 2013
Applications
1 19 (1.82%) Microsoft Excel
2 6 (0.57%) Microsoft Office
3 3 (0.29%) SPSS
4 2 (0.19%) Spreadsheet
5 1 (0.096%) Chatbot
5 1 (0.096%) Microsoft Project
5 1 (0.096%) Stata
Business Applications
1 10 (0.96%) FINCAD Analytics
2 6 (0.57%) SAP Oil and Gas
3 4 (0.38%) SAP CRM
3 4 (0.38%) SAP ERP
3 4 (0.38%) SAP IS-U
4 3 (0.29%) SAP FI-CA
4 3 (0.29%) SAP PSCD
4 3 (0.29%) Sparx Enterprise Architect
5 2 (0.19%) Hyperion Planning
5 2 (0.19%) Oracle Subledger Accounting
6 1 (0.096%) ENOVIA
6 1 (0.096%) NetSuite
6 1 (0.096%) SAP MDM
Cloud Services
1 304 (29.12%) AWS
2 206 (19.73%) Azure
3 111 (10.63%) GCP
4 59 (5.65%) Amazon EMR
5 53 (5.08%) Power Platform
6 50 (4.79%) Amazon S3
7 36 (3.45%) Serverless
8 35 (3.35%) Amazon Kinesis
9 34 (3.26%) GitHub
10 31 (2.97%) AWS Lambda
11 21 (2.01%) OpenShift
12 17 (1.63%) Cloud Computing
13 16 (1.53%) Amazon SQS
13 16 (1.53%) AWS Step Functions
14 14 (1.34%) Azure Data Factory
15 13 (1.25%) Amazon EC2
15 13 (1.25%) Azure Machine Learning
16 11 (1.05%) Amazon EKS
16 11 (1.05%) PaaS
17 10 (0.96%) Virtual Private Cloud
Communications & Networking
1 26 (2.49%) Kerberos
2 14 (1.34%) Firewall
3 13 (1.25%) SSH
4 10 (0.96%) SAN
5 8 (0.77%) SSL
6 7 (0.67%) HTTP
7 6 (0.57%) Internet
7 6 (0.57%) VPN
8 5 (0.48%) Wi-Fi
9 4 (0.38%) 5G
9 4 (0.38%) F5 BIG-IP GTM
9 4 (0.38%) F5 BIG-IP LTM
9 4 (0.38%) Radio Access Network
10 3 (0.29%) 4G
10 3 (0.29%) DNS
10 3 (0.29%) HTTPS
10 3 (0.29%) TCP/IP
10 3 (0.29%) Wireshark
11 2 (0.19%) Extranet
11 2 (0.19%) Intranet
Database & Business Intelligence
1 628 (60.15%) Big Data
2 266 (25.48%) Apache Hive
3 175 (16.76%) NoSQL
4 152 (14.56%) Data Warehouse
5 150 (14.37%) HBase
6 125 (11.97%) Relational Database
7 107 (10.25%) Data Lake
8 104 (9.96%) Impala
8 104 (9.96%) MongoDB
9 92 (8.81%) PostgreSQL
10 87 (8.33%) SQL Server
11 67 (6.42%) MySQL
12 66 (6.32%) Amazon Redshift
13 60 (5.75%) Apache Cassandra
14 56 (5.36%) Spark SQL
15 53 (5.08%) Power BI
15 53 (5.08%) RDBMS
16 49 (4.69%) Flink
17 44 (4.21%) BigQuery
18 41 (3.93%) Amazon Athena
Development Applications
1 154 (14.75%) Jenkins
2 124 (11.88%) Git (software)
3 62 (5.94%) JIRA
4 44 (4.21%) Maven
5 27 (2.59%) GitLab
6 21 (2.01%) Sonatype Nexus
7 20 (1.92%) Cucumber
8 18 (1.72%) Atlassian Bamboo
9 14 (1.34%) Artifactory
9 14 (1.34%) Team Foundation Server
9 14 (1.34%) Vagrant
10 13 (1.25%) Subversion
11 12 (1.15%) IBM UrbanCode
12 11 (1.05%) ERwin
12 11 (1.05%) Mercurial
12 11 (1.05%) Selenium
13 10 (0.96%) ER Studio
13 10 (0.96%) Gradle
13 10 (0.96%) PowerDesigner
14 9 (0.86%) SonarQube
General
1 218 (20.88%) Finance
2 90 (8.62%) Banking
3 45 (4.31%) Retail
4 28 (2.68%) Front Office
4 28 (2.68%) Investment Banking
5 27 (2.59%) Marketing
6 26 (2.49%) Public Sector
7 21 (2.01%) Telecoms
8 13 (1.25%) Pharmaceutical
9 9 (0.86%) Manufacturing
10 8 (0.77%) Games
10 8 (0.77%) Legal
11 7 (0.67%) Retail Banking
12 5 (0.48%) Automotive
13 3 (0.29%) Financial Institution
13 3 (0.29%) International Banking
14 2 (0.19%) Advertising
14 2 (0.19%) Aerospace
14 2 (0.19%) Billing
15 1 (0.096%) Publishing
Job Titles
1 285 (27.30%) Data Engineer
2 216 (20.69%) Developer
3 134 (12.84%) Architect
4 97 (9.29%) Data Architect
5 95 (9.10%) Big Data Engineer
6 81 (7.76%) Analyst
7 68 (6.51%) Administrator
8 66 (6.32%) Data Scientist
9 56 (5.36%) Java Developer
10 49 (4.69%) DevOps Engineer
11 44 (4.21%) Data Analyst
11 44 (4.21%) Python Developer
11 44 (4.21%) Scala Developer
12 42 (4.02%) Big Data Architect
12 42 (4.02%) Java Engineer
12 42 (4.02%) Python Engineer
13 41 (3.93%) Big Data Developer
14 39 (3.74%) Hadoop Engineer
15 36 (3.45%) Hadoop Developer
16 35 (3.35%) AWS Engineer
Libraries, Frameworks & Software Standards
1 324 (31.03%) Kafka
2 91 (8.72%) PySpark
3 58 (5.56%) Spring
4 49 (4.69%) React
5 48 (4.60%) Apache NiFi
6 46 (4.41%) Spark Streaming
7 44 (4.21%) Web Services
8 40 (3.83%) XML
9 36 (3.45%) Pandas
10 31 (2.97%) RabbitMQ
11 28 (2.68%) TensorFlow
12 27 (2.59%) REST
12 27 (2.59%) scikit-learn
13 26 (2.49%) AngularJS
13 26 (2.49%) JSON
14 25 (2.39%) Node.js
14 25 (2.39%) RESTful
15 24 (2.30%) NumPy
15 24 (2.30%) Spring Boot
16 22 (2.11%) Hibernate
Miscellaneous
1 100 (9.58%) Analytical Skills
2 85 (8.14%) Algorithms
3 41 (3.93%) Greenfield Project
4 36 (3.45%) Data Structures
5 24 (2.30%) Management Information System
6 23 (2.20%) Public Cloud
7 18 (1.72%) Self-Motivation
8 15 (1.44%) Hedge funds
9 14 (1.34%) Derivative
9 14 (1.34%) Linux Command Line
9 14 (1.34%) User Experience
10 13 (1.25%) Fixed Income
10 13 (1.25%) Reference Data
11 12 (1.15%) Credit Risk
12 11 (1.05%) Data Centre
13 9 (0.86%) Replication
14 8 (0.77%) Cloud Native
15 7 (0.67%) Clustering
15 7 (0.67%) Grid Computing
15 7 (0.67%) Market Risk
Operating Systems
1 179 (17.15%) Linux
2 92 (8.81%) Unix
3 48 (4.60%) Windows
4 36 (3.45%) Red Hat Enterprise Linux
5 32 (3.07%) CentOS
6 21 (2.01%) Windows Server
7 17 (1.63%) Ubuntu
8 10 (0.96%) Debian
9 4 (0.38%) Android
9 4 (0.38%) Apple iOS
10 3 (0.29%) Solaris
11 2 (0.19%) Mac OS X
11 2 (0.19%) SUSE Linux Enterprise Server
12 1 (0.096%) AIX
12 1 (0.096%) CoreOS
12 1 (0.096%) Oracle Linux
12 1 (0.096%) VME
12 1 (0.096%) Windows Server 2012
12 1 (0.096%) zOS
Processes & Methodologies
1 281 (26.92%) Agile Software Development
2 270 (25.86%) Analytics
3 248 (23.75%) ETL
4 191 (18.30%) Data Science
5 182 (17.43%) DevOps
6 174 (16.67%) Data Modelling
7 146 (13.98%) Machine Learning
8 139 (13.31%) Business Intelligence
9 113 (10.82%) Migration
10 108 (10.34%) Software Engineering
11 107 (10.25%) Data Architecture
12 98 (9.39%) Visualisation
13 95 (9.10%) Continuous Integration
14 94 (9.00%) Microservices
15 93 (8.91%) Computer Science
16 89 (8.52%) Data Engineering
17 87 (8.33%) Open Source
17 87 (8.33%) Problem-Solving
18 83 (7.95%) Data Analytics
19 82 (7.85%) Data Management
Programming Languages
1 506 (48.47%) Python
2 410 (39.27%) SQL
3 357 (34.20%) Java
4 261 (25.00%) Scala
5 86 (8.24%) R
6 63 (6.03%) Bash Shell
7 57 (5.46%) JavaScript
8 45 (4.31%) C
9 42 (4.02%) C#
9 42 (4.02%) Shell Script
10 41 (3.93%) C++
11 29 (2.78%) Perl
11 29 (2.78%) PL/SQL
12 28 (2.68%) Go
12 28 (2.68%) Java 8
13 26 (2.49%) Ruby
14 24 (2.30%) PowerShell
15 17 (1.63%) Kotlin
16 12 (1.15%) Groovy
17 11 (1.05%) T-SQL
Qualifications
1 91 (8.72%) Security Cleared
2 88 (8.43%) Degree
3 59 (5.65%) SC Cleared
4 40 (3.83%) Computer Science Degree
5 31 (2.97%) DV Cleared
6 21 (2.01%) PhD
7 11 (1.05%) CCA
8 10 (0.96%) Master's Degree
9 9 (0.86%) Numerate Degree
10 7 (0.67%) BPSS Clearance
10 7 (0.67%) Cisco Certification
11 5 (0.48%) CCIE
11 5 (0.48%) DBS Check
11 5 (0.48%) ISTQB
11 5 (0.48%) SAS Certification
12 4 (0.38%) AWS Certification
12 4 (0.38%) CCNP
12 4 (0.38%) TOGAF Certification
13 3 (0.29%) ISEB
13 3 (0.29%) ISTQB Foundation Certification
Quality Assurance & Compliance
1 47 (4.50%) Data Quality
2 20 (1.92%) QA
3 6 (0.57%) Sarbanes-Oxley
4 5 (0.48%) PMO
5 4 (0.38%) Disclosure Scotland
5 4 (0.38%) GDPR
5 4 (0.38%) GxP
5 4 (0.38%) ISO/IEC 27001
6 2 (0.19%) PCI DSS
System Software
1 165 (15.80%) Docker
2 161 (15.42%) HDFS
3 29 (2.78%) Apache Flume
4 12 (1.15%) Apache ZooKeeper
5 10 (0.96%) Virtual Desktop
5 10 (0.96%) Virtual Servers
6 9 (0.86%) VirtualBox
7 3 (0.29%) Active Directory
Systems Management
1 146 (13.98%) Kubernetes
2 120 (11.49%) Ansible
3 93 (8.91%) Terraform
4 67 (6.42%) Hadoop YARN
5 66 (6.32%) Puppet
6 39 (3.74%) Oozie
7 24 (2.30%) Opscode Chef
8 13 (1.25%) Packer
9 12 (1.15%) ITRS Geneos
10 11 (1.05%) Ambari
11 10 (0.96%) Single Sign-On
12 9 (0.86%) Cobbler
12 9 (0.86%) Docker Swarm
12 9 (0.86%) Kibana
13 7 (0.67%) logstash
13 7 (0.67%) Prometheus
14 5 (0.48%) ZABBIX
15 4 (0.38%) FortiGate
15 4 (0.38%) Red Hat Satellite
15 4 (0.38%) Salt
Vendors
1 184 (17.62%) Cloudera
2 146 (13.98%) Oracle
3 125 (11.97%) Hortonworks
4 83 (7.95%) Microsoft
5 78 (7.47%) Tableau
6 77 (7.38%) Google
7 59 (5.65%) Informatica
8 55 (5.27%) Teradata
9 45 (4.31%) SAP
10 42 (4.02%) Talend
11 29 (2.78%) Red Hat
12 25 (2.39%) Databricks
13 24 (2.30%) Ab Initio
14 22 (2.11%) IBM
15 21 (2.01%) MapR
15 21 (2.01%) Palantir
16 20 (1.92%) Splunk
17 17 (1.63%) StreamSets
18 14 (1.34%) Atlassian
19 11 (1.05%) Alteryx