Period
to 23 October 2019

The following table provides summary statistics for contract job vacancies with a requirement for Hadoop skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Hadoop over the 6 months to 23 October 2019 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Hadoop
UK
6 months to
23 Oct 2019
Same period 2018 Same period 2017
Rank 164 128 116
Rank change year-on-year -36 -12 +22
Contract jobs citing Hadoop 1,372 2,179 2,163
As % of all contract IT jobs advertised in the UK 1.62% 2.11% 2.16%
As % of the Database & Business Intelligence category 7.50% 9.30% 9.72%
Number of daily rates quoted 953 1,473 1,488
UK median daily rate £550 £530 £525
Median daily rate % change year-on-year +3.77% +0.95% -
10th Percentile £425 £406 £376
90th Percentile £720 £688 £685
UK excluding London median daily rate £525 £500 £466
% change year-on-year +5.00% +7.30% +1.86%
Number of hourly rates quoted 9 6 9
UK median hourly rate £49.13 £56.25 £35.00
Median hourly rate % change year-on-year -12.66% +60.71% -23.50%
10th Percentile £38.00 £28.88 £16.56
90th Percentile £55.83 £127.13 £50.00
UK excluding London median hourly rate £35.00 £47.50 £35.00
% change year-on-year -26.32% +35.71% -23.50%

Hadoop is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Contract vacancies with a requirement for database or business intelligence skills 18,286 23,420 22,256
As % of all contract IT jobs advertised in the UK 21.59% 22.64% 22.21%
Number of daily rates quoted 12,470 16,339 15,064
UK median daily rate £475 £475 £450
Median daily rate % change year-on-year - +5.56% +5.88%
10th Percentile £325 £313 £308
90th Percentile £656 £638 £625
UK excluding London median daily rate £438 £425 £400
% change year-on-year +2.94% +6.25% +6.67%
Number of hourly rates quoted 258 227 277
UK median hourly rate £35.00 £30.00 £30.00
Median hourly rate % change year-on-year +16.67% - +1.69%
10th Percentile £12.59 £12.46 £13.71
90th Percentile £60.00 £56.55 £54.25
UK excluding London median hourly rate £31.50 £33.00 £33.00
% change year-on-year -4.55% - +15.79%

Hadoop
Job Vacancy Trend

Job postings citing Hadoop as a percentage of all IT jobs advertised.

Job vacancy trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Hadoop.

Contractor daily rate trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing Hadoop over the 6 months to 23 October 2019.

Contractor daily rate histogram for Hadoop in the UK

Hadoop
Contractor Hourly Rate Trend

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing Hadoop.

Contractor hourly rate trend for Hadoop in the UK

Hadoop
Contractor Hourly Rate Histogram

The hourly rate distribution of IT jobs citing Hadoop over the 6 months to 23 October 2019.

Contractor hourly rate histogram for Hadoop in the UK

Hadoop
Top 16 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Hadoop within the UK over the 6 months to 23 October 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -43 1,265 £550 - 248
London -29 786 £575 +2.22% 138
UK excluding London -41 561 £525 +5.00% 127
North of England +20 172 £509 +1.70% 19
South East -63 156 £515 -1.90% 39
North West +55 112 £525 -3.00% 11
Scotland +15 85 £500 +14.29% 16
East of England -15 53 £600 +20.00% 12
Yorkshire -21 53 £450 -5.26% 8
South West -56 51 £575 +9.52% 29
Midlands +25 36 £525 +23.53% 11
East Midlands +4 20 £525 +28.05% 7
West Midlands +32 16 £468 -1.58% 4
North East +20 7 £538 -19.26%
Wales +12 6 £675 +34.33% 1
Northern Ireland +5 2 - -

For the 6 months to 23 October 2019, IT contractor jobs citing Hadoop also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for Hadoop.

1 834 (60.79%) Big Data
2 780 (56.85%) Apache Spark
3 718 (52.33%) Python
4 535 (38.99%) SQL
5 452 (32.94%) Java
6 437 (31.85%) Kafka
7 418 (30.47%) AWS
8 410 (29.88%) Agile Software Development
9 391 (28.50%) Apache Hive
10 366 (26.68%) Analytics
11 340 (24.78%) Scala
12 309 (22.52%) Finance
13 306 (22.30%) ETL
14 294 (21.43%) Data Science
15 272 (19.83%) Docker
16 262 (19.10%) Linux
17 260 (18.95%) DevOps
18 228 (16.62%) Azure
19 221 (16.11%) Cloudera
20 218 (15.89%) Machine Learning
21 211 (15.38%) Jenkins
22 209 (15.23%) NoSQL
23 204 (14.87%) Data Warehouse
24 203 (14.80%) Data Modelling
25 199 (14.50%) GCP
26 198 (14.43%) Kubernetes
27 182 (13.27%) HBase
28 177 (12.90%) HDFS
29 170 (12.39%) Oracle
29 170 (12.39%) Google

Hadoop
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 780 (56.85%) Apache Spark
2 120 (8.75%) Apache
3 102 (7.43%) SAS
4 80 (5.83%) Elasticsearch
5 38 (2.77%) Jupyter
6 37 (2.70%) JBoss
7 36 (2.62%) Apache Airflow
8 33 (2.41%) Confluence
9 29 (2.11%) Tomcat
10 23 (1.68%) Apache Pig
11 17 (1.24%) Jetty
12 12 (0.87%) Apache Solr
13 11 (0.80%) SharePoint
14 10 (0.73%) Luigi
14 10 (0.73%) Pachyderm
15 7 (0.51%) WebLogic
16 6 (0.44%) Oracle Self-Service HR
17 3 (0.22%) nginx
17 3 (0.22%) SharePoint 2013
18 2 (0.15%) CMS
Applications
1 24 (1.75%) Microsoft Excel
2 9 (0.66%) SPSS
3 7 (0.51%) Stata
4 6 (0.44%) Spreadsheet
5 5 (0.36%) Microsoft Office
6 2 (0.15%) MS Visio
7 1 (0.073%) Microsoft Project
Business Applications
1 10 (0.73%) SAP Oil and Gas
2 8 (0.58%) FINCAD Analytics
3 2 (0.15%) Oracle Subledger Accounting
4 1 (0.073%) Cognos Business Intelligence
4 1 (0.073%) Oracle Receivables
4 1 (0.073%) Sparx Enterprise Architect
Cloud Services
1 418 (30.47%) AWS
2 228 (16.62%) Azure
3 199 (14.50%) GCP
4 88 (6.41%) Amazon EMR
5 72 (5.25%) Amazon S3
6 66 (4.81%) Serverless
7 52 (3.79%) Power Platform
8 50 (3.64%) AWS Lambda
9 48 (3.50%) GitHub
10 41 (2.99%) Amazon Kinesis
11 32 (2.33%) Amazon EC2
12 27 (1.97%) Azure Data Factory
13 22 (1.60%) OpenShift
14 18 (1.31%) Amazon SageMaker
15 17 (1.24%) Amazon SQS
15 17 (1.24%) Cloud Computing
16 16 (1.17%) AWS CloudFormation
17 15 (1.09%) Amazon CloudWatch
18 14 (1.02%) Virtual Private Cloud
19 12 (0.87%) Amazon EKS
Communications & Networking
1 24 (1.75%) Kerberos
2 16 (1.17%) SSH
3 13 (0.95%) Internet
4 12 (0.87%) Firewall
5 8 (0.58%) TCP/IP
6 7 (0.51%) SAN
6 7 (0.51%) SSL
7 5 (0.36%) OpenLDAP
7 5 (0.36%) Wi-Fi
8 4 (0.29%) F5 BIG-IP GTM
8 4 (0.29%) F5 BIG-IP LTM
8 4 (0.29%) InfiniBand
8 4 (0.29%) Multicast
9 3 (0.22%) 5G
9 3 (0.22%) HTTP
9 3 (0.22%) Radio Access Network
9 3 (0.22%) Software-Defined Networking
10 2 (0.15%) Intranet
10 2 (0.15%) LAN
10 2 (0.15%) WAN
Database & Business Intelligence
1 834 (60.79%) Big Data
2 391 (28.50%) Apache Hive
3 209 (15.23%) NoSQL
4 204 (14.87%) Data Warehouse
5 182 (13.27%) HBase
6 160 (11.66%) Data Lake
7 159 (11.59%) MongoDB
8 142 (10.35%) Impala
9 125 (9.11%) PostgreSQL
10 109 (7.94%) Relational Database
11 91 (6.63%) SQL Server
12 89 (6.49%) Apache Cassandra
13 83 (6.05%) MySQL
14 80 (5.83%) Amazon Redshift
15 72 (5.25%) BigQuery
15 72 (5.25%) Spark SQL
16 61 (4.45%) Sqoop
17 52 (3.79%) Power BI
18 45 (3.28%) Flink
18 45 (3.28%) Pentaho
Development Applications
1 211 (15.38%) Jenkins
2 143 (10.42%) Git (software)
3 98 (7.14%) JIRA
4 47 (3.43%) Maven
5 44 (3.21%) Atlassian Bamboo
6 34 (2.48%) Cucumber
7 32 (2.33%) GitLab
8 25 (1.82%) Gradle
9 18 (1.31%) Sonatype Nexus
9 18 (1.31%) Subversion
10 15 (1.09%) Hudson
10 15 (1.09%) Mercurial
10 15 (1.09%) SonarQube
10 15 (1.09%) Vagrant
11 14 (1.02%) TeamCity
12 12 (0.87%) Ant
13 11 (0.80%) Bitbucket
13 11 (0.80%) Selenium
14 10 (0.73%) Team Foundation Server
14 10 (0.73%) Travis CI
General
1 309 (22.52%) Finance
2 150 (10.93%) Banking
3 56 (4.08%) Investment Banking
4 52 (3.79%) Retail
5 44 (3.21%) Front Office
6 27 (1.97%) Legal
7 24 (1.75%) Public Sector
8 19 (1.38%) Telecoms
9 18 (1.31%) Retail Banking
10 17 (1.24%) Pharmaceutical
11 16 (1.17%) Marketing
12 10 (0.73%) Financial Institution
12 10 (0.73%) Games
12 10 (0.73%) Publishing
13 7 (0.51%) Manufacturing
14 5 (0.36%) Advertising
15 4 (0.29%) Automotive
16 3 (0.22%) Aerospace
17 2 (0.15%) International Banking
17 2 (0.15%) Military
Job Titles
1 403 (29.37%) Data Engineer
2 289 (21.06%) Developer
3 172 (12.54%) Architect
4 125 (9.11%) Big Data Engineer
5 112 (8.16%) Data Architect
6 97 (7.07%) Java Developer
7 81 (5.90%) Administrator
8 79 (5.76%) Hadoop Engineer
8 79 (5.76%) Python Engineer
9 74 (5.39%) Data Scientist
10 72 (5.25%) Analyst
11 71 (5.17%) Big Data Developer
12 64 (4.66%) DevOps Engineer
13 59 (4.30%) Python Developer
14 58 (4.23%) Hadoop Developer
14 58 (4.23%) Scala Developer
15 57 (4.15%) Java Engineer
16 53 (3.86%) AWS Engineer
17 47 (3.43%) Senior Developer
18 46 (3.35%) Big Data Architect
Libraries, Frameworks & Software Standards
1 437 (31.85%) Kafka
2 92 (6.71%) PySpark
3 88 (6.41%) Spring
4 63 (4.59%) Web Services
5 58 (4.23%) Spark Streaming
6 55 (4.01%) Apache NiFi
6 55 (4.01%) Pandas
7 50 (3.64%) React
7 50 (3.64%) REST
8 41 (2.99%) Hibernate
8 41 (2.99%) RESTful
9 37 (2.70%) AngularJS
9 37 (2.70%) Flask
9 37 (2.70%) JSON
9 37 (2.70%) RabbitMQ
10 35 (2.55%) Natural Language Processing
10 35 (2.55%) Spring Boot
11 32 (2.33%) XML
12 31 (2.26%) Node.js
13 28 (2.04%) scikit-learn
Miscellaneous
1 146 (10.64%) Algorithms
2 133 (9.69%) Analytical Skills
3 67 (4.88%) Data Structures
4 59 (4.30%) Greenfield Project
5 39 (2.84%) Self-Motivation
6 26 (1.90%) User Experience
7 23 (1.68%) Derivative
8 20 (1.46%) Virtual Team
9 19 (1.38%) Linux Command Line
10 18 (1.31%) Management Information System
11 16 (1.17%) Fixed Income
12 15 (1.09%) Cloud Native
12 15 (1.09%) PKI
12 15 (1.09%) Public Cloud
13 13 (0.95%) Credit Risk
14 12 (0.87%) Hedge funds
15 10 (0.73%) Industry 4.0
16 9 (0.66%) Foreign Exchange (FX)
16 9 (0.66%) Replication
17 8 (0.58%) Clustering
Operating Systems
1 262 (19.10%) Linux
2 131 (9.55%) Unix
3 54 (3.94%) Windows
4 52 (3.79%) Red Hat Enterprise Linux
5 40 (2.92%) CentOS
6 26 (1.90%) Ubuntu
7 18 (1.31%) Windows Server
8 13 (0.95%) Debian
9 11 (0.80%) Android
9 11 (0.80%) Apple iOS
10 8 (0.58%) Mac OS X
11 3 (0.22%) Solaris
12 2 (0.15%) SUSE Linux Enterprise Server
13 1 (0.073%) AIX
13 1 (0.073%) CoreOS
13 1 (0.073%) Oracle Linux
13 1 (0.073%) Windows Server 2012
13 1 (0.073%) zOS
Processes & Methodologies
1 410 (29.88%) Agile Software Development
2 366 (26.68%) Analytics
3 306 (22.30%) ETL
4 294 (21.43%) Data Science
5 260 (18.95%) DevOps
6 218 (15.89%) Machine Learning
7 203 (14.80%) Data Modelling
8 163 (11.88%) Software Engineering
9 155 (11.30%) Business Intelligence
10 153 (11.15%) Migration
11 138 (10.06%) Problem-Solving
12 131 (9.55%) Microservices
13 130 (9.48%) Continuous Integration
14 129 (9.40%) Computer Science
15 128 (9.33%) TDD
16 114 (8.31%) Data Architecture
17 105 (7.65%) Visualisation
18 99 (7.22%) Data Engineering
18 99 (7.22%) Scrum
19 92 (6.71%) Open Source
Programming Languages
1 718 (52.33%) Python
2 535 (38.99%) SQL
3 452 (32.94%) Java
4 340 (24.78%) Scala
5 134 (9.77%) R
6 94 (6.85%) JavaScript
7 57 (4.15%) Bash Shell
8 54 (3.94%) C++
9 49 (3.57%) Java 8
10 46 (3.35%) C
11 44 (3.21%) C#
12 41 (2.99%) Shell Script
13 39 (2.84%) Ruby
14 38 (2.77%) Perl
15 31 (2.26%) Go
16 26 (1.90%) Groovy
17 24 (1.75%) PL/SQL
18 22 (1.60%) PowerShell
18 22 (1.60%) T-SQL
19 21 (1.53%) MATLAB
Qualifications
1 127 (9.26%) Security Cleared
2 116 (8.45%) Degree
3 80 (5.83%) SC Cleared
4 50 (3.64%) DV Cleared
5 49 (3.57%) Computer Science Degree
6 33 (2.41%) PhD
7 22 (1.60%) Master's Degree
8 10 (0.73%) CCA
9 9 (0.66%) Cisco Certification
10 8 (0.58%) BPSS Clearance
11 7 (0.51%) CISSP
11 7 (0.51%) Numerate Degree
11 7 (0.51%) TOGAF Certification
12 5 (0.36%) (ISC)2 CCSP
12 5 (0.36%) CSSLP
12 5 (0.36%) DBS Check
12 5 (0.36%) ISSAP
12 5 (0.36%) SAS Certification
13 4 (0.29%) AWS Certification
13 4 (0.29%) CCIE
Quality Assurance & Compliance
1 62 (4.52%) Data Quality
2 17 (1.24%) QA
3 13 (0.95%) Sarbanes-Oxley
4 7 (0.51%) GDPR
4 7 (0.51%) GxP
5 6 (0.44%) ISO/IEC 27001
6 4 (0.29%) Disclosure Scotland
7 3 (0.22%) PCI DSS
7 3 (0.22%) PMO
7 3 (0.22%) SLA
8 1 (0.073%) Basel III
8 1 (0.073%) EMIR
8 1 (0.073%) IFRS
8 1 (0.073%) IFRS 9
8 1 (0.073%) MiFID
System Software
1 272 (19.83%) Docker
2 177 (12.90%) HDFS
3 52 (3.79%) Apache Flume
4 20 (1.46%) Apache ZooKeeper
5 10 (0.73%) Active Directory
6 9 (0.66%) VirtualBox
7 8 (0.58%) Virtual Servers
7 8 (0.58%) vSphere
8 7 (0.51%) Virtual Desktop
8 7 (0.51%) VMware ESXi
9 6 (0.44%) VMware Infrastructure
10 4 (0.29%) Samba
11 2 (0.15%) VMware NSX
12 1 (0.073%) NFS
12 1 (0.073%) Virtual Machines
Systems Management
1 198 (14.43%) Kubernetes
2 163 (11.88%) Ansible
3 125 (9.11%) Terraform
4 96 (7.00%) Puppet
5 74 (5.39%) Hadoop YARN
6 66 (4.81%) Oozie
7 39 (2.84%) Opscode Chef
8 15 (1.09%) Packer
9 14 (1.02%) Cobbler
10 11 (0.80%) Ambari
11 10 (0.73%) Kibana
11 10 (0.73%) Kubeflow
11 10 (0.73%) ZABBIX
12 9 (0.66%) Docker Swarm
12 9 (0.66%) logstash
13 8 (0.58%) Consul
13 8 (0.58%) ITRS Geneos
13 8 (0.58%) Prometheus
13 8 (0.58%) Salt
14 7 (0.51%) Control-M
Vendors
1 221 (16.11%) Cloudera
2 170 (12.39%) Google
2 170 (12.39%) Oracle
3 118 (8.60%) Hortonworks
4 96 (7.00%) Tableau
5 84 (6.12%) Microsoft
6 72 (5.25%) Teradata
7 68 (4.96%) Talend
8 67 (4.88%) Informatica
9 53 (3.86%) Red Hat
10 41 (2.99%) Databricks
11 30 (2.19%) Splunk
12 28 (2.04%) SAP
13 23 (1.68%) Atlassian
13 23 (1.68%) MapR
14 18 (1.31%) Palantir
15 16 (1.17%) Sybase
16 14 (1.02%) Ab Initio
16 14 (1.02%) IBM
17 13 (0.95%) Alteryx