Period
to 27 June 2019

The following table provides summary statistics for contract job vacancies with a requirement for Hadoop skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Hadoop over the 6 months to 27 June 2019 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Hadoop
UK
6 months to
27 Jun 2019
Same period 2018 Same period 2017
Rank 137 118 111
Rank change year-on-year -19 -7 +72
Contract jobs citing Hadoop 1,750 2,165 1,952
As % of all contract IT jobs advertised in the UK 1.92% 2.14% 2.12%
As % of the Database & Business Intelligence category 8.65% 9.50% 9.39%
Number of daily rates quoted 1,203 1,545 1,301
UK median daily rate £550 £550 £513
Median daily rate % change year-on-year - +7.32% -2.38%
10th Percentile £413 £400 £370
90th Percentile £700 £678 £675
UK excluding London median daily rate £500 £500 £456
% change year-on-year - +9.59% -3.95%
Number of hourly rates quoted 13 8 6
UK median hourly rate £44.57 £47.50 £47.00
Median hourly rate % change year-on-year -6.18% +1.06% +23.68%
10th Percentile £33.00 £21.73 £22.38
90th Percentile £66.46 £91.75 £50.00
UK excluding London median hourly rate £35.00 £46.25 £47.00
% change year-on-year -24.32% -1.60% +23.68%

Hadoop is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Contract vacancies with a requirement for database or business intelligence skills 20,223 22,797 20,782
As % of all contract IT jobs advertised in the UK 22.19% 22.55% 22.57%
Number of daily rates quoted 13,856 15,872 13,826
UK median daily rate £475 £463 £445
Median daily rate % change year-on-year +2.70% +3.93% +4.71%
10th Percentile £325 £313 £300
90th Percentile £650 £634 £613
UK excluding London median daily rate £430 £413 £400
% change year-on-year +4.24% +3.13% +6.67%
Number of hourly rates quoted 246 232 272
UK median hourly rate £35.00 £33.50 £30.50
Median hourly rate % change year-on-year +4.48% +9.84% +7.02%
10th Percentile £14.27 £14.00 £10.50
90th Percentile £59.56 £54.88 £52.17
UK excluding London median hourly rate £35.00 £33.50 £31.00
% change year-on-year +4.48% +8.06% +3.33%

Hadoop
Job Vacancy Trend

Job postings citing Hadoop as a percentage of all IT jobs advertised.

Job vacancy trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Hadoop.

Contractor daily rate trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing Hadoop over the 6 months to 27 June 2019.

Contractor daily rate histogram for Hadoop in the UK

Hadoop
Contractor Hourly Rate Trend

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing Hadoop.

Contractor hourly rate trend for Hadoop in the UK

Hadoop
Contractor Hourly Rate Histogram

The hourly rate distribution of IT jobs citing Hadoop over the 6 months to 27 June 2019.

Contractor hourly rate histogram for Hadoop in the UK

Hadoop
Top 16 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Hadoop within the UK over the 6 months to 27 June 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -21 1,641 £550 - 97
London -28 965 £575 +4.55% 57
UK excluding London -8 752 £500 - 43
North of England +55 235 £525 +5.00% 5
South East -43 219 £500 -2.44% 13
Yorkshire +31 124 £500 -
North West +41 103 £550 +4.76% 5
South West -16 93 £519 +9.21% 15
Scotland +38 91 £450 +9.09% 3
Midlands +12 65 £525 +16.67% 3
East of England -24 40 £563 +18.42% 4
West Midlands +24 36 £600 +65.52%
East Midlands -10 29 £475 +5.56% 3
North East +29 8 £476 -32.70%
Wales -8 5 £525 -4.55%
Northern Ireland - 3 - -

For the 6 months to 27 June 2019, IT contractor jobs citing Hadoop also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for Hadoop.

1 1,178 (67.31%) Big Data
2 988 (56.46%) Apache Spark
3 857 (48.97%) Python
4 634 (36.23%) SQL
5 613 (35.03%) Java
6 610 (34.86%) Apache Hive
7 594 (33.94%) Amazon AWS
8 583 (33.31%) Agile Software Development
9 564 (32.23%) Kafka
10 448 (25.60%) Analytics
11 418 (23.89%) Scala
12 409 (23.37%) Finance
13 384 (21.94%) DevOps
14 373 (21.31%) Data Science
15 360 (20.57%) Linux
16 327 (18.69%) Data Warehouse
17 318 (18.17%) NoSQL
18 317 (18.11%) Microsoft Azure
19 311 (17.77%) Jenkins
20 308 (17.60%) HBase
21 296 (16.91%) Docker
22 291 (16.63%) ETL
23 285 (16.29%) Machine Learning
24 265 (15.14%) Data Modelling
25 253 (14.46%) HDFS
26 245 (14.00%) Cloudera
27 231 (13.20%) Kubernetes
28 230 (13.14%) Google Cloud Platform
29 220 (12.57%) Google
30 218 (12.46%) Data Lake

Hadoop
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 988 (56.46%) Apache Spark
2 165 (9.43%) Apache
3 109 (6.23%) Elasticsearch
4 88 (5.03%) SAS
5 54 (3.09%) Apache Pig
6 50 (2.86%) Apache Airflow
7 47 (2.69%) Confluence
8 36 (2.06%) Apache Solr
9 28 (1.60%) Luigi
10 26 (1.49%) JBoss
11 22 (1.26%) nginx
12 21 (1.20%) Tomcat
13 19 (1.09%) Jupyter
14 13 (0.74%) Pachyderm
15 11 (0.63%) Jetty
16 9 (0.51%) OpenStack
17 7 (0.40%) Cloud Foundry
18 5 (0.29%) Oracle SOA Suite
18 5 (0.29%) SharePoint
18 5 (0.29%) SharePoint 2013
Applications
1 12 (0.69%) Microsoft Excel
2 10 (0.57%) SPSS
3 9 (0.51%) Stata
4 8 (0.46%) Microsoft PowerPoint
5 5 (0.29%) MS Visio
6 3 (0.17%) Microsoft Office
7 2 (0.11%) Chatbot
8 1 (0.057%) Spreadsheet
Business Applications
1 17 (0.97%) SAP Oil and Gas
2 5 (0.29%) Dynamics NAV
2 5 (0.29%) Oracle BPM
3 4 (0.23%) Revive Adserver
3 4 (0.23%) SAP ERP
3 4 (0.23%) SAP S/4HANA
4 1 (0.057%) Cognos Business Intelligence
4 1 (0.057%) Dynamics CRM
4 1 (0.057%) Eloqua
4 1 (0.057%) Oracle Receivables
4 1 (0.057%) Oracle Retail
4 1 (0.057%) SAP IS-U
Cloud Services
1 594 (33.94%) Amazon AWS
2 317 (18.11%) Microsoft Azure
3 230 (13.14%) Google Cloud Platform
4 124 (7.09%) Amazon EMR
5 95 (5.43%) Serverless
6 94 (5.37%) Amazon S3
7 80 (4.57%) GitHub
8 78 (4.46%) Power Platform
9 61 (3.49%) Amazon Kinesis
10 55 (3.14%) Amazon EC2
11 49 (2.80%) AWS Lambda
12 43 (2.46%) Azure Data Factory
13 39 (2.23%) AWS CloudFormation
14 36 (2.06%) PaaS
15 30 (1.71%) Cloud Computing
16 22 (1.26%) SaaS
17 21 (1.20%) Amazon CloudWatch
18 17 (0.97%) Amazon SageMaker
18 17 (0.97%) Azure Functions
19 16 (0.91%) Data Lake Analytics
Communications & Networking
1 42 (2.40%) Kerberos
2 18 (1.03%) Internet
3 16 (0.91%) TCP/IP
4 15 (0.86%) Firewall
5 14 (0.80%) MQTT
6 10 (0.57%) DNS
7 8 (0.46%) OpenLDAP
8 7 (0.40%) VPN
9 6 (0.34%) SSL
10 5 (0.29%) Cisco IOS
11 4 (0.23%) HTTP
12 3 (0.17%) InfiniBand
12 3 (0.17%) NAS
12 3 (0.17%) Network Security
12 3 (0.17%) SAN
12 3 (0.17%) Software-Defined Networking
13 2 (0.11%) DSL
13 2 (0.11%) ICMP
13 2 (0.11%) VLAN
13 2 (0.11%) VSAN
Database & Business Intelligence
1 1,178 (67.31%) Big Data
2 610 (34.86%) Apache Hive
3 327 (18.69%) Data Warehouse
4 318 (18.17%) NoSQL
5 308 (17.60%) HBase
6 218 (12.46%) Data Lake
7 181 (10.34%) MongoDB
8 165 (9.43%) Apache Cassandra
9 160 (9.14%) Impala
10 150 (8.57%) PostgreSQL
11 126 (7.20%) Relational Database
12 120 (6.86%) SQL Server
13 95 (5.43%) Amazon Redshift
14 78 (4.46%) Power BI
15 75 (4.29%) Sqoop
16 74 (4.23%) MySQL
17 68 (3.89%) RDBMS
18 64 (3.66%) Parquet
19 56 (3.20%) BigQuery
20 55 (3.14%) Amazon RDS
Development Applications
1 311 (17.77%) Jenkins
2 204 (11.66%) Git (software)
3 118 (6.74%) JIRA
4 72 (4.11%) Maven
5 57 (3.26%) Gradle
6 45 (2.57%) Subversion
7 42 (2.40%) GitLab
8 39 (2.23%) Sonatype Nexus
9 33 (1.89%) Atlassian Bamboo
10 31 (1.77%) Cucumber
11 24 (1.37%) Mercurial
11 24 (1.37%) Visual Studio Team System
12 21 (1.20%) JUnit
13 20 (1.14%) RStudio
13 20 (1.14%) TeamCity
14 18 (1.03%) Hudson
15 17 (0.97%) Ant
15 17 (0.97%) Apache Ivy
15 17 (0.97%) Bitbucket
15 17 (0.97%) Selenium
General
1 409 (23.37%) Finance
2 177 (10.11%) Banking
3 64 (3.66%) Retail
4 54 (3.09%) Investment Banking
5 42 (2.40%) Front Office
6 36 (2.06%) Legal
7 35 (2.00%) Marketing
8 20 (1.14%) Telecoms
9 17 (0.97%) Retail Banking
10 15 (0.86%) Financial Institution
11 14 (0.80%) Pharmaceutical
12 11 (0.63%) Publishing
13 8 (0.46%) Advertising
14 7 (0.40%) Games
15 6 (0.34%) Manufacturing
16 5 (0.29%) Aerospace
16 5 (0.29%) General Ledger
17 3 (0.17%) Back Office
17 3 (0.17%) Greek Language
18 2 (0.11%) Social Science
Job Titles
1 494 (28.23%) Data Engineer
2 430 (24.57%) Developer
3 227 (12.97%) Architect
4 217 (12.40%) Big Data Engineer
5 170 (9.71%) Java Developer
6 165 (9.43%) Hadoop Engineer
7 136 (7.77%) Data Architect
8 131 (7.49%) DevOps Engineer
9 119 (6.80%) Big Data Developer
10 110 (6.29%) Python Engineer
11 88 (5.03%) Hadoop Developer
12 78 (4.46%) Data Scientist
13 77 (4.40%) Administrator
14 76 (4.34%) Cloud Engineer
15 70 (4.00%) AWS Engineer
16 66 (3.77%) Analyst
17 63 (3.60%) Hadoop Administrator
18 62 (3.54%) Python Developer
19 61 (3.49%) Big Data Architect
20 57 (3.26%) Scala Developer
Libraries, Frameworks & Software Standards
1 564 (32.23%) Kafka
2 124 (7.09%) Spring
3 97 (5.54%) Web Services
4 80 (4.57%) PySpark
5 68 (3.89%) REST
5 68 (3.89%) Spark Streaming
6 59 (3.37%) RESTful
7 57 (3.26%) RabbitMQ
8 52 (2.97%) Natural Language Processing
8 52 (2.97%) Spring Boot
9 49 (2.80%) Apache NiFi
10 43 (2.46%) AngularJS
10 43 (2.46%) HTML
10 43 (2.46%) JSON
11 41 (2.34%) TensorFlow
12 36 (2.06%) Hibernate
12 36 (2.06%) HTML5
13 34 (1.94%) Elastic Stack
13 34 (1.94%) Pandas
14 32 (1.83%) Flask
Miscellaneous
1 162 (9.26%) Analytical Skills
2 127 (7.26%) Algorithms
3 106 (6.06%) Greenfield Project
4 63 (3.60%) Data Structures
5 60 (3.43%) Self-Motivation
6 36 (2.06%) Virtual Team
7 31 (1.77%) Linux Command Line
8 30 (1.71%) User Experience
9 24 (1.37%) Credit Risk
10 23 (1.31%) Public Cloud
11 21 (1.20%) Data Centre
12 13 (0.74%) Cloud Native
12 13 (0.74%) Hedge funds
12 13 (0.74%) Market Risk
13 11 (0.63%) Data Protection Act
13 11 (0.63%) Enterprise Software
14 10 (0.57%) Derivative
14 10 (0.57%) Foreign Exchange (FX)
14 10 (0.57%) Industry 4.0
15 8 (0.46%) Management Information System
Operating Systems
1 360 (20.57%) Linux
2 192 (10.97%) Unix
3 47 (2.69%) Red Hat Enterprise Linux
4 41 (2.34%) Windows
5 19 (1.09%) CentOS
6 13 (0.74%) Ubuntu
7 10 (0.57%) Solaris
8 7 (0.40%) Windows Server
9 6 (0.34%) Debian
10 5 (0.29%) Android
10 5 (0.29%) Apple iOS
11 3 (0.17%) SUSE Linux Enterprise Server
11 3 (0.17%) Windows Server 2003
11 3 (0.17%) Windows Server 2008
11 3 (0.17%) Windows Server 2012
11 3 (0.17%) Windows Vista
12 2 (0.11%) zOS
13 1 (0.057%) Data ONTAP
13 1 (0.057%) Mac OS X
13 1 (0.057%) Mesosphere
Processes & Methodologies
1 583 (33.31%) Agile Software Development
2 448 (25.60%) Analytics
3 384 (21.94%) DevOps
4 373 (21.31%) Data Science
5 291 (16.63%) ETL
6 285 (16.29%) Machine Learning
7 265 (15.14%) Data Modelling
8 206 (11.77%) TDD
9 197 (11.26%) Software Engineering
10 185 (10.57%) Continuous Integration
11 180 (10.29%) Business Intelligence
12 158 (9.03%) Problem-Solving
13 149 (8.51%) Microservices
14 146 (8.34%) MapReduce
15 132 (7.54%) Scrum
16 131 (7.49%) Visualisation
17 119 (6.80%) Data Architecture
18 118 (6.74%) Computer Science
19 117 (6.69%) Artificial Intelligence
20 110 (6.29%) Migration
Programming Languages
1 857 (48.97%) Python
2 634 (36.23%) SQL
3 613 (35.03%) Java
4 418 (23.89%) Scala
5 159 (9.09%) R
6 103 (5.89%) JavaScript
7 75 (4.29%) C#
7 75 (4.29%) Java 8
7 75 (4.29%) Shell Script
8 63 (3.60%) C++
9 53 (3.03%) Ruby
10 50 (2.86%) Bash Shell
11 44 (2.51%) C
12 32 (1.83%) MATLAB
12 32 (1.83%) Perl
13 30 (1.71%) Go
14 28 (1.60%) T-SQL
15 18 (1.03%) U-SQL
16 15 (0.86%) PL/SQL
17 14 (0.80%) Julia
Qualifications
1 134 (7.66%) Security Cleared
2 119 (6.80%) Degree
3 99 (5.66%) SC Cleared
4 37 (2.11%) DV Cleared
5 35 (2.00%) Computer Science Degree
6 32 (1.83%) PhD
7 30 (1.71%) Master's Degree
8 19 (1.09%) RHCE
9 18 (1.03%) CCA
10 11 (0.63%) TOGAF Certification
11 9 (0.51%) AWS Certification
12 6 (0.34%) BPSS Clearance
13 5 (0.29%) CISSP
13 5 (0.29%) ISTQB
14 4 (0.23%) CEH
14 4 (0.23%) CISA
14 4 (0.23%) CISM
14 4 (0.23%) GIAC
14 4 (0.23%) Microsoft Certification
14 4 (0.23%) OSCP
Quality Assurance & Compliance
1 80 (4.57%) Data Quality
2 17 (0.97%) QA
3 14 (0.80%) GDPR
4 10 (0.57%) SLA
5 5 (0.29%) ISO/IEC 27001
6 4 (0.23%) IFRS
6 4 (0.23%) Sarbanes-Oxley
7 3 (0.17%) PMO
8 2 (0.11%) Actionable Recommendations
8 2 (0.11%) MiFID
8 2 (0.11%) PSD2
9 1 (0.057%) Basel III
9 1 (0.057%) Disclosure Scotland
9 1 (0.057%) EMIR
9 1 (0.057%) GxP
9 1 (0.057%) IFRS 9
9 1 (0.057%) PCI DSS
System Software
1 296 (16.91%) Docker
2 253 (14.46%) HDFS
3 109 (6.23%) Apache Flume
4 37 (2.11%) Apache ZooKeeper
5 27 (1.54%) VMware Infrastructure
6 17 (0.97%) Active Directory
6 17 (0.97%) Oracle RAC
7 11 (0.63%) vCloud
8 7 (0.40%) VMware ESXi
8 7 (0.40%) vSphere
9 5 (0.29%) Virtual Machines
10 4 (0.23%) Hyper-V
10 4 (0.23%) Samba
11 3 (0.17%) NFS
12 2 (0.11%) VMware NSX
13 1 (0.057%) KVM
13 1 (0.057%) Virtual Servers
13 1 (0.057%) VirtualBox
Systems Management
1 231 (13.20%) Kubernetes
2 186 (10.63%) Ansible
3 148 (8.46%) Terraform
4 129 (7.37%) Puppet
5 107 (6.11%) Hadoop YARN
5 107 (6.11%) Oozie
6 64 (3.66%) Opscode Chef
7 22 (1.26%) Kibana
7 22 (1.26%) logstash
8 21 (1.20%) ZABBIX
9 16 (0.91%) Packer
10 15 (0.86%) Ambari
11 13 (0.74%) Kubeflow
12 10 (0.57%) Nagios
13 7 (0.40%) Cobbler
13 7 (0.40%) Mesos
14 6 (0.34%) vCenter Server
15 5 (0.29%) Grafana
15 5 (0.29%) Marathon
16 4 (0.23%) Microsoft Clustering
Vendors
1 245 (14.00%) Cloudera
2 220 (12.57%) Google
3 176 (10.06%) Hortonworks
4 150 (8.57%) Oracle
5 120 (6.86%) Tableau
6 97 (5.54%) Microsoft
7 70 (4.00%) Talend
8 68 (3.89%) Databricks
9 63 (3.60%) Informatica
10 62 (3.54%) Teradata
11 46 (2.63%) Red Hat
12 44 (2.51%) MapR
13 42 (2.40%) SAP
14 33 (1.89%) Splunk
15 32 (1.83%) VMware
16 17 (0.97%) Ab Initio
16 17 (0.97%) Atlassian
16 17 (0.97%) MicroStrategy
17 15 (0.86%) Dataiku
18 14 (0.80%) IBM