Period
to 23 October 2018

The following table provides summary statistics for contract job vacancies with a requirement for Hadoop skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Hadoop over the 6 months to 23 October 2018 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Hadoop
UK
6 months to
23 Oct 2018
Same period 2017 Same period 2016
Rank 127 114 136
Rank change year-on-year -13 +22 +88
Contract jobs citing Hadoop 2,179 2,163 2,042
As % of all contract IT jobs advertised in the UK 2.11% 2.16% 1.94%
As % of the Database & Business Intelligence category 9.30% 9.72% 8.59%
Number of daily rates quoted 1,473 1,488 1,315
UK median daily rate £530 £525 £525
Median daily rate % change year-on-year +0.95% - -
10th Percentile £406 £376 £363
90th Percentile £688 £685 £713
UK excluding London median daily rate £500 £466 £458
% change year-on-year +7.30% +1.86% -3.68%
Number of hourly rates quoted 6 9 6
UK median hourly rate £56.25 £35.00 £45.75
Median hourly rate % change year-on-year +60.71% -23.50% +28.78%
10th Percentile £28.88 £16.56 £35.00
90th Percentile £127.13 £50.00 £46.38
UK excluding London median hourly rate £47.50 £35.00 £45.75
% change year-on-year +35.71% -23.50% +36.36%

Hadoop is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Contract vacancies with a requirement for database or business intelligence skills 23,419 22,254 23,758
As % of all contract IT jobs advertised in the UK 22.64% 22.21% 22.54%
Number of daily rates quoted 16,338 15,062 15,824
UK median daily rate £475 £450 £425
Median daily rate % change year-on-year +5.56% +5.88% +6.25%
10th Percentile £313 £308 £281
90th Percentile £638 £625 £588
UK excluding London median daily rate £425 £400 £375
% change year-on-year +6.25% +6.67% -
Number of hourly rates quoted 227 277 389
UK median hourly rate £30.00 £30.00 £29.50
Median hourly rate % change year-on-year - +1.69% +5.12%
10th Percentile £12.46 £13.71 £10.85
90th Percentile £56.55 £54.25 £50.20
UK excluding London median hourly rate £33.00 £33.00 £28.50
% change year-on-year - +15.79% -3.39%

Hadoop
Job Vacancy Trend

Job postings citing Hadoop as a percentage of all IT jobs advertised.

Job vacancy trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Hadoop.

Contractor daily rate trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing Hadoop over the 6 months to 23 October 2018.

Contractor daily rate histogram for Hadoop in the UK

Hadoop
Contractor Hourly Rate Trend

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing Hadoop.

Contractor hourly rate trend for Hadoop in the UK

Hadoop
Contractor Hourly Rate Histogram

The hourly rate distribution of IT jobs citing Hadoop over the 6 months to 23 October 2018.

Contractor hourly rate histogram for Hadoop in the UK

Hadoop
Top 16 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Hadoop within the UK over the 6 months to 23 October 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -8 2,037 £550 +2.33% 337
London -15 1,245 £563 +2.27% 177
UK excluding London +16 909 £500 +7.30% 186
South East +9 342 £525 +10.53% 55
North of England +24 178 £500 +5.26% 42
Yorkshire +27 106 £475 +2.15% 17
South West -2 103 £525 +10.53% 28
East of England +15 101 £500 +2.56% 19
Scotland +9 100 £438 +9.38% 17
North West +2 72 £541 -9.79% 20
Midlands +20 64 £425 -19.05% 16
East Midlands +47 47 £410 -21.90% 9
Wales -8 19 £503 +0.50% 3
West Midlands -28 16 £475 - 7
Northern Ireland +22 3 - - 5
North East -17 2 £666 +9.14% 5

For the 6 months to 23 October 2018, IT contractor jobs citing Hadoop also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for Hadoop.

1 1,321 (60.62%) Big Data
2 1,013 (46.49%) Apache Spark
3 905 (41.53%) Python
4 893 (40.98%) Agile Software Development
5 869 (39.88%) Java
6 692 (31.76%) Apache Hive
7 688 (31.57%) Kafka
8 664 (30.47%) SQL
9 589 (27.03%) Finance
10 574 (26.34%) Amazon AWS
11 536 (24.60%) Analytics
12 480 (22.03%) DevOps
13 478 (21.94%) Linux
14 430 (19.73%) Scala
15 421 (19.32%) Jenkins
16 413 (18.95%) Ansible
17 410 (18.82%) Data Science
18 390 (17.90%) NoSQL
19 370 (16.98%) HBase
20 357 (16.38%) ETL
21 340 (15.60%) Docker
22 322 (14.78%) Scrum
23 309 (14.18%) Business Intelligence
24 305 (14.00%) TDD
25 304 (13.95%) MongoDB
26 295 (13.54%) Cloudera
27 291 (13.35%) Data Warehouse
28 289 (13.26%) HDFS
29 284 (13.03%) Puppet
30 282 (12.94%) Machine Learning

Hadoop
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 1,013 (46.49%) Apache Spark
2 252 (11.56%) Apache
3 196 (8.99%) Elasticsearch
4 119 (5.46%) SAS
5 101 (4.64%) Apache Pig
6 93 (4.27%) Tomcat
7 85 (3.90%) Confluence
8 66 (3.03%) JBoss
9 65 (2.98%) Jetty
10 63 (2.89%) Apache Solr
11 29 (1.33%) nginx
12 17 (0.78%) Jupyter
12 17 (0.78%) SharePoint
12 17 (0.78%) WebLogic
12 17 (0.78%) WebSphere
13 16 (0.73%) OpenStack
14 13 (0.60%) Apache Airflow
15 12 (0.55%) Cloud Foundry
16 11 (0.50%) Oracle SOA Suite
17 10 (0.46%) GemFire
Applications
1 66 (3.03%) Microsoft Excel
2 46 (2.11%) MS Visio
3 8 (0.37%) Microsoft Office
4 4 (0.18%) Chatbot
4 4 (0.18%) SPSS
5 3 (0.14%) Microsoft PowerPoint
6 2 (0.092%) Spreadsheet
7 1 (0.046%) GNU Octave
7 1 (0.046%) InVision
7 1 (0.046%) Shockwave
Business Applications
1 10 (0.46%) SAP Oil and Gas
2 7 (0.32%) Dynamics CRM
3 5 (0.23%) Oracle BPM
4 4 (0.18%) Dynamics AX
5 3 (0.14%) SAP ERP
5 3 (0.14%) SAP MM
5 3 (0.14%) SAP R/3
6 2 (0.092%) ION MarketView
6 2 (0.092%) Remedy ITSM
6 2 (0.092%) Summit
7 1 (0.046%) Actimize
7 1 (0.046%) Dynamics NAV
7 1 (0.046%) SAP MDG
7 1 (0.046%) SAP MDM
7 1 (0.046%) SAP S/4HANA
Cloud Services
1 574 (26.34%) Amazon AWS
2 278 (12.76%) Microsoft Azure
3 106 (4.86%) Amazon S3
3 106 (4.86%) Google Cloud Platform
4 97 (4.45%) Amazon EMR
5 91 (4.18%) Amazon EC2
6 83 (3.81%) GitHub
7 59 (2.71%) PaaS
8 51 (2.34%) Virtual Private Cloud
9 47 (2.16%) Amazon Kinesis
10 43 (1.97%) Azure Data Factory
11 41 (1.88%) AWS CloudFormation
12 34 (1.56%) Serverless
13 25 (1.15%) IaaS
14 23 (1.06%) Cloud Computing
14 23 (1.06%) SaaS
15 21 (0.96%) AWS Lambda
16 15 (0.69%) Azure Machine Learning
17 12 (0.55%) Heroku
18 10 (0.46%) Akamai
Communications & Networking
1 87 (3.99%) Firewall
2 82 (3.76%) Kerberos
3 48 (2.20%) HTTP
4 47 (2.16%) DNS
5 44 (2.02%) TCP/IP
6 33 (1.51%) SSH
6 33 (1.51%) SSL
7 30 (1.38%) Internet
8 28 (1.28%) VPN
9 24 (1.10%) Network Security
10 21 (0.96%) SAN
11 18 (0.83%) FTP
12 13 (0.60%) OpenLDAP
13 12 (0.55%) NAS
14 11 (0.50%) AMQP
15 10 (0.46%) DAB
15 10 (0.46%) WAN
16 9 (0.41%) HTTPS
16 9 (0.41%) Unicast
17 7 (0.32%) RTMP
Database & Business Intelligence
1 1,321 (60.62%) Big Data
2 692 (31.76%) Apache Hive
3 390 (17.90%) NoSQL
4 370 (16.98%) HBase
5 304 (13.95%) MongoDB
6 291 (13.35%) Data Warehouse
7 275 (12.62%) SQL Server
8 219 (10.05%) Relational Database
9 212 (9.73%) Impala
10 191 (8.77%) Apache Cassandra
11 179 (8.21%) Data Lake
12 146 (6.70%) Sqoop
13 130 (5.97%) PostgreSQL
14 124 (5.69%) MySQL
15 120 (5.51%) Power BI
16 89 (4.08%) SQL Server Integration Services
17 88 (4.04%) RDBMS
18 87 (3.99%) Amazon Redshift
19 59 (2.71%) OLAP
20 55 (2.52%) Enterprise Data Warehouse
Development Applications
1 421 (19.32%) Jenkins
2 281 (12.90%) Git (software)
3 145 (6.65%) JIRA
4 108 (4.96%) Maven
5 65 (2.98%) Atlassian Bamboo
6 57 (2.62%) Cucumber
7 53 (2.43%) Artifactory
8 41 (1.88%) Selenium
9 38 (1.74%) Vagrant
10 37 (1.70%) Hudson
11 36 (1.65%) Visual Studio Team System
12 33 (1.51%) Subversion
13 32 (1.47%) Gradle
14 27 (1.24%) Sonatype Nexus
15 25 (1.15%) GitLab
16 24 (1.10%) IntelliJ
16 24 (1.10%) JUnit
17 21 (0.96%) TeamCity
18 20 (0.92%) Bitbucket
18 20 (0.92%) Eclipse
General
1 589 (27.03%) Finance
2 210 (9.64%) Banking
3 92 (4.22%) Front Office
4 70 (3.21%) Retail
5 47 (2.16%) Investment Banking
6 46 (2.11%) Legal
7 38 (1.74%) Telecoms
8 29 (1.33%) Games
8 29 (1.33%) Marketing
9 23 (1.06%) Publishing
10 13 (0.60%) Advertising
10 13 (0.60%) Financial Institution
11 12 (0.55%) Retail Banking
12 10 (0.46%) Manufacturing
12 10 (0.46%) Pharmaceutical
13 7 (0.32%) Back Office
14 6 (0.28%) Electronics
15 3 (0.14%) Automotive
16 2 (0.092%) Corporate Banking
16 2 (0.092%) French Language
Job Titles
1 696 (31.94%) Developer
2 408 (18.72%) Data Engineer
3 298 (13.68%) Java Developer
4 276 (12.67%) Big Data Engineer
5 270 (12.39%) Architect
6 185 (8.49%) Hadoop Engineer
7 163 (7.48%) Big Data Developer
8 146 (6.70%) Data Architect
9 137 (6.29%) Consultant
10 129 (5.92%) DevOps Engineer
11 105 (4.82%) AWS Engineer
12 102 (4.68%) Analyst
13 101 (4.64%) Data Scientist
14 99 (4.54%) Hadoop Developer
15 90 (4.13%) Big Data Architect
16 89 (4.08%) Software Engineer
17 87 (3.99%) Senior Developer
18 79 (3.63%) Python Engineer
19 75 (3.44%) DevOps Consultant
20 66 (3.03%) Java Engineer
Libraries, Frameworks & Software Standards
1 688 (31.57%) Kafka
2 211 (9.68%) Spring
3 155 (7.11%) Web Services
4 135 (6.20%) REST
5 134 (6.15%) JSON
6 118 (5.42%) AngularJS
7 102 (4.68%) RESTful
7 102 (4.68%) Spark Streaming
8 101 (4.64%) RabbitMQ
9 98 (4.50%) Hibernate
10 87 (3.99%) Spring Boot
11 85 (3.90%) HTML
12 76 (3.49%) React
13 72 (3.30%) J2EE
14 70 (3.21%) .NET
15 66 (3.03%) Elastic Stack
16 62 (2.85%) Natural Language Processing
17 54 (2.48%) NumPy
18 51 (2.34%) JSON Schema
18 51 (2.34%) Middleware
Miscellaneous
1 234 (10.74%) Greenfield Project
2 219 (10.05%) Analytical Skills
3 161 (7.39%) Algorithms
4 116 (5.32%) Computer Science
5 65 (2.98%) Data Structures
6 48 (2.20%) Wiki
7 39 (1.79%) Fintech
7 39 (1.79%) Self-Motivation
8 33 (1.51%) Enterprise Software
9 32 (1.47%) User Experience
10 29 (1.33%) Virtual Team
11 27 (1.24%) Cyberthreat
12 24 (1.10%) PKI
13 21 (0.96%) Data Centre
14 19 (0.87%) Management Information System
15 17 (0.78%) Derivative
16 16 (0.73%) Public Cloud
17 15 (0.69%) HoloLens
18 14 (0.64%) Linux Command Line
19 12 (0.55%) Replication
Operating Systems
1 478 (21.94%) Linux
2 256 (11.75%) Unix
3 131 (6.01%) Windows
4 57 (2.62%) CentOS
5 53 (2.43%) Red Hat Enterprise Linux
6 16 (0.73%) Solaris
7 12 (0.55%) AIX
8 8 (0.37%) Android
9 7 (0.32%) CoreOS
10 4 (0.18%) Apple iOS
11 3 (0.14%) DC/OS
11 3 (0.14%) Mac OS X
11 3 (0.14%) Ubuntu
12 2 (0.092%) Windows 10
12 2 (0.092%) Windows Server
13 1 (0.046%) Data ONTAP
13 1 (0.046%) Debian
13 1 (0.046%) Mesosphere
13 1 (0.046%) SUSE
13 1 (0.046%) SUSE Linux Enterprise Server
Processes & Methodologies
1 893 (40.98%) Agile Software Development
2 536 (24.60%) Analytics
3 480 (22.03%) DevOps
4 410 (18.82%) Data Science
5 357 (16.38%) ETL
6 322 (14.78%) Scrum
7 309 (14.18%) Business Intelligence
8 305 (14.00%) TDD
9 282 (12.94%) Machine Learning
10 275 (12.62%) Data Modelling
11 240 (11.01%) Software Engineering
12 239 (10.97%) Visualisation
13 231 (10.60%) Continuous Integration
14 220 (10.10%) Microservices
15 207 (9.50%) BDD
16 184 (8.44%) Open Source
17 155 (7.11%) Test Automation
18 153 (7.02%) MapReduce
19 148 (6.79%) Problem-Solving
20 142 (6.52%) Data Analytics
Programming Languages
1 905 (41.53%) Python
2 869 (39.88%) Java
3 664 (30.47%) SQL
4 430 (19.73%) Scala
5 179 (8.21%) Bash Shell
6 174 (7.99%) JavaScript
7 172 (7.89%) R
8 170 (7.80%) Java 8
9 143 (6.56%) Shell Script
10 102 (4.68%) C++
11 98 (4.50%) Groovy
12 88 (4.04%) PowerShell
13 77 (3.53%) Ruby
14 69 (3.17%) C#
15 64 (2.94%) Perl
16 47 (2.16%) MATLAB
17 45 (2.07%) Korn
18 40 (1.84%) C
19 36 (1.65%) T-SQL
20 26 (1.19%) Go
Qualifications
1 209 (9.59%) Security Cleared
2 126 (5.78%) Degree
3 108 (4.96%) SC Cleared
4 46 (2.11%) Computer Science Degree
5 43 (1.97%) AWS Certification
6 40 (1.84%) PhD
7 27 (1.24%) CISSP
8 23 (1.06%) BPSS Clearance
9 21 (0.96%) CISM
9 21 (0.96%) Master's Degree
10 18 (0.83%) ISTQB
11 16 (0.73%) BC Cleared
12 12 (0.55%) DV Cleared
12 12 (0.55%) Microsoft Certification
13 8 (0.37%) SANS
14 7 (0.32%) CEH
14 7 (0.32%) MCSE
14 7 (0.32%) Numerate Degree
15 4 (0.18%) ISEB
16 3 (0.14%) CCA
Quality Assurance & Compliance
1 125 (5.74%) Data Quality
2 74 (3.40%) QA
3 49 (2.25%) GDPR
4 16 (0.73%) FINRA
5 11 (0.50%) ISO/IEC 27001
6 8 (0.37%) PMO
6 8 (0.37%) SLA
7 2 (0.092%) COBIT
7 2 (0.092%) Sarbanes-Oxley
8 1 (0.046%) Dodd-Frank
8 1 (0.046%) EMIR
8 1 (0.046%) MiFID
8 1 (0.046%) NIST 800
8 1 (0.046%) PSD2
System Software
1 340 (15.60%) Docker
2 289 (13.26%) HDFS
3 100 (4.59%) Apache Flume
4 72 (3.30%) Apache ZooKeeper
5 44 (2.02%) Active Directory
6 25 (1.15%) VMware Infrastructure
7 7 (0.32%) NFS
8 5 (0.23%) Virtual Machines
9 4 (0.18%) Veritas Cluster Server
10 3 (0.14%) SRDF
10 3 (0.14%) vSphere
10 3 (0.14%) Xen
11 2 (0.092%) iptables
11 2 (0.092%) LXC
11 2 (0.092%) Tivoli Workload Scheduler
11 2 (0.092%) Virtual Servers
12 1 (0.046%) Ceph
12 1 (0.046%) Lustre
12 1 (0.046%) OpenDJ
12 1 (0.046%) OpenIDM
Systems Management
1 413 (18.95%) Ansible
2 284 (13.03%) Puppet
3 244 (11.20%) Terraform
4 239 (10.97%) Opscode Chef
5 215 (9.87%) Kubernetes
6 179 (8.21%) Hadoop YARN
7 160 (7.34%) Oozie
8 63 (2.89%) Cobbler
9 49 (2.25%) Kibana
10 40 (1.84%) logstash
11 28 (1.28%) Ambari
12 25 (1.15%) Salt
13 19 (0.87%) ZABBIX
14 17 (0.78%) Mesos
14 17 (0.78%) Nagios
15 15 (0.69%) Docker Swarm
15 15 (0.69%) ITRS Geneos
16 12 (0.55%) Opsview
17 11 (0.50%) Control-M
18 7 (0.32%) Tivoli
Vendors
1 295 (13.54%) Cloudera
2 269 (12.35%) Oracle
3 178 (8.17%) Hortonworks
4 174 (7.99%) Tableau
5 161 (7.39%) Microsoft
6 118 (5.42%) Google
7 101 (4.64%) Informatica
8 61 (2.80%) Talend
9 58 (2.66%) Teradata
10 55 (2.52%) SAP
11 52 (2.39%) Splunk
12 51 (2.34%) Red Hat
13 46 (2.11%) IBM
14 44 (2.02%) MapR
15 40 (1.84%) Ab Initio
16 36 (1.65%) Alteryx
17 34 (1.56%) VMware
18 26 (1.19%) Sybase
19 21 (0.96%) NetApp
20 20 (0.92%) TIBCO