Period
to 24 April 2018

The following table provides summary statistics for contract job vacancies with a requirement for Hadoop skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Hadoop over the 6 months to 24 April 2018 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Hadoop
UK
6 months to
24 Apr 2018
Same period 2017 Same period 2016
Rank 118 115 185
Rank change year-on-year -3 +70 +61
Contract jobs citing Hadoop 2,168 1,933 1,459
As % of all contract IT jobs advertised in the UK 2.15% 2.12% 1.38%
As % of the Database & Business Intelligence category 9.60% 9.43% 6.17%
Number of daily rates quoted 1,511 1,263 1,012
UK median daily rate £550 £525 £550
Median daily rate % change year-on-year +4.76% -4.55% +4.76%
10th Percentile £400 £375 £375
90th Percentile £675 £675 £700
UK excluding London median daily rate £500 £475 £500
% change year-on-year +5.26% -5.00% +0.25%
Number of hourly rates quoted 8 5 3
UK median hourly rate £41.25 £47.00 £38.00
Median hourly rate % change year-on-year -12.23% +23.68% +16.03%
UK excluding London median hourly rate £34.00 £39.23 £38.00
% change year-on-year -13.32% +3.22% +16.03%

Hadoop is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Contract vacancies with a requirement for database or business intelligence skills 22,580 20,501 23,632
As % of all contract IT jobs advertised in the UK 22.34% 22.48% 22.31%
Number of daily rates quoted 15,667 13,709 15,895
UK median daily rate £470 £438 £425
Median daily rate % change year-on-year +7.43% +2.94% +6.25%
10th Percentile £312 £292 £280
90th Percentile £625 £610 £600
UK excluding London median daily rate £413 £390 £375
% change year-on-year +5.77% +4.00% +2.04%
Number of hourly rates quoted 242 259 393
UK median hourly rate £34.25 £27.50 £25.00
Median hourly rate % change year-on-year +24.55% +10.00% -16.67%
10th Percentile £14.62 £9.35 £12.00
90th Percentile £55.00 £48.15 £52.50
UK excluding London median hourly rate £37.50 £25.00 £24.50
% change year-on-year +50.00% +2.04% -14.60%

Hadoop
Job Vacancy Trend

Job postings citing Hadoop as a percentage of all IT jobs advertised.

Job vacancy trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Hadoop.

Contractor daily rate trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing Hadoop over the 6 months to 24 April 2018.

Contractor daily rate histogram for Hadoop in the UK

Hadoop
Contractor Hourly Rate Trend

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing Hadoop.

Contractor hourly rate trend for Hadoop in the UK

Hadoop
Contractor Hourly Rate Histogram

The hourly rate distribution of IT jobs citing Hadoop over the 6 months to 24 April 2018.

Contractor hourly rate histogram for Hadoop in the UK

Hadoop
Top 16 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Hadoop within the UK over the 6 months to 24 April 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -11 2,053 £550 +4.76% 390
London -16 1,283 £550 - 244
UK excluding London +55 862 £500 +5.26% 163
South East +27 365 £525 +10.53% 51
North of England +46 157 £505 +6.32% 38
South West +12 98 £453 -9.50% 20
Midlands +29 86 £500 - 24
Yorkshire +31 80 £490 +8.89% 11
North West +7 77 £538 +6.70% 25
Scotland +18 68 £400 -15.79% 13
East of England +10 64 £475 +11.76% 13
East Midlands +51 62 £500 -4.76% 8
West Midlands -10 24 £488 -2.50% 16
Wales -15 22 £550 +7.32% 2
North East +19 5 £610 +22.00% 2
Northern Ireland -6 3 - - 2

For the 6 months to 24 April 2018, IT contractor jobs citing Hadoop also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for Hadoop.

1 1,463 (67.48%) Big Data
2 1,108 (51.11%) Apache Spark
3 872 (40.22%) Java
4 805 (37.13%) Agile Software Development
5 769 (35.47%) Python
6 696 (32.10%) SQL
7 682 (31.46%) Kafka
8 673 (31.04%) Apache Hive
9 596 (27.49%) Scala
10 571 (26.34%) Analytics
11 524 (24.17%) Linux
12 508 (23.43%) Finance
13 499 (23.02%) HBase
14 498 (22.97%) Amazon AWS
15 413 (19.05%) DevOps
16 378 (17.44%) NoSQL
17 374 (17.25%) Cloudera
18 371 (17.11%) Data Science
19 361 (16.65%) ETL
20 347 (16.01%) Data Warehouse
21 330 (15.22%) Docker
22 327 (15.08%) Jenkins
23 325 (14.99%) HDFS
24 324 (14.94%) Apache
25 303 (13.98%) TDD
26 290 (13.38%) Git (software)
27 288 (13.28%) Data Modelling
28 281 (12.96%) Machine Learning
29 278 (12.82%) Banking
30 273 (12.59%) Ansible

Hadoop
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 1,108 (51.11%) Apache Spark
2 324 (14.94%) Apache
3 179 (8.26%) Apache Pig
4 170 (7.84%) Elasticsearch
5 117 (5.40%) SAS
6 101 (4.66%) Tomcat
7 81 (3.74%) Confluence
8 70 (3.23%) JBoss
9 67 (3.09%) nginx
10 58 (2.68%) Apache Solr
11 33 (1.52%) Cloud Foundry
11 33 (1.52%) Jetty
12 29 (1.34%) Oracle SOA Suite
13 26 (1.20%) Jupyter
14 24 (1.11%) WebLogic
15 14 (0.65%) Apache Airflow
15 14 (0.65%) PowerExchange
15 14 (0.65%) SharePoint
16 12 (0.55%) GemFire
16 12 (0.55%) Oracle Coherence
Applications
1 20 (0.92%) Microsoft Excel
2 10 (0.46%) SPSS
3 8 (0.37%) Chatbot
3 8 (0.37%) HipChat
3 8 (0.37%) Microsoft Office
4 3 (0.14%) Microsoft PowerPoint
4 3 (0.14%) Stata
5 2 (0.092%) MS Visio
5 2 (0.092%) Pipeline Pilot
6 1 (0.046%) Gephi
6 1 (0.046%) Shockwave
Business Applications
1 15 (0.69%) SAP Oil and Gas
2 8 (0.37%) Oracle BPM
3 5 (0.23%) ION MarketView
4 3 (0.14%) Actimize
4 3 (0.14%) Dynamics CRM
5 2 (0.092%) Oracle Financials
5 2 (0.092%) Summit
6 1 (0.046%) Distributed Ledger
6 1 (0.046%) Finacle
6 1 (0.046%) FLEXCUBE
6 1 (0.046%) MoSes
6 1 (0.046%) Revive Adserver
Cloud Services
1 498 (22.97%) Amazon AWS
2 230 (10.61%) Microsoft Azure
3 87 (4.01%) GitHub
4 70 (3.23%) Amazon S3
5 61 (2.81%) Google Cloud Platform
6 56 (2.58%) Serverless
7 46 (2.12%) Amazon EMR
8 45 (2.08%) AWS Lambda
9 44 (2.03%) Amazon Kinesis
9 44 (2.03%) PaaS
10 33 (1.52%) Amazon EC2
11 32 (1.48%) Azure Data Factory
12 30 (1.38%) AWS CloudFormation
13 26 (1.20%) SaaS
14 25 (1.15%) IaaS
15 23 (1.06%) Google Analytics
16 21 (0.97%) Google Tag Manager
17 15 (0.69%) Data Lake Analytics
18 13 (0.60%) Amazon CloudWatch
19 12 (0.55%) Virtual Private Cloud
Communications & Networking
1 107 (4.94%) Kerberos
2 49 (2.26%) Network Security
3 38 (1.75%) SSL
4 36 (1.66%) Firewall
5 26 (1.20%) Internet
6 24 (1.11%) HTTP
7 19 (0.88%) VPN
8 18 (0.83%) DNS
9 16 (0.74%) TCP/IP
10 11 (0.51%) OpenLDAP
11 10 (0.46%) SSH
12 8 (0.37%) DSL
12 8 (0.37%) ICMP
13 7 (0.32%) SAN
14 6 (0.28%) SMS
15 5 (0.23%) NAS
16 3 (0.14%) AMQP
16 3 (0.14%) iSCSI
16 3 (0.14%) Multicast
17 2 (0.092%) FCoE
Database & Business Intelligence
1 1,463 (67.48%) Big Data
2 673 (31.04%) Apache Hive
3 499 (23.02%) HBase
4 378 (17.44%) NoSQL
5 347 (16.01%) Data Warehouse
6 272 (12.55%) MongoDB
7 252 (11.62%) SQL Server
8 228 (10.52%) Data Lake
9 227 (10.47%) PostgreSQL
10 198 (9.13%) Apache Cassandra
11 196 (9.04%) Impala
12 190 (8.76%) Relational Database
13 172 (7.93%) Sqoop
14 147 (6.78%) MySQL
15 95 (4.38%) Amazon Redshift
16 92 (4.24%) Power BI
17 79 (3.64%) RDBMS
18 74 (3.41%) OLAP
19 67 (3.09%) Enterprise Data Warehouse
20 61 (2.81%) Redis
Development Applications
1 327 (15.08%) Jenkins
2 290 (13.38%) Git (software)
3 146 (6.73%) JIRA
4 80 (3.69%) Maven
5 52 (2.40%) GitLab
5 52 (2.40%) JUnit
6 49 (2.26%) Subversion
7 47 (2.17%) Gradle
8 45 (2.08%) Artifactory
9 44 (2.03%) Vagrant
10 43 (1.98%) Cucumber
11 42 (1.94%) Sonatype Nexus
12 38 (1.75%) TeamCity
13 36 (1.66%) Team Foundation Server
14 31 (1.43%) ERwin
15 28 (1.29%) Selenium
16 25 (1.15%) Mockito
17 24 (1.11%) Visual Studio Team System
18 23 (1.06%) IntelliJ
19 21 (0.97%) Eclipse
General
1 508 (23.43%) Finance
2 278 (12.82%) Banking
3 113 (5.21%) Retail
4 75 (3.46%) Marketing
5 53 (2.44%) Investment Banking
6 45 (2.08%) Games
7 35 (1.61%) Telecoms
8 30 (1.38%) Retail Banking
9 29 (1.34%) Front Office
10 26 (1.20%) Manufacturing
11 16 (0.74%) Publishing
12 15 (0.69%) Electronics
13 14 (0.65%) Advertising
13 14 (0.65%) Automotive
14 12 (0.55%) Corporate Banking
15 11 (0.51%) Legal
16 10 (0.46%) Financial Institution
16 10 (0.46%) Private Banking
17 7 (0.32%) French Language
17 7 (0.32%) Pharmaceutical
Job Titles
1 658 (30.35%) Developer
2 393 (18.13%) Data Engineer
3 284 (13.10%) Architect
4 268 (12.36%) Big Data Developer
5 251 (11.58%) Big Data Engineer
6 247 (11.39%) Java Developer
7 180 (8.30%) Data Architect
8 171 (7.89%) Hadoop Engineer
9 147 (6.78%) DevOps Engineer
10 146 (6.73%) Consultant
11 134 (6.18%) Hadoop Developer
12 126 (5.81%) Analyst
13 125 (5.77%) Data Scientist
14 104 (4.80%) Big Data Architect
15 89 (4.11%) Senior Developer
16 87 (4.01%) Scala Developer
17 86 (3.97%) AWS Engineer
18 66 (3.04%) Apache Spark Developer
19 62 (2.86%) Data Analyst
19 62 (2.86%) Senior Java Developer
Libraries, Frameworks & Software Standards
1 682 (31.46%) Kafka
2 180 (8.30%) Spring
3 160 (7.38%) REST
4 157 (7.24%) JSON
5 156 (7.20%) Web Services
6 153 (7.06%) RESTful
7 114 (5.26%) Spark Streaming
8 108 (4.98%) RabbitMQ
9 105 (4.84%) Hibernate
10 101 (4.66%) Middleware
11 86 (3.97%) AngularJS
12 83 (3.83%) XML
13 76 (3.51%) J2EE
14 72 (3.32%) HTML
15 70 (3.23%) Elastic Stack
16 69 (3.18%) SOAP
17 67 (3.09%) Node.js
18 58 (2.68%) Apache Avro
19 52 (2.40%) React
19 52 (2.40%) Spring Boot
Miscellaneous
1 214 (9.87%) Analytical Skills
2 136 (6.27%) Computer Science
3 134 (6.18%) Algorithms
4 122 (5.63%) Greenfield Project
5 58 (2.68%) Data Structures
6 56 (2.58%) Virtual Team
7 53 (2.44%) Linux Command Line
8 49 (2.26%) Self-Motivation
9 38 (1.75%) Data Centre
10 37 (1.71%) Fintech
11 34 (1.57%) User Experience
12 33 (1.52%) Management Information System
13 28 (1.29%) Public Cloud
14 25 (1.15%) Enterprise Software
15 24 (1.11%) Derivative
16 23 (1.06%) Market Risk
17 18 (0.83%) Replication
18 17 (0.78%) Credit Risk
19 16 (0.74%) Mobile App
20 13 (0.60%) Digital Media
Operating Systems
1 524 (24.17%) Linux
2 234 (10.79%) Unix
3 83 (3.83%) CentOS
4 77 (3.55%) Red Hat Enterprise Linux
5 59 (2.72%) Windows
6 36 (1.66%) Ubuntu
7 20 (0.92%) CoreOS
8 14 (0.65%) Windows Server
9 11 (0.51%) Android
9 11 (0.51%) Apple iOS
9 11 (0.51%) Debian
10 8 (0.37%) Windows Server 2008
11 5 (0.23%) Solaris
12 4 (0.18%) DC/OS
13 3 (0.14%) Mac OS X
13 3 (0.14%) Mesosphere
13 3 (0.14%) Windows Server 2003
13 3 (0.14%) Windows Vista
14 2 (0.092%) AIX
14 2 (0.092%) SUSE Linux Enterprise Server
Processes & Methodologies
1 805 (37.13%) Agile Software Development
2 571 (26.34%) Analytics
3 413 (19.05%) DevOps
4 371 (17.11%) Data Science
5 361 (16.65%) ETL
6 303 (13.98%) TDD
7 288 (13.28%) Data Modelling
8 281 (12.96%) Machine Learning
9 265 (12.22%) Business Intelligence
10 251 (11.58%) Continuous Integration
11 239 (11.02%) Scrum
12 202 (9.32%) Open Source
13 192 (8.86%) MapReduce
13 192 (8.86%) Visualisation
14 174 (8.03%) Microservices
15 161 (7.43%) BDD
16 152 (7.01%) Problem-Solving
17 143 (6.60%) Continuous Delivery
17 143 (6.60%) Data Analytics
18 139 (6.41%) Test Automation
Programming Languages
1 872 (40.22%) Java
2 769 (35.47%) Python
3 696 (32.10%) SQL
4 596 (27.49%) Scala
5 183 (8.44%) R
6 162 (7.47%) JavaScript
7 147 (6.78%) Bash Shell
8 119 (5.49%) Shell Script
9 103 (4.75%) Java 8
10 61 (2.81%) Groovy
10 61 (2.81%) Ruby
11 53 (2.44%) C#
12 51 (2.35%) C++
13 47 (2.17%) Perl
14 36 (1.66%) C
15 31 (1.43%) MATLAB
15 31 (1.43%) PowerShell
16 29 (1.34%) Go
17 25 (1.15%) PL/SQL
18 24 (1.11%) T-SQL
Qualifications
1 243 (11.21%) Security Cleared
2 182 (8.39%) SC Cleared
3 157 (7.24%) Degree
4 38 (1.75%) Computer Science Degree
5 33 (1.52%) Master's Degree
6 31 (1.43%) DV Cleared
7 27 (1.25%) Numerate Degree
7 27 (1.25%) PhD
8 9 (0.42%) Microsoft Certification
9 8 (0.37%) CHECK Team Member
10 7 (0.32%) CISSP
11 6 (0.28%) BPSS Clearance
11 6 (0.28%) CESG Certified Professional
11 6 (0.28%) CLAS
11 6 (0.28%) ISTQB
12 4 (0.18%) BC Cleared
12 4 (0.18%) TOGAF Certification
13 3 (0.14%) CTC Cleared
13 3 (0.14%) MCITP
13 3 (0.14%) VMware Certified Professional (VCP)
Quality Assurance & Compliance
1 137 (6.32%) Data Quality
2 86 (3.97%) QA
3 39 (1.80%) GDPR
4 10 (0.46%) Disclosure Scotland
5 9 (0.42%) GCP
5 9 (0.42%) MiFID
6 8 (0.37%) SLA
7 7 (0.32%) PMO
8 6 (0.28%) JSP 440
8 6 (0.28%) RMADS
9 5 (0.23%) EMIR
9 5 (0.23%) Sarbanes-Oxley
10 4 (0.18%) Dodd-Frank
10 4 (0.18%) ISO/IEC 20000
11 3 (0.14%) ISO/IEC 27001
11 3 (0.14%) NIST 800
12 2 (0.092%) GAAP
13 1 (0.046%) Basel II
13 1 (0.046%) COBIT
13 1 (0.046%) HL7
System Software
1 330 (15.22%) Docker
2 325 (14.99%) HDFS
3 209 (9.64%) Apache Flume
4 148 (6.83%) Apache ZooKeeper
5 35 (1.61%) Active Directory
6 24 (1.11%) VMware Infrastructure
7 7 (0.32%) Virtual Machines
8 6 (0.28%) Squid
9 5 (0.23%) Hyper-V
9 5 (0.23%) NFS
9 5 (0.23%) Xen
10 4 (0.18%) Ceph
10 4 (0.18%) KVM
10 4 (0.18%) vCloud
11 3 (0.14%) Lustre
12 2 (0.092%) OpenDJ
12 2 (0.092%) OpenIDM
12 2 (0.092%) Oracle RAC
12 2 (0.092%) Varnish
12 2 (0.092%) VMware ESXi
Systems Management
1 273 (12.59%) Ansible
2 220 (10.15%) Oozie
3 173 (7.98%) Hadoop YARN
4 151 (6.96%) Puppet
5 148 (6.83%) Kubernetes
6 125 (5.77%) Terraform
7 111 (5.12%) Opscode Chef
8 60 (2.77%) Ambari
9 55 (2.54%) Kibana
10 40 (1.85%) Mesos
11 37 (1.71%) logstash
12 28 (1.29%) Salt
13 27 (1.25%) Cobbler
14 26 (1.20%) Prometheus
15 20 (0.92%) Consul
15 20 (0.92%) Host Intrusion Detection System
15 20 (0.92%) Nagios
15 20 (0.92%) Network Intrusion Detection System
16 19 (0.88%) Sysdig
16 19 (0.88%) ZABBIX
Vendors
1 374 (17.25%) Cloudera
2 271 (12.50%) Oracle
3 226 (10.42%) Hortonworks
4 154 (7.10%) Microsoft
5 140 (6.46%) Tableau
6 110 (5.07%) Google
7 99 (4.57%) Informatica
8 96 (4.43%) Talend
8 96 (4.43%) Teradata
9 90 (4.15%) SAP
10 89 (4.11%) MapR
11 85 (3.92%) Splunk
12 66 (3.04%) IBM
13 60 (2.77%) Red Hat
14 49 (2.26%) Salesforce.com
15 45 (2.08%) Alteryx
16 41 (1.89%) Ab Initio
17 32 (1.48%) VMware
18 28 (1.29%) CA
19 27 (1.25%) SaltStack