Period
to 22 October 2017

The following table provides summary statistics for contract job vacancies with a requirement for Hadoop skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Hadoop over the 6 months to 22 October 2017 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Hadoop
UK
6 months to
22 Oct 2017
Same period 2016 Same period 2015
Rank 113 134 221
Rank change year-on-year +21 +87 +146
Contract jobs citing Hadoop 2145 2044 1266
As % of all contract IT jobs advertised in the UK 2.16% 1.94% 1.11%
As % of the Database & Business Intelligence category 9.72% 8.61% 4.76%
Number of daily rates quoted 1472 1317 824
UK median daily rate £525 £525 £525
10th Percentile £376 £363 £363
90th Percentile £687 £713 £688
UK excluding London median daily rate £475 £458 £475
% change year-on-year +3.83% -3.68% +7.04%
Number of hourly rates quoted 9 6 6
UK median hourly rate £35.00 £45.75 £35.53
Median hourly rate % change year-on-year -23.50% +28.78% +11.02%
10th Percentile £16.56 £35.00 £29.53
90th Percentile £50.00 £46.38 £47.50
UK excluding London median hourly rate £35.00 £45.75 £33.55
% change year-on-year -23.50% +36.36% -

Hadoop is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Contract vacancies with a requirement for database or business intelligence skills 22060 23749 26586
As % of all contract IT jobs advertised in the UK 22.20% 22.53% 23.31%
Number of daily rates quoted 14926 15815 17444
UK median daily rate £450 £425 £400
Median daily rate % change year-on-year +5.88% +6.25% +3.23%
10th Percentile £308 £281 £263
90th Percentile £625 £588 £575
UK excluding London median daily rate £400 £375 £375
% change year-on-year +6.67% - +7.14%
Number of hourly rates quoted 275 389 389
UK median hourly rate £30.00 £29.50 £28.00
Median hourly rate % change year-on-year +1.69% +5.36% +12.00%
10th Percentile £13.69 £10.85 £12.46
90th Percentile £54.50 £50.20 £52.48
UK excluding London median hourly rate £32.50 £28.50 £28.69
% change year-on-year +14.04% -0.66% +27.51%

Hadoop
Job Vacancy Trend

Job postings citing Hadoop as a percentage of all IT jobs advertised.

Job vacancy trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Hadoop.

Contractor daily rate trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing Hadoop over the 6 months to 22 October 2017.

Contractor daily rate histogram for Hadoop in the UK

Hadoop
Contractor Hourly Rate Trend

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing Hadoop.

Contractor hourly rate trend for Hadoop in the UK

Hadoop
Contractor Hourly Rate Histogram

The hourly rate distribution of IT jobs citing Hadoop over the 6 months to 22 October 2017.

Contractor hourly rate histogram for Hadoop in the UK

Hadoop
Top 16 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Hadoop within the UK over the 6 months to 22 October 2017. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +18 2041 £538 +2.38% 155
London +1 1401 £550 - 105
UK excluding London +50 722 £475 +3.83% 53
South East +22 292 £475 +5.56% 19
North of England +67 130 £488 +2.63% 14
East of England +11 90 £488 +39.29% 9
South West +21 86 £475 +5.56% 6
Yorkshire +19 69 £475 - 6
Scotland +52 66 £400 -17.95% 3
North West +29 57 £600 +29.73% 7
Midlands +15 33 £525 +5.00% 2
West Midlands +13 22 £475 -5.00%
Northern Ireland -33 17 £463 -
North East +82 16 £610 +22.00% 1
East Midlands -2 11 £525 +23.53% 2
Wales +5 8 £500 -2.44%

For the 6 months to 22 October 2017, IT contractor jobs citing Hadoop also mentioned the following skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all contract ads with a requirement for Hadoop.

1 1515 (70.63%) Big Data
2 1087 (50.68%) Apache Spark
3 1016 (47.37%) Java
4 881 (41.07%) Agile Software Development
5 666 (31.05%) Python
6 647 (30.16%) Apache Hive
7 619 (28.86%) SQL
8 574 (26.76%) Kafka
9 525 (24.48%) HBase
10 487 (22.70%) Scala
11 471 (21.96%) Linux
12 469 (21.86%) TDD
13 466 (21.72%) Finance
14 461 (21.49%) Analytics
15 433 (20.19%) Scrum
16 432 (20.14%) Amazon AWS
17 397 (18.51%) HDFS
18 371 (17.30%) DevOps
19 345 (16.08%) NoSQL
20 341 (15.90%) MapReduce
21 339 (15.80%) Cloudera
22 338 (15.76%) ETL
23 334 (15.57%) Jenkins
24 324 (15.10%) Docker
25 322 (15.01%) Banking
26 303 (14.13%) Security Cleared
27 300 (13.99%) BDD
28 286 (13.33%) JIRA
29 283 (13.19%) Spring
30 275 (12.82%) Unix

Hadoop
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 1087 (50.68%) Apache Spark
2 250 (11.66%) Apache
3 165 (7.69%) Confluence
4 145 (6.76%) Apache Pig
5 131 (6.11%) Elasticsearch
6 130 (6.06%) Apache Solr
7 111 (5.17%) SAS
8 81 (3.78%) Tomcat
9 55 (2.56%) JBoss
10 30 (1.40%) Oracle SOA Suite
11 29 (1.35%) Jetty
12 25 (1.17%) WebSphere
13 24 (1.12%) nginx
13 24 (1.12%) OpenStack
14 18 (0.84%) WebLogic
15 16 (0.75%) Apache Airflow
16 14 (0.65%) Cloud Foundry
17 12 (0.56%) GemFire
18 11 (0.51%) GlassFish
18 11 (0.51%) IIS
Applications
1 29 (1.35%) SPSS
2 23 (1.07%) Microsoft Excel
3 8 (0.37%) Pipeline Pilot
3 8 (0.37%) Stata
4 7 (0.33%) Microsoft Office
5 2 (0.093%) Microsoft Project
6 1 (0.047%) Adobe Illustrator
6 1 (0.047%) GNU Octave
6 1 (0.047%) HipChat
6 1 (0.047%) Microsoft PowerPoint
6 1 (0.047%) MS Visio
6 1 (0.047%) Photoshop
Business Applications
1 10 (0.47%) Oracle BPM
2 9 (0.42%) Actimize
3 2 (0.093%) SAP CRM
3 2 (0.093%) SAP HR
3 2 (0.093%) WebTrends
Cloud Services
1 432 (20.14%) Amazon AWS
2 134 (6.25%) GitHub
3 124 (5.78%) Microsoft Azure
4 58 (2.70%) Google Cloud Platform
5 48 (2.24%) Amazon EC2
5 48 (2.24%) Amazon S3
6 38 (1.77%) IBM Cloud
7 36 (1.68%) Bluemix
8 35 (1.63%) PaaS
9 32 (1.49%) Virtual Private Cloud
10 30 (1.40%) Amazon EMR
11 25 (1.17%) Cloud Computing
12 17 (0.79%) SaaS
13 15 (0.70%) Amazon CloudWatch
14 12 (0.56%) AWS CloudFormation
15 11 (0.51%) IaaS
15 11 (0.51%) Serverless
16 10 (0.47%) Google Analytics
16 10 (0.47%) OpenShift
17 9 (0.42%) Amazon Kinesis
Communications & Networking
1 168 (7.83%) Kerberos
2 25 (1.17%) LDAP
3 23 (1.07%) VPN
4 21 (0.98%) HTTP
5 20 (0.93%) Firewall
5 20 (0.93%) Network Security
6 17 (0.79%) LAN
6 17 (0.79%) WAN
7 16 (0.75%) AMQP
8 14 (0.65%) Internet
8 14 (0.65%) SSL
9 12 (0.56%) SAN
10 11 (0.51%) DNS
10 11 (0.51%) DSL
10 11 (0.51%) NAS
11 9 (0.42%) DMZ
12 6 (0.28%) TCP/IP
13 5 (0.23%) Remote Desktop
13 5 (0.23%) tcpdump
13 5 (0.23%) Wireshark
Database & Business Intelligence
1 1515 (70.63%) Big Data
2 647 (30.16%) Apache Hive
3 525 (24.48%) HBase
4 345 (16.08%) NoSQL
5 263 (12.26%) MongoDB
6 228 (10.63%) Data Warehouse
7 224 (10.44%) PostgreSQL
8 210 (9.79%) Impala
9 198 (9.23%) Apache Cassandra
10 181 (8.44%) SQL Server
11 175 (8.16%) Sqoop
12 153 (7.13%) Relational Database
13 112 (5.22%) MySQL
14 79 (3.68%) Data Lake
15 70 (3.26%) RDBMS
16 67 (3.12%) Data Mining
17 50 (2.33%) BusinessObjects
18 46 (2.14%) QlikView
19 44 (2.05%) SQL Server Integration Services
20 42 (1.96%) Oracle Database
Development Applications
1 334 (15.57%) Jenkins
2 286 (13.33%) JIRA
3 270 (12.59%) Git (software)
4 128 (5.97%) Cucumber
5 67 (3.12%) Maven
6 64 (2.98%) IntelliJ
7 54 (2.52%) JUnit
8 45 (2.10%) Vagrant
9 44 (2.05%) TeamCity
10 40 (1.86%) GitLab
11 33 (1.54%) Artifactory
11 33 (1.54%) Eclipse
12 32 (1.49%) Gradle
12 32 (1.49%) Team Foundation Server
13 30 (1.40%) Selenium
14 28 (1.31%) Hudson
14 28 (1.31%) Talend Open Studio
15 26 (1.21%) Ant
15 26 (1.21%) Mockito
15 26 (1.21%) Subversion
General
1 466 (21.72%) Finance
2 322 (15.01%) Banking
3 123 (5.73%) Investment Banking
4 104 (4.85%) Telecoms
5 100 (4.66%) Marketing
6 73 (3.40%) Retail
7 64 (2.98%) Front Office
8 56 (2.61%) Electronics
9 54 (2.52%) Manufacturing
10 29 (1.35%) Retail Banking
11 21 (0.98%) Publishing
12 16 (0.75%) Games
12 16 (0.75%) Legal
13 11 (0.51%) Financial Institution
13 11 (0.51%) Pharmaceutical
14 8 (0.37%) Back Office
15 7 (0.33%) Law
16 6 (0.28%) Corporate Banking
17 3 (0.14%) Advertising
17 3 (0.14%) Private Banking
Job Titles
1 736 (34.31%) Developer
2 333 (15.52%) Data Engineer
3 307 (14.31%) Java Developer
4 279 (13.01%) Architect
5 274 (12.77%) Big Data Engineer
6 270 (12.59%) Big Data Developer
7 175 (8.16%) Data Architect
8 167 (7.79%) Hadoop Developer
9 161 (7.51%) Hadoop Engineer
10 147 (6.85%) Analyst
11 128 (5.97%) Scala Developer
12 106 (4.94%) Big Data Architect
13 105 (4.90%) Data Scientist
13 105 (4.90%) DevOps Engineer
14 93 (4.34%) Administrator
15 89 (4.15%) Senior Developer
16 86 (4.01%) Data Analyst
17 81 (3.78%) Apache Spark Developer
18 79 (3.68%) Java Engineer
19 76 (3.54%) Consultant
Libraries, Frameworks & Software Standards
1 574 (26.76%) Kafka
2 283 (13.19%) Spring
3 250 (11.66%) REST
4 170 (7.93%) Spring Boot
5 164 (7.65%) Web Services
6 140 (6.53%) JSON
7 128 (5.97%) RESTful
8 110 (5.13%) Apache Camel
9 94 (4.38%) AngularJS
10 90 (4.20%) Hibernate
11 82 (3.82%) .NET
12 81 (3.78%) Spark Streaming
13 79 (3.68%) XML
14 75 (3.50%) RabbitMQ
15 73 (3.40%) HTML
16 72 (3.36%) Drools
17 68 (3.17%) JMS
17 68 (3.17%) PySpark
18 64 (2.98%) Akka
19 56 (2.61%) HTML5
Miscellaneous
1 230 (10.72%) Analytical Skills
2 171 (7.97%) Algorithms
3 96 (4.48%) Computer Science
4 81 (3.78%) Greenfield Project
5 51 (2.38%) Market Risk
6 42 (1.96%) Data Centre
6 42 (1.96%) Data Structures
7 36 (1.68%) Management Information System
8 33 (1.54%) User Experience
9 29 (1.35%) Replication
10 27 (1.26%) Foreign Exchange (FX)
11 26 (1.21%) Fintech
12 24 (1.12%) Linux Command Line
12 24 (1.12%) Public Cloud
13 23 (1.07%) Virtual Team
14 19 (0.89%) Credit Risk
15 17 (0.79%) Equities
16 16 (0.75%) Derivative
17 14 (0.65%) Reference Data
18 13 (0.61%) Mobile App
Operating Systems
1 471 (21.96%) Linux
2 275 (12.82%) Unix
3 91 (4.24%) Red Hat Enterprise Linux
4 89 (4.15%) Windows
5 40 (1.86%) Ubuntu
6 39 (1.82%) CentOS
7 18 (0.84%) Apple iOS
8 16 (0.75%) Android
9 13 (0.61%) Debian
10 9 (0.42%) Windows Server 2012
11 8 (0.37%) Solaris
11 8 (0.37%) Windows Server
12 7 (0.33%) Windows Vista
13 6 (0.28%) CoreOS
14 5 (0.23%) AIX
14 5 (0.23%) VMS
15 4 (0.19%) Mac OS X
16 3 (0.14%) Windows Server 2003
17 2 (0.093%) Fedora
17 2 (0.093%) SUSE
Processes & Methodologies
1 881 (41.07%) Agile Software Development
2 469 (21.86%) TDD
3 461 (21.49%) Analytics
4 433 (20.19%) Scrum
5 371 (17.30%) DevOps
6 341 (15.90%) MapReduce
7 338 (15.76%) ETL
8 300 (13.99%) BDD
9 247 (11.52%) Open Source
10 245 (11.42%) Business Intelligence
11 243 (11.33%) Microservices
12 239 (11.14%) Continuous Integration
13 225 (10.49%) Data Modelling
14 223 (10.40%) Machine Learning
15 181 (8.44%) Data Architecture
16 159 (7.41%) Software Engineering
17 152 (7.09%) Visualisation
18 151 (7.04%) Data Analytics
19 144 (6.71%) Test Automation
20 143 (6.67%) Kanban
Programming Languages
1 1016 (47.37%) Java
2 666 (31.05%) Python
3 619 (28.86%) SQL
4 487 (22.70%) Scala
5 193 (9.00%) R
6 144 (6.71%) Shell Script
7 141 (6.57%) JavaScript
8 106 (4.94%) C#
9 105 (4.90%) Bash Shell
10 67 (3.12%) Ruby
11 42 (1.96%) Perl
12 38 (1.77%) C++
13 32 (1.49%) C
13 32 (1.49%) Groovy
14 31 (1.45%) PowerShell
15 29 (1.35%) Go
16 27 (1.26%) PL/SQL
17 24 (1.12%) MATLAB
18 20 (0.93%) T-SQL
19 16 (0.75%) PHP
Qualifications
1 303 (14.13%) Security Cleared
2 234 (10.91%) SC Cleared
3 129 (6.01%) Degree
4 49 (2.28%) DV Cleared
5 30 (1.40%) PhD
6 21 (0.98%) Computer Science Degree
6 21 (0.98%) Master's Degree
7 12 (0.56%) CISSP
8 11 (0.51%) Microsoft Certification
9 8 (0.37%) Numerate Degree
10 7 (0.33%) CESG Certified Professional
10 7 (0.33%) CISM
10 7 (0.33%) CLAS
10 7 (0.33%) DBS Check
10 7 (0.33%) MCITP
10 7 (0.33%) MCSE
11 6 (0.28%) BPSS Clearance
11 6 (0.28%) Cisco Certification
11 6 (0.28%) ISTQB
12 5 (0.23%) VMware Certified Professional (VCP)
Quality Assurance & Compliance
1 90 (4.20%) QA
2 79 (3.68%) Data Quality
3 22 (1.03%) GDPR
4 13 (0.61%) MiFID
5 10 (0.47%) GCP
6 8 (0.37%) Actionable Recommendations
7 7 (0.33%) PCI DSS
8 6 (0.28%) JSP 440
9 4 (0.19%) IFRS
9 4 (0.19%) IFRS 9
10 3 (0.14%) Dodd-Frank
10 3 (0.14%) FedRAMP
10 3 (0.14%) ISAE 3402
10 3 (0.14%) ISO/IEC 27001
10 3 (0.14%) SLA
10 3 (0.14%) SSAE 16
11 2 (0.093%) CDISC
11 2 (0.093%) Disclosure Scotland
11 2 (0.093%) EMIR
12 1 (0.047%) ISO 22301
System Software
1 397 (18.51%) HDFS
2 324 (15.10%) Docker
3 228 (10.63%) Apache Flume
4 117 (5.45%) Apache ZooKeeper
5 44 (2.05%) Active Directory
6 21 (0.98%) VMware Infrastructure
7 19 (0.89%) Virtual Machines
8 11 (0.51%) Hyper-V
8 11 (0.51%) vCloud
9 4 (0.19%) OpenAM
9 4 (0.19%) Squid
10 3 (0.14%) Oracle RAC
11 2 (0.093%) etcd
11 2 (0.093%) flannel
11 2 (0.093%) NFS
11 2 (0.093%) Veritas Volume Manager
11 2 (0.093%) VMware ESXi
12 1 (0.047%) Postfix
12 1 (0.047%) PowerHA
Systems Management
1 232 (10.82%) Ansible
2 172 (8.02%) Oozie
3 164 (7.65%) Kubernetes
4 163 (7.60%) Hadoop YARN
5 146 (6.81%) Puppet
6 135 (6.29%) Terraform
7 103 (4.80%) Opscode Chef
8 96 (4.48%) Ambari
9 45 (2.10%) Mesos
10 25 (1.17%) ZABBIX
11 22 (1.03%) Prometheus
12 21 (0.98%) Cobbler
13 20 (0.93%) Graphite
13 20 (0.93%) Kibana
13 20 (0.93%) Nagios
14 15 (0.70%) Control-M
15 12 (0.56%) ITRS Geneos
16 11 (0.51%) logstash
17 9 (0.42%) Salt
18 7 (0.33%) Microsoft Clustering
Vendors
1 339 (15.80%) Cloudera
2 258 (12.03%) Oracle
3 157 (7.32%) Hortonworks
3 157 (7.32%) Tableau
4 107 (4.99%) SAP
5 102 (4.76%) IBM
6 99 (4.62%) Google
7 93 (4.34%) Microsoft
8 91 (4.24%) Red Hat
9 77 (3.59%) Teradata
10 58 (2.70%) Informatica
11 53 (2.47%) Splunk
12 42 (1.96%) MapR
13 38 (1.77%) VMware
14 28 (1.31%) Talend
15 20 (0.93%) Atlassian
15 20 (0.93%) MicroStrategy
16 18 (0.84%) AppDynamics
17 17 (0.79%) HP
17 17 (0.79%) SaltStack