Period
to 21 January 2018

The following table provides summary statistics for contract job vacancies with a requirement for Hadoop skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Hadoop over the 6 months to 21 January 2018 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Hadoop
UK
6 months to
21 Jan 2018
Same period 2017 Same period 2016
Rank 122 104 200
Rank change year-on-year -18 +96 +95
Contract jobs citing Hadoop 2,080 2,158 1,363
As % of all contract IT jobs advertised in the UK 2.11% 2.25% 1.26%
As % of the Database & Business Intelligence category 9.58% 9.96% 5.48%
Number of daily rates quoted 1,471 1,390 915
UK median daily rate £540 £526 £550
Median daily rate % change year-on-year +2.76% -4.45% +4.76%
10th Percentile £400 £385 £363
90th Percentile £675 £688 £688
UK excluding London median daily rate £488 £500 £490
% change year-on-year -2.50% +2.04% +8.89%
Number of hourly rates quoted 9 7 3
UK median hourly rate £35.00 £46.00 £37.50
Median hourly rate % change year-on-year -23.91% +22.67% +2.74%
10th Percentile £16.64 £39.58 £34.09
90th Percentile £56.25 £72.05 £39.75
UK excluding London median hourly rate £32.75 £45.75 £36.78
% change year-on-year -28.42% +24.41% +0.75%

Hadoop is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Contract vacancies with a requirement for database or business intelligence skills 21,709 21,676 24,865
As % of all contract IT jobs advertised in the UK 21.99% 22.57% 23.03%
Number of daily rates quoted 15,106 14,450 16,593
UK median daily rate £450 £425 £415
Median daily rate % change year-on-year +5.88% +2.41% +3.75%
10th Percentile £310 £288 £275
90th Percentile £625 £600 £588
UK excluding London median daily rate £400 £388 £375
% change year-on-year +3.23% +3.33% +7.14%
Number of hourly rates quoted 250 313 428
UK median hourly rate £30.63 £28.00 £30.00
Median hourly rate % change year-on-year +9.38% -6.67% +9.09%
10th Percentile £14.54 £10.00 £12.00
90th Percentile £55.72 £48.30 £52.50
UK excluding London median hourly rate £35.00 £26.50 £29.00
% change year-on-year +32.08% -8.62% +9.43%

Hadoop
Job Vacancy Trend

Job postings citing Hadoop as a percentage of all IT jobs advertised.

Job vacancy trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Hadoop.

Contractor daily rate trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing Hadoop over the 6 months to 21 January 2018.

Contractor daily rate histogram for Hadoop in the UK

Hadoop
Contractor Hourly Rate Trend

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing Hadoop.

Contractor hourly rate trend for Hadoop in the UK

Hadoop
Contractor Hourly Rate Histogram

The hourly rate distribution of IT jobs citing Hadoop over the 6 months to 21 January 2018.

Contractor hourly rate histogram for Hadoop in the UK

Hadoop
Top 16 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Hadoop within the UK over the 6 months to 21 January 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -11 1,980 £550 +3.38% 487
London -23 1,320 £550 - 332
UK excluding London +49 738 £488 -2.50% 175
South East +24 369 £500 +5.26% 59
North of England +37 107 £488 -2.01% 48
South West +17 96 £475 -5.00% 12
North West +17 64 £550 +4.76% 28
Scotland +24 60 £400 -19.19% 14
East of England -8 49 £488 +39.29% 16
Yorkshire +2 48 £450 -4.26% 16
Midlands +1 37 £500 +11.11% 20
East Midlands +14 24 £500 +25.00% 12
Wales 0 13 £465 -9.27% 6
West Midlands -2 13 £494 -1.25% 8
North East +52 9 £610 +22.00% 4
Northern Ireland -17 7 - -

For the 6 months to 21 January 2018, IT contractor jobs citing Hadoop also mentioned the following skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all contract ads with a requirement for Hadoop.

1 1,436 (69.04%) Big Data
2 1,074 (51.63%) Apache Spark
3 946 (45.48%) Java
4 783 (37.64%) Agile Software Development
5 678 (32.60%) Python
6 615 (29.57%) SQL
7 605 (29.09%) Apache Hive
8 579 (27.84%) Kafka
9 521 (25.05%) HBase
10 505 (24.28%) Scala
11 488 (23.46%) Linux
12 458 (22.02%) Finance
13 449 (21.59%) Amazon AWS
14 448 (21.54%) Analytics
15 423 (20.34%) HDFS
16 417 (20.05%) TDD
17 381 (18.32%) Jenkins
18 363 (17.45%) NoSQL
19 348 (16.73%) DevOps
20 335 (16.11%) Docker
21 321 (15.43%) Scrum
22 320 (15.38%) ETL
23 314 (15.10%) Cloudera
24 297 (14.28%) Banking
25 286 (13.75%) MapReduce
25 286 (13.75%) Git (software)
26 281 (13.51%) Apache
27 278 (13.37%) Oracle
28 260 (12.50%) Ansible
29 259 (12.45%) Security Cleared

Hadoop
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 1,074 (51.63%) Apache Spark
2 281 (13.51%) Apache
3 144 (6.92%) Apache Pig
4 132 (6.35%) Confluence
5 127 (6.11%) Tomcat
6 125 (6.01%) Elasticsearch
7 95 (4.57%) SAS
8 77 (3.70%) JBoss
9 73 (3.51%) Apache Solr
10 51 (2.45%) Jetty
11 38 (1.83%) nginx
12 29 (1.39%) Oracle SOA Suite
13 28 (1.35%) WebLogic
14 23 (1.11%) Apache Airflow
14 23 (1.11%) Jupyter
15 18 (0.87%) WebSphere
16 17 (0.82%) OpenStack
17 16 (0.77%) IIS
18 15 (0.72%) GemFire
18 15 (0.72%) Oracle Coherence
Applications
1 20 (0.96%) SPSS
2 18 (0.87%) Microsoft Excel
3 10 (0.48%) Pipeline Pilot
4 5 (0.24%) Microsoft Office
5 3 (0.14%) Stata
6 2 (0.096%) HipChat
6 2 (0.096%) Microsoft PowerPoint
6 2 (0.096%) Microsoft Project
7 1 (0.048%) Adobe Illustrator
7 1 (0.048%) Gephi
7 1 (0.048%) GNU Octave
7 1 (0.048%) MS Visio
7 1 (0.048%) Photoshop
Business Applications
1 16 (0.77%) Oracle BPM
2 6 (0.29%) SAP Oil and Gas
3 4 (0.19%) Actimize
4 2 (0.096%) Dynamics CRM
4 2 (0.096%) SAP CRM
4 2 (0.096%) Summit
4 2 (0.096%) WebTrends
5 1 (0.048%) Oracle Financials
5 1 (0.048%) SAP HR
Cloud Services
1 449 (21.59%) Amazon AWS
2 148 (7.12%) Microsoft Azure
3 106 (5.10%) GitHub
4 68 (3.27%) Google Cloud Platform
5 59 (2.84%) Amazon S3
6 36 (1.73%) Bluemix
6 36 (1.73%) IBM Cloud
6 36 (1.73%) PaaS
7 35 (1.68%) Amazon EC2
8 33 (1.59%) Amazon EMR
9 26 (1.25%) AWS CloudFormation
10 25 (1.20%) Serverless
11 23 (1.11%) SaaS
12 21 (1.01%) AWS Lambda
13 19 (0.91%) Amazon Kinesis
13 19 (0.91%) IaaS
14 17 (0.82%) Google Tag Manager
14 17 (0.82%) Virtual Private Cloud
15 14 (0.67%) Amazon CloudWatch
15 14 (0.67%) Google Analytics
Communications & Networking
1 149 (7.16%) Kerberos
2 43 (2.07%) Network Security
3 29 (1.39%) Firewall
4 28 (1.35%) SSL
5 23 (1.11%) Internet
6 16 (0.77%) AMQP
7 13 (0.63%) HTTP
8 11 (0.53%) OpenLDAP
9 10 (0.48%) DNS
10 8 (0.38%) SAN
10 8 (0.38%) TCP/IP
11 7 (0.34%) VPN
12 6 (0.29%) DMZ
13 5 (0.24%) DSL
13 5 (0.24%) NAS
13 5 (0.24%) Remote Desktop
13 5 (0.24%) SSH
14 4 (0.19%) ICMP
15 3 (0.14%) WAN
16 2 (0.096%) LAN
Database & Business Intelligence
1 1,436 (69.04%) Big Data
2 605 (29.09%) Apache Hive
3 521 (25.05%) HBase
4 363 (17.45%) NoSQL
5 255 (12.26%) MongoDB
6 239 (11.49%) Data Warehouse
7 215 (10.34%) PostgreSQL
8 187 (8.99%) Sqoop
9 180 (8.65%) SQL Server
10 171 (8.22%) Impala
11 164 (7.88%) Apache Cassandra
12 154 (7.40%) Data Lake
12 154 (7.40%) Relational Database
13 95 (4.57%) MySQL
14 67 (3.22%) RDBMS
15 57 (2.74%) Amazon Redshift
16 48 (2.31%) Oracle Database
16 48 (2.31%) SQL Server Integration Services
17 44 (2.12%) Data Mining
18 43 (2.07%) Redis
Development Applications
1 381 (18.32%) Jenkins
2 286 (13.75%) Git (software)
3 213 (10.24%) JIRA
4 85 (4.09%) Cucumber
5 71 (3.41%) Maven
6 58 (2.79%) Gradle
7 52 (2.50%) IntelliJ
8 51 (2.45%) Vagrant
9 44 (2.12%) JUnit
9 44 (2.12%) TeamCity
10 41 (1.97%) Subversion
11 39 (1.88%) Selenium
12 37 (1.78%) Sonatype Nexus
13 34 (1.63%) Artifactory
14 31 (1.49%) GitLab
15 27 (1.30%) Team Foundation Server
16 23 (1.11%) ERwin
17 19 (0.91%) Eclipse
18 18 (0.87%) Ant
18 18 (0.87%) Bitbucket
General
1 458 (22.02%) Finance
2 297 (14.28%) Banking
3 95 (4.57%) Marketing
4 89 (4.28%) Retail
5 86 (4.13%) Telecoms
6 77 (3.70%) Investment Banking
7 46 (2.21%) Manufacturing
8 42 (2.02%) Electronics
9 39 (1.88%) Retail Banking
10 37 (1.78%) Front Office
11 31 (1.49%) Publishing
12 27 (1.30%) Games
13 14 (0.67%) Financial Institution
14 12 (0.58%) Legal
15 10 (0.48%) Automotive
15 10 (0.48%) Pharmaceutical
16 8 (0.38%) Advertising
17 6 (0.29%) French Language
17 6 (0.29%) Law
18 4 (0.19%) Spanish Language
Job Titles
1 672 (32.31%) Developer
2 310 (14.90%) Data Engineer
3 288 (13.85%) Java Developer
4 282 (13.56%) Architect
5 251 (12.07%) Big Data Developer
6 243 (11.68%) Big Data Engineer
7 188 (9.04%) Data Architect
8 153 (7.36%) Hadoop Engineer
9 132 (6.35%) Data Scientist
10 131 (6.30%) Hadoop Developer
11 128 (6.15%) DevOps Engineer
12 120 (5.77%) Analyst
13 115 (5.53%) Big Data Architect
14 105 (5.05%) Consultant
15 98 (4.71%) Scala Developer
16 87 (4.18%) Senior Developer
17 72 (3.46%) Administrator
18 71 (3.41%) Apache Spark Developer
19 67 (3.22%) AWS Engineer
20 60 (2.88%) Data Analyst
Libraries, Frameworks & Software Standards
1 579 (27.84%) Kafka
2 249 (11.97%) Spring
3 199 (9.57%) REST
4 178 (8.56%) Web Services
5 162 (7.79%) RESTful
6 149 (7.16%) JSON
7 119 (5.72%) Spring Boot
8 112 (5.38%) Hibernate
9 90 (4.33%) PySpark
10 85 (4.09%) AngularJS
11 84 (4.04%) XML
12 79 (3.80%) RabbitMQ
13 76 (3.65%) Spark Streaming
14 72 (3.46%) Apache Camel
15 71 (3.41%) Middleware
16 70 (3.37%) J2EE
17 67 (3.22%) JMS
18 65 (3.13%) ActiveMQ
19 60 (2.88%) HTML
20 59 (2.84%) Elastic Stack
Miscellaneous
1 209 (10.05%) Analytical Skills
2 123 (5.91%) Algorithms
2 123 (5.91%) Greenfield Project
3 120 (5.77%) Computer Science
4 50 (2.40%) Virtual Team
5 46 (2.21%) Data Centre
6 34 (1.63%) Management Information System
7 33 (1.59%) Data Structures
7 33 (1.59%) Linux Command Line
8 31 (1.49%) Fintech
8 31 (1.49%) Market Risk
8 31 (1.49%) Replication
9 27 (1.30%) User Experience
10 19 (0.91%) Foreign Exchange (FX)
10 19 (0.91%) Life Science
11 18 (0.87%) Derivative
12 15 (0.72%) Credit Risk
12 15 (0.72%) Equities
13 13 (0.63%) Digital Media
13 13 (0.63%) Public Cloud
Operating Systems
1 488 (23.46%) Linux
2 230 (11.06%) Unix
3 84 (4.04%) Red Hat Enterprise Linux
4 73 (3.51%) Windows
5 68 (3.27%) CentOS
6 35 (1.68%) Ubuntu
7 11 (0.53%) CoreOS
8 10 (0.48%) Windows Server
9 7 (0.34%) VMS
10 6 (0.29%) Android
10 6 (0.29%) Apple iOS
10 6 (0.29%) Windows Server 2012
11 3 (0.14%) AIX
11 3 (0.14%) Debian
11 3 (0.14%) Mac OS X
11 3 (0.14%) SUSE
11 3 (0.14%) Windows Server 2003
11 3 (0.14%) Windows Server 2008
11 3 (0.14%) Windows Vista
12 2 (0.096%) Mesosphere
Processes & Methodologies
1 783 (37.64%) Agile Software Development
2 448 (21.54%) Analytics
3 417 (20.05%) TDD
4 348 (16.73%) DevOps
5 321 (15.43%) Scrum
6 320 (15.38%) ETL
7 286 (13.75%) MapReduce
8 258 (12.40%) Business Intelligence
9 253 (12.16%) BDD
10 241 (11.59%) Continuous Integration
11 221 (10.63%) Open Source
12 216 (10.38%) Machine Learning
13 202 (9.71%) Data Modelling
14 192 (9.23%) Microservices
15 173 (8.32%) Visualisation
16 142 (6.83%) Test Automation
17 134 (6.44%) Software Engineering
18 132 (6.35%) Data Architecture
19 122 (5.87%) Continuous Delivery
20 121 (5.82%) Data Analytics
Programming Languages
1 946 (45.48%) Java
2 678 (32.60%) Python
3 615 (29.57%) SQL
4 505 (24.28%) Scala
5 168 (8.08%) R
6 143 (6.88%) JavaScript
7 122 (5.87%) Shell Script
8 108 (5.19%) Bash Shell
9 73 (3.51%) C#
10 71 (3.41%) Ruby
11 45 (2.16%) Groovy
12 38 (1.83%) C++
12 38 (1.83%) Perl
13 37 (1.78%) C
14 33 (1.59%) Go
15 30 (1.44%) PL/SQL
16 29 (1.39%) PowerShell
17 22 (1.06%) Gremlin
17 22 (1.06%) MATLAB
18 19 (0.91%) T-SQL
Qualifications
1 259 (12.45%) Security Cleared
2 197 (9.47%) SC Cleared
3 149 (7.16%) Degree
4 39 (1.88%) PhD
5 32 (1.54%) DV Cleared
6 27 (1.30%) Computer Science Degree
7 26 (1.25%) Master's Degree
8 18 (0.87%) Numerate Degree
9 8 (0.38%) Microsoft Certification
10 7 (0.34%) CISSP
11 6 (0.29%) CESG Certified Professional
11 6 (0.29%) CLAS
12 5 (0.24%) BPSS Clearance
12 5 (0.24%) CTC Cleared
13 3 (0.14%) BC Cleared
13 3 (0.14%) CCNA
13 3 (0.14%) CCSP
13 3 (0.14%) Cisco Certification
13 3 (0.14%) MCSE
13 3 (0.14%) SAS Certification
Quality Assurance & Compliance
1 90 (4.33%) QA
2 77 (3.70%) Data Quality
3 26 (1.25%) GDPR
4 10 (0.48%) GCP
4 10 (0.48%) MiFID
5 8 (0.38%) Disclosure Scotland
6 6 (0.29%) JSP 440
6 6 (0.29%) RMADS
7 5 (0.24%) Dodd-Frank
8 4 (0.19%) Actionable Recommendations
8 4 (0.19%) EMIR
9 3 (0.14%) IFRS
9 3 (0.14%) IFRS 9
9 3 (0.14%) SLA
10 2 (0.096%) COBIT
10 2 (0.096%) GAAP
10 2 (0.096%) ISO/IEC 20000
10 2 (0.096%) PMO
10 2 (0.096%) Sarbanes-Oxley
11 1 (0.048%) NIST 800
System Software
1 423 (20.34%) HDFS
2 335 (16.11%) Docker
3 241 (11.59%) Apache Flume
4 117 (5.63%) Apache ZooKeeper
5 32 (1.54%) Active Directory
6 23 (1.11%) VMware Infrastructure
7 13 (0.63%) vCloud
8 12 (0.58%) Virtual Machines
9 6 (0.29%) Squid
10 4 (0.19%) Hyper-V
10 4 (0.19%) NFS
11 3 (0.14%) Veritas Volume Manager
12 2 (0.096%) OpenDJ
12 2 (0.096%) OpenIDM
13 1 (0.048%) Ceph
13 1 (0.048%) etcd
13 1 (0.048%) flannel
13 1 (0.048%) Postfix
13 1 (0.048%) Varnish
13 1 (0.048%) VMware ESXi
Systems Management
1 260 (12.50%) Ansible
2 206 (9.90%) Oozie
3 181 (8.70%) Puppet
4 180 (8.65%) Hadoop YARN
5 147 (7.07%) Kubernetes
6 135 (6.49%) Opscode Chef
7 119 (5.72%) Ambari
8 114 (5.48%) Terraform
9 51 (2.45%) Mesos
10 42 (2.02%) Cobbler
11 37 (1.78%) Kibana
12 34 (1.63%) ZABBIX
13 27 (1.30%) Nagios
14 21 (1.01%) logstash
14 21 (1.01%) Salt
15 20 (0.96%) ITRS Geneos
16 12 (0.58%) Apache Slider
17 11 (0.53%) Prometheus
18 10 (0.48%) Control-M
19 9 (0.43%) Tivoli
Vendors
1 314 (15.10%) Cloudera
2 278 (13.37%) Oracle
3 221 (10.63%) Hortonworks
4 161 (7.74%) Tableau
5 114 (5.48%) Microsoft
6 113 (5.43%) Google
7 110 (5.29%) SAP
8 95 (4.57%) IBM
9 92 (4.42%) Red Hat
10 76 (3.65%) Teradata
11 72 (3.46%) Splunk
12 71 (3.41%) MapR
13 63 (3.03%) Informatica
14 39 (1.88%) VMware
15 36 (1.73%) Talend
16 34 (1.63%) Salesforce.com
17 29 (1.39%) AppDynamics
18 28 (1.35%) SaltStack
19 22 (1.06%) Ab Initio
19 22 (1.06%) MicroStrategy