Period
to 22 July 2018

The following table provides summary statistics for contract job vacancies with a requirement for Hadoop skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Hadoop over the 6 months to 22 July 2018 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Hadoop
UK
6 months to
22 Jul 2018
Same period 2017 Same period 2016
Rank 118 113 173
Rank change year-on-year -5 +60 +81
Contract jobs citing Hadoop 2,222 2,029 1,707
As % of all contract IT jobs advertised in the UK 2.15% 2.12% 1.55%
As % of the Database & Business Intelligence category 9.47% 9.42% 7.03%
Number of daily rates quoted 1,575 1,336 1,145
UK median daily rate £550 £513 £525
Median daily rate % change year-on-year +7.32% -2.38% -
10th Percentile £400 £369 £363
90th Percentile £688 £675 £713
UK excluding London median daily rate £500 £450 £475
% change year-on-year +11.11% -5.26% -
Number of hourly rates quoted 8 7 4
UK median hourly rate £47.50 £47.00 £38.00
Median hourly rate % change year-on-year +1.06% +23.68% +26.67%
UK excluding London median hourly rate £46.25 £47.00 £38.00
% change year-on-year -1.60% +23.68% +26.67%

Hadoop is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Contract vacancies with a requirement for database or business intelligence skills 23,455 21,539 24,282
As % of all contract IT jobs advertised in the UK 22.70% 22.45% 22.06%
Number of daily rates quoted 16,368 14,311 16,259
UK median daily rate £463 £450 £425
Median daily rate % change year-on-year +2.78% +5.88% +6.25%
10th Percentile £313 £300 £280
90th Percentile £638 £619 £600
UK excluding London median daily rate £413 £400 £375
% change year-on-year +3.13% +6.67% +3.45%
Number of hourly rates quoted 245 269 377
UK median hourly rate £32.50 £31.00 £28.50
Median hourly rate % change year-on-year +4.84% +8.77% +14.00%
10th Percentile £14.11 £11.17 £11.25
90th Percentile £55.00 £52.50 £51.00
UK excluding London median hourly rate £33.75 £32.25 £30.10
% change year-on-year +4.65% +7.13% +20.41%

Hadoop
Job Vacancy Trend

Job postings citing Hadoop as a percentage of all IT jobs advertised.

Job vacancy trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Hadoop.

Contractor daily rate trend for Hadoop in the UK

Hadoop
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing Hadoop over the 6 months to 22 July 2018.

Contractor daily rate histogram for Hadoop in the UK

Hadoop
Contractor Hourly Rate Trend

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing Hadoop.

Contractor hourly rate trend for Hadoop in the UK

Hadoop
Contractor Hourly Rate Histogram

The hourly rate distribution of IT jobs citing Hadoop over the 6 months to 22 July 2018.

Contractor hourly rate histogram for Hadoop in the UK

Hadoop
Top 15 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Hadoop within the UK over the 6 months to 22 July 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -9 2,081 £550 +4.76% 395
London -11 1,299 £550 +2.52% 217
UK excluding London +36 899 £500 +11.11% 214
South East +26 335 £513 +13.89% 56
North of England +50 209 £500 +2.56% 57
Yorkshire +40 118 £500 +5.26% 13
Scotland +17 105 £420 -6.67% 32
North West +20 88 £525 -4.55% 43
East of England -8 84 £475 +8.57% 24
South West +5 82 £460 +6.67% 27
Midlands +12 70 £400 -23.81% 14
East Midlands +46 55 £438 -16.67% 8
Wales -17 15 £550 +10.00%
West Midlands -48 14 £363 -26.85% 6
North East -10 4 £707 +24.77% 1

For the 6 months to 22 July 2018, IT contractor jobs citing Hadoop also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for Hadoop.

1 1,515 (68.18%) Big Data
2 1,120 (50.41%) Apache Spark
3 908 (40.86%) Java
4 848 (38.16%) Python
5 847 (38.12%) Agile Software Development
6 809 (36.41%) Kafka
7 716 (32.22%) Apache Hive
8 712 (32.04%) SQL
9 626 (28.17%) Analytics
10 593 (26.69%) Finance
11 550 (24.75%) Amazon AWS
12 536 (24.12%) Scala
13 473 (21.29%) HBase
13 473 (21.29%) Linux
14 439 (19.76%) DevOps
15 409 (18.41%) Data Science
16 407 (18.32%) NoSQL
17 388 (17.46%) ETL
18 375 (16.88%) Data Warehouse
19 347 (15.62%) Data Modelling
20 333 (14.99%) Jenkins
21 332 (14.94%) Cloudera
22 330 (14.85%) HDFS
23 318 (14.31%) Docker
24 306 (13.77%) Business Intelligence
25 301 (13.55%) TDD
26 299 (13.46%) Machine Learning
27 298 (13.41%) Git (software)
28 292 (13.14%) Scrum
29 283 (12.74%) Apache

Hadoop
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 1,120 (50.41%) Apache Spark
2 283 (12.74%) Apache
3 199 (8.96%) Elasticsearch
4 171 (7.70%) Apache Pig
5 140 (6.30%) SAS
6 90 (4.05%) Confluence
7 85 (3.83%) Apache Solr
8 76 (3.42%) Tomcat
9 55 (2.48%) JBoss
10 46 (2.07%) nginx
11 39 (1.76%) Cloud Foundry
12 34 (1.53%) Jetty
13 25 (1.13%) Oracle SOA Suite
13 25 (1.13%) WebLogic
14 15 (0.68%) GemFire
15 14 (0.63%) Apache Airflow
16 13 (0.59%) SharePoint
17 12 (0.54%) Jupyter
18 11 (0.50%) Luigi
18 11 (0.50%) PowerExchange
Applications
1 40 (1.80%) Microsoft Excel
2 20 (0.90%) MS Visio
3 12 (0.54%) Chatbot
4 7 (0.32%) HipChat
4 7 (0.32%) Microsoft Office
5 6 (0.27%) SPSS
6 5 (0.23%) Microsoft PowerPoint
7 3 (0.14%) Stata
8 2 (0.090%) Shockwave
Business Applications
1 15 (0.68%) SAP Oil and Gas
2 8 (0.36%) Dynamics CRM
3 7 (0.32%) ION MarketView
4 4 (0.18%) Dynamics AX
4 4 (0.18%) Oracle BPM
5 3 (0.14%) SAP ERP
6 2 (0.090%) Actimize
6 2 (0.090%) Summit
7 1 (0.045%) Distributed Ledger
7 1 (0.045%) Dynamics NAV
7 1 (0.045%) Finacle
7 1 (0.045%) FLEXCUBE
7 1 (0.045%) MoSes
7 1 (0.045%) Oracle Financials
7 1 (0.045%) Remedy ITSM
7 1 (0.045%) Revive Adserver
7 1 (0.045%) SAP S/4HANA
Cloud Services
1 550 (24.75%) Amazon AWS
2 283 (12.74%) Microsoft Azure
3 89 (4.01%) GitHub
3 89 (4.01%) Google Cloud Platform
4 82 (3.69%) Amazon EMR
4 82 (3.69%) Amazon S3
5 66 (2.97%) Amazon Kinesis
6 62 (2.79%) Amazon EC2
7 53 (2.39%) Azure Data Factory
8 52 (2.34%) Serverless
9 45 (2.03%) PaaS
10 35 (1.58%) AWS Lambda
11 27 (1.22%) Virtual Private Cloud
12 23 (1.04%) AWS CloudFormation
13 20 (0.90%) Google Analytics
14 19 (0.86%) SaaS
15 17 (0.77%) Cloud Computing
15 17 (0.77%) Data Lake Analytics
16 14 (0.63%) Azure Machine Learning
16 14 (0.63%) IaaS
Communications & Networking
1 77 (3.47%) Kerberos
2 38 (1.71%) Firewall
3 36 (1.62%) DNS
3 36 (1.62%) HTTP
3 36 (1.62%) SSL
4 35 (1.58%) VPN
5 27 (1.22%) TCP/IP
6 20 (0.90%) Internet
6 20 (0.90%) Network Security
7 15 (0.68%) FTP
7 15 (0.68%) SSH
8 11 (0.50%) SAN
9 8 (0.36%) AMQP
9 8 (0.36%) ICMP
9 8 (0.36%) SMS
9 8 (0.36%) WAN
10 7 (0.32%) NAS
10 7 (0.32%) RTMP
11 6 (0.27%) DSL
11 6 (0.27%) SNA
Database & Business Intelligence
1 1,515 (68.18%) Big Data
2 716 (32.22%) Apache Hive
3 473 (21.29%) HBase
4 407 (18.32%) NoSQL
5 375 (16.88%) Data Warehouse
6 273 (12.29%) MongoDB
7 269 (12.11%) SQL Server
8 236 (10.62%) Relational Database
9 218 (9.81%) Data Lake
10 215 (9.68%) Apache Cassandra
11 203 (9.14%) PostgreSQL
12 200 (9.00%) Impala
13 143 (6.44%) MySQL
14 139 (6.26%) Sqoop
15 133 (5.99%) Power BI
16 118 (5.31%) Amazon Redshift
17 101 (4.55%) RDBMS
18 79 (3.56%) OLAP
19 77 (3.47%) Enterprise Data Warehouse
19 77 (3.47%) SQL Server Integration Services
Development Applications
1 333 (14.99%) Jenkins
2 298 (13.41%) Git (software)
3 139 (6.26%) JIRA
4 71 (3.20%) Maven
5 60 (2.70%) Cucumber
6 50 (2.25%) GitLab
7 43 (1.94%) Visual Studio Team System
8 39 (1.76%) Selenium
9 38 (1.71%) JUnit
10 35 (1.58%) Vagrant
11 34 (1.53%) Gradle
11 34 (1.53%) Subversion
12 33 (1.49%) Artifactory
13 30 (1.35%) Team Foundation Server
14 28 (1.26%) IntelliJ
15 27 (1.22%) Sonatype Nexus
16 24 (1.08%) Atlassian Bamboo
17 23 (1.04%) ERwin
18 21 (0.95%) TeamCity
19 20 (0.90%) Eclipse
General
1 593 (26.69%) Finance
2 245 (11.03%) Banking
3 106 (4.77%) Retail
4 54 (2.43%) Investment Banking
5 45 (2.03%) Marketing
6 44 (1.98%) Games
7 32 (1.44%) Front Office
8 25 (1.13%) Telecoms
9 17 (0.77%) Retail Banking
10 14 (0.63%) Advertising
11 13 (0.59%) Corporate Banking
11 13 (0.59%) Legal
11 13 (0.59%) Manufacturing
12 9 (0.41%) Private Banking
13 8 (0.36%) Publishing
14 7 (0.32%) Electronics
15 6 (0.27%) Automotive
16 5 (0.23%) Back Office
16 5 (0.23%) Financial Institution
17 4 (0.18%) Pharmaceutical
Job Titles
1 719 (32.36%) Developer
2 442 (19.89%) Data Engineer
3 294 (13.23%) Big Data Engineer
4 280 (12.60%) Java Developer
5 269 (12.11%) Architect
6 257 (11.57%) Big Data Developer
7 183 (8.24%) Hadoop Engineer
8 174 (7.83%) Data Architect
9 136 (6.12%) Analyst
10 132 (5.94%) Hadoop Developer
11 130 (5.85%) Consultant
12 111 (5.00%) Big Data Architect
13 110 (4.95%) DevOps Engineer
14 108 (4.86%) Data Scientist
15 82 (3.69%) AWS Engineer
16 80 (3.60%) Senior Developer
17 72 (3.24%) Data Analyst
18 69 (3.11%) Apache Spark Developer
19 67 (3.02%) Scala Developer
20 66 (2.97%) Python Engineer
Libraries, Frameworks & Software Standards
1 809 (36.41%) Kafka
2 209 (9.41%) Spring
3 188 (8.46%) REST
4 160 (7.20%) JSON
5 125 (5.63%) Web Services
6 121 (5.45%) RabbitMQ
7 119 (5.36%) RESTful
8 106 (4.77%) Spark Streaming
9 104 (4.68%) HTML
10 103 (4.64%) AngularJS
11 96 (4.32%) Hibernate
12 94 (4.23%) Middleware
13 92 (4.14%) Spring Boot
14 78 (3.51%) J2EE
15 75 (3.38%) XML
16 71 (3.20%) Node.js
17 59 (2.66%) .NET
18 57 (2.57%) HTML5
19 55 (2.48%) React
20 53 (2.39%) Natural Language Processing
Miscellaneous
1 232 (10.44%) Analytical Skills
2 164 (7.38%) Algorithms
3 138 (6.21%) Greenfield Project
4 104 (4.68%) Computer Science
5 71 (3.20%) Data Structures
6 52 (2.34%) Self-Motivation
7 43 (1.94%) Fintech
7 43 (1.94%) Linux Command Line
8 39 (1.76%) Enterprise Software
9 37 (1.67%) Virtual Team
10 36 (1.62%) User Experience
11 33 (1.49%) Public Cloud
12 30 (1.35%) Management Information System
13 29 (1.31%) Data Centre
14 25 (1.13%) Derivative
15 23 (1.04%) HoloLens
16 19 (0.86%) Wiki
17 18 (0.81%) Credit Risk
18 17 (0.77%) Cyberthreat
19 16 (0.72%) Mobile App
Operating Systems
1 473 (21.29%) Linux
2 248 (11.16%) Unix
3 83 (3.74%) Windows
4 57 (2.57%) Red Hat Enterprise Linux
5 45 (2.03%) CentOS
6 23 (1.04%) Ubuntu
7 19 (0.86%) CoreOS
8 9 (0.41%) Android
8 9 (0.41%) Debian
8 9 (0.41%) Solaris
9 8 (0.36%) Apple iOS
9 8 (0.36%) Windows Server
10 6 (0.27%) AIX
10 6 (0.27%) DC/OS
11 5 (0.23%) Windows Server 2008
12 3 (0.14%) Mac OS X
12 3 (0.14%) SUSE Linux Enterprise Server
13 1 (0.045%) Mesosphere
13 1 (0.045%) SUSE
13 1 (0.045%) VMS
Processes & Methodologies
1 847 (38.12%) Agile Software Development
2 626 (28.17%) Analytics
3 439 (19.76%) DevOps
4 409 (18.41%) Data Science
5 388 (17.46%) ETL
6 347 (15.62%) Data Modelling
7 306 (13.77%) Business Intelligence
8 301 (13.55%) TDD
9 299 (13.46%) Machine Learning
10 292 (13.14%) Scrum
11 246 (11.07%) Continuous Integration
12 224 (10.08%) Visualisation
13 217 (9.77%) Open Source
14 212 (9.54%) Microservices
15 170 (7.65%) BDD
16 168 (7.56%) Problem-Solving
17 165 (7.43%) MapReduce
18 158 (7.11%) Test Automation
19 154 (6.93%) Data Analytics
20 153 (6.89%) Software Engineering
Programming Languages
1 908 (40.86%) Java
2 848 (38.16%) Python
3 712 (32.04%) SQL
4 536 (24.12%) Scala
5 196 (8.82%) R
6 176 (7.92%) JavaScript
7 167 (7.52%) Bash Shell
8 155 (6.98%) Java 8
9 122 (5.49%) Shell Script
10 74 (3.33%) C++
11 69 (3.11%) Ruby
12 66 (2.97%) Groovy
13 58 (2.61%) Perl
14 56 (2.52%) PowerShell
15 53 (2.39%) C#
15 53 (2.39%) MATLAB
16 44 (1.98%) T-SQL
17 38 (1.71%) C
18 28 (1.26%) PL/SQL
19 24 (1.08%) Go
Qualifications
1 228 (10.26%) Security Cleared
2 146 (6.57%) SC Cleared
3 122 (5.49%) Degree
4 41 (1.85%) Computer Science Degree
5 36 (1.62%) PhD
6 32 (1.44%) DV Cleared
7 19 (0.86%) BC Cleared
8 17 (0.77%) ISTQB
9 16 (0.72%) BPSS Clearance
10 13 (0.59%) Master's Degree
10 13 (0.59%) Numerate Degree
11 11 (0.50%) CISSP
12 9 (0.41%) CISM
13 8 (0.36%) CHECK Team Member
13 8 (0.36%) Microsoft Certification
14 4 (0.18%) AWS Certification
14 4 (0.18%) CEH
14 4 (0.18%) SANS
15 3 (0.14%) ISEB
16 2 (0.090%) CCA
Quality Assurance & Compliance
1 160 (7.20%) Data Quality
2 92 (4.14%) QA
3 44 (1.98%) GDPR
4 31 (1.40%) GCP
5 11 (0.50%) PMO
5 11 (0.50%) SLA
6 5 (0.23%) FINRA
7 3 (0.14%) Disclosure Scotland
7 3 (0.14%) Sarbanes-Oxley
8 2 (0.090%) EMIR
8 2 (0.090%) ISO/IEC 20000
8 2 (0.090%) ISO/IEC 27001
8 2 (0.090%) MiFID
8 2 (0.090%) NIST 800
9 1 (0.045%) Basel III
9 1 (0.045%) Dodd-Frank
9 1 (0.045%) HL7
9 1 (0.045%) Solvency II
System Software
1 330 (14.85%) HDFS
2 318 (14.31%) Docker
3 169 (7.61%) Apache Flume
4 131 (5.90%) Apache ZooKeeper
5 30 (1.35%) Active Directory
6 18 (0.81%) VMware Infrastructure
7 8 (0.36%) Virtual Machines
8 5 (0.23%) Xen
9 4 (0.18%) Ceph
9 4 (0.18%) KVM
9 4 (0.18%) Lustre
9 4 (0.18%) NFS
10 2 (0.090%) Hyper-V
10 2 (0.090%) Oracle RAC
10 2 (0.090%) SRDF
10 2 (0.090%) vCloud
10 2 (0.090%) VMware ESXi
11 1 (0.045%) Postfix
11 1 (0.045%) Varnish
11 1 (0.045%) Veritas Cluster Server
Systems Management
1 264 (11.88%) Ansible
2 214 (9.63%) Oozie
3 184 (8.28%) Hadoop YARN
3 184 (8.28%) Puppet
4 159 (7.16%) Kubernetes
5 157 (7.07%) Terraform
6 132 (5.94%) Opscode Chef
7 44 (1.98%) Kibana
8 42 (1.89%) Ambari
9 29 (1.31%) Cobbler
10 24 (1.08%) logstash
11 23 (1.04%) Prometheus
12 21 (0.95%) Mesos
13 19 (0.86%) Salt
14 18 (0.81%) Consul
14 18 (0.81%) HAProxy
14 18 (0.81%) Host Intrusion Detection System
14 18 (0.81%) Network Intrusion Detection System
15 15 (0.68%) Sysdig
16 13 (0.59%) Docker Swarm
Vendors
1 332 (14.94%) Cloudera
2 270 (12.15%) Oracle
3 199 (8.96%) Hortonworks
4 176 (7.92%) Microsoft
5 168 (7.56%) Tableau
6 122 (5.49%) Google
7 116 (5.22%) Informatica
8 87 (3.92%) Talend
9 85 (3.83%) Teradata
10 67 (3.02%) IBM
10 67 (3.02%) MapR
11 66 (2.97%) SAP
12 65 (2.93%) Splunk
13 59 (2.66%) Alteryx
14 46 (2.07%) Ab Initio
15 30 (1.35%) Salesforce.com
16 27 (1.22%) CA
17 23 (1.04%) Atlassian
18 22 (0.99%) Sybase
18 22 (0.99%) VMware