Period
to 22 September 2019

The following table provides summary statistics for permanent job vacancies with a requirement for Hadoop skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Hadoop over the 6 months to 22 September 2019 with a comparison to the same period in the previous 2 years.

Apache Hadoop
UK
6 months to
22 Sep 2019
Same period 2018 Same period 2017
Rank 218 191 205
Rank change year-on-year -27 +14 +8
Permanent jobs citing Hadoop 2,134 3,071 3,151
As % of all permanent IT jobs advertised in the UK 1.53% 1.84% 1.78%
As % of the Database & Business Intelligence category 5.67% 6.21% 5.83%
Number of salaries quoted 1,577 2,301 2,341
UK median annual salary £70,000 £67,500 £60,000
Median salary % change year-on-year +3.70% +12.50% -
10th Percentile £43,750 £38,750 £36,465
90th Percentile £105,000 £97,500 £98,000
UK excluding London median annual salary £57,275 £57,500 £51,000
% change year-on-year -0.39% +12.75% -0.97%

Hadoop is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Permanent vacancies with a requirement for database or business intelligence skills 37,647 49,485 54,024
As % of all permanent IT jobs advertised in the UK 27.06% 29.72% 30.57%
Number of salaries quoted 30,738 40,368 44,486
UK median annual salary £52,500 £50,000 £47,500
Median salary % change year-on-year +5.00% +5.26% -
10th Percentile £31,250 £31,250 £30,000
90th Percentile £85,000 £82,500 £77,500
UK excluding London median annual salary £45,000 £45,000 £42,500
% change year-on-year - +5.88% +2.41%

Hadoop
Job Vacancy Trend

Job postings citing Hadoop as a percentage of all IT jobs advertised.

Job vacancy trend for Hadoop in the UK

Hadoop
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Hadoop.

Salary trend for Hadoop in the UK

Hadoop
Salary Histogram

The salary distribution of IT jobs citing Hadoop over the 6 months to 22 September 2019.

Salary histogram for Hadoop in the UK

Hadoop
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Hadoop within the UK over the 6 months to 22 September 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -24 2,034 £70,000 - 262
London -25 1,248 £80,000 +6.67% 158
UK excluding London -38 848 £57,275 -0.39% 129
South West +43 236 £53,700 +13.05% 20
South East -69 219 £55,750 -7.08% 29
Midlands +31 139 £57,500 -4.17% 9
North of England -77 135 £54,500 -0.91% 32
East Midlands +28 86 £57,500 -17.86% 2
Scotland -36 75 £52,500 +5.00% 22
Yorkshire +9 66 £52,500 +0.96% 11
North West -81 55 £60,000 +9.09% 18
West Midlands +29 53 £57,500 -4.17% 7
East of England -38 41 £70,000 +6.06% 14
North East +31 14 £45,412 +15.70% 3
Northern Ireland -3 3 £80,000 - 1
Wales -1 1 £39,000 +4.00% 2
Channel Islands - 1 £37,500 -

For the 6 months to 22 September 2019, IT jobs citing Hadoop also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Hadoop.

1 1,341 (62.84%) Big Data
2 1,265 (59.28%) Apache Spark
3 1,229 (57.59%) Python
4 1,042 (48.83%) Agile Software Development
5 966 (45.27%) Java
6 941 (44.10%) SQL
7 891 (41.75%) Analytics
8 744 (34.86%) AWS
9 723 (33.88%) Scala
10 683 (32.01%) Data Science
11 669 (31.35%) Apache Hive
12 540 (25.30%) Azure
13 530 (24.84%) Machine Learning
14 481 (22.54%) NoSQL
14 481 (22.54%) Kafka
15 454 (21.27%) ETL
16 434 (20.34%) Data Warehouse
17 419 (19.63%) Business Intelligence
18 409 (19.17%) Analytical Skills
19 407 (19.07%) DevOps
20 405 (18.98%) Software Engineering
21 365 (17.10%) Finance
22 362 (16.96%) Data Modelling
23 356 (16.68%) R
24 352 (16.49%) Linux
25 324 (15.18%) Docker
26 320 (15.00%) Visualisation
26 320 (15.00%) Degree
27 319 (14.95%) Data Engineering
28 318 (14.90%) GCP

Hadoop
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 1,265 (59.28%) Apache Spark
2 162 (7.59%) Elasticsearch
3 160 (7.50%) Apache
4 138 (6.47%) Apache Pig
5 113 (5.30%) SAS
6 56 (2.62%) Apache Solr
7 35 (1.64%) Apache Airflow
8 24 (1.12%) Hyperledger
8 24 (1.12%) nginx
9 23 (1.08%) Confluence
9 23 (1.08%) JBoss
10 21 (0.98%) SharePoint
11 18 (0.84%) Jupyter
12 15 (0.70%) WebSphere
13 13 (0.61%) IIS
14 10 (0.47%) MQSeries
14 10 (0.47%) Tomcat
14 10 (0.47%) WebSphere Application Server
15 9 (0.42%) IPython
16 8 (0.37%) Luigi
Applications
1 11 (0.52%) Microsoft Office
2 10 (0.47%) SPSS
3 5 (0.23%) Microsoft Excel
3 5 (0.23%) Spreadsheet
4 4 (0.19%) GNU Octave
4 4 (0.19%) Microsoft PowerPoint
5 3 (0.14%) Chatbot
6 2 (0.094%) Microsoft Project
7 1 (0.047%) CPLEX
7 1 (0.047%) MS Visio
7 1 (0.047%) SPSS Modeler
7 1 (0.047%) Stata
7 1 (0.047%) Weka
Business Applications
1 3 (0.14%) Dynamics CRM
2 2 (0.094%) Adobe Campaign
2 2 (0.094%) Magento
3 1 (0.047%) Data Workbench
3 1 (0.047%) FINCAD Analytics
3 1 (0.047%) SAP ERP
3 1 (0.047%) SAP InfiniteInsight
3 1 (0.047%) WebTrends
Cloud Services
1 744 (34.86%) AWS
2 540 (25.30%) Azure
3 318 (14.90%) GCP
4 146 (6.84%) Power Platform
5 104 (4.87%) Amazon EMR
6 98 (4.59%) PaaS
7 79 (3.70%) Amazon S3
8 68 (3.19%) Azure Data Factory
9 67 (3.14%) SaaS
10 66 (3.09%) GitHub
10 66 (3.09%) Serverless
11 46 (2.16%) Amazon EC2
11 46 (2.16%) Amazon Kinesis
11 46 (2.16%) Cloud Computing
12 26 (1.22%) Amazon ECS
13 24 (1.12%) IaaS
14 23 (1.08%) AWS Lambda
15 22 (1.03%) Office 365
16 18 (0.84%) Cortana Intelligence Suite
17 17 (0.80%) AWS CloudFormation
Communications & Networking
1 55 (2.58%) Internet
2 20 (0.94%) Firewall
3 19 (0.89%) HTTP
4 17 (0.80%) TCP/IP
5 14 (0.66%) Broadband
5 14 (0.66%) Kerberos
6 7 (0.33%) SSL
7 6 (0.28%) Network Security
7 6 (0.28%) VLAN
7 6 (0.28%) VPN
8 5 (0.23%) SSH
9 4 (0.19%) LAN
9 4 (0.19%) Software-Defined Networking
9 4 (0.19%) WAN
10 3 (0.14%) DNS
10 3 (0.14%) HTTPS
11 2 (0.094%) SMS
12 1 (0.047%) Unified Communications
Database & Business Intelligence
1 1,341 (62.84%) Big Data
2 669 (31.35%) Apache Hive
3 481 (22.54%) NoSQL
4 434 (20.34%) Data Warehouse
5 304 (14.25%) HBase
6 268 (12.56%) MongoDB
7 256 (12.00%) Apache Cassandra
8 226 (10.59%) Impala
9 191 (8.95%) SQL Server
10 189 (8.86%) Data Lake
11 182 (8.53%) Amazon Redshift
12 156 (7.31%) PostgreSQL
13 146 (6.84%) Power BI
14 133 (6.23%) Relational Database
15 130 (6.09%) Sqoop
16 120 (5.62%) MySQL
17 89 (4.17%) BigQuery
17 89 (4.17%) SQL Server Integration Services
18 83 (3.89%) Data Mining
18 83 (3.89%) Neo4j
Development Applications
1 265 (12.42%) Jenkins
2 234 (10.97%) Git (software)
3 111 (5.20%) JIRA
4 107 (5.01%) Maven
5 87 (4.08%) Bitbucket
6 62 (2.91%) Selenium
7 50 (2.34%) Artifactory
7 50 (2.34%) Atlassian Bamboo
8 48 (2.25%) Cucumber
9 26 (1.22%) Visual Studio
10 25 (1.17%) Subversion
11 24 (1.12%) ERwin
11 24 (1.12%) GitLab
12 23 (1.08%) Gradle
12 23 (1.08%) Team Foundation Server
13 21 (0.98%) Vagrant
14 20 (0.94%) Ant
14 20 (0.94%) TeamCity
15 17 (0.80%) Eclipse
16 16 (0.75%) SonarQube
General
1 365 (17.10%) Finance
2 285 (13.36%) Banking
3 223 (10.45%) Retail
4 98 (4.59%) Law
5 82 (3.84%) Marketing
6 71 (3.33%) Public Sector
7 69 (3.23%) Telecoms
8 49 (2.30%) Games
9 46 (2.16%) Manufacturing
10 37 (1.73%) Advertising
11 30 (1.41%) Investment Banking
12 27 (1.27%) Electronics
13 25 (1.17%) Front Office
14 16 (0.75%) Automotive
15 11 (0.52%) Financial Institution
16 9 (0.42%) Billing
16 9 (0.42%) Military
17 8 (0.37%) Legal
17 8 (0.37%) Retail Banking
18 7 (0.33%) Aerospace
Job Titles
1 599 (28.07%) Data Engineer
2 456 (21.37%) Developer
3 219 (10.26%) Java Developer
4 204 (9.56%) Big Data Engineer
5 199 (9.33%) Architect
6 179 (8.39%) Data Scientist
7 152 (7.12%) Software Engineer
8 150 (7.03%) Senior Data Engineer
9 134 (6.28%) Data Architect
10 106 (4.97%) Senior Developer
11 102 (4.78%) Hadoop Engineer
12 93 (4.36%) Big Data Developer
13 90 (4.22%) Python Engineer
14 77 (3.61%) DevOps Engineer
15 70 (3.28%) Java Engineer
16 64 (3.00%) Senior Java Developer
17 58 (2.72%) Senior Data Scientist
18 56 (2.62%) Consultant
18 56 (2.62%) Solutions Architect
19 55 (2.58%) Analyst
Libraries, Frameworks & Software Standards
1 481 (22.54%) Kafka
2 136 (6.37%) Spring
3 115 (5.39%) Web Services
4 93 (4.36%) Pandas
4 93 (4.36%) scikit-learn
5 88 (4.12%) Apache NiFi
6 80 (3.75%) Natural Language Processing
6 80 (3.75%) Node.js
6 80 (3.75%) TensorFlow
7 75 (3.51%) REST
8 72 (3.37%) HTML
9 70 (3.28%) AngularJS
10 66 (3.09%) RESTful
10 66 (3.09%) Spark Streaming
11 56 (2.62%) React
12 52 (2.44%) Apache Beam
13 50 (2.34%) NumPy
13 50 (2.34%) Spring Boot
14 48 (2.25%) JSON
15 46 (2.16%) Hibernate
Miscellaneous
1 409 (19.17%) Analytical Skills
2 214 (10.03%) Algorithms
3 115 (5.39%) Data Structures
4 60 (2.81%) Greenfield Project
5 55 (2.58%) Public Cloud
6 52 (2.44%) Linux Command Line
7 44 (2.06%) Management Information System
8 38 (1.78%) User Experience
9 36 (1.69%) Private Cloud
10 32 (1.50%) Social Media
11 30 (1.41%) Virtual Team
12 25 (1.17%) Cloud Native
13 24 (1.12%) Self-Motivation
14 20 (0.94%) Blog
15 19 (0.89%) Mobile App
16 17 (0.80%) Enterprise Software
16 17 (0.80%) Mainframe
16 17 (0.80%) Video on Demand
17 16 (0.75%) Market Risk
18 14 (0.66%) Data Centre
Operating Systems
1 352 (16.49%) Linux
2 246 (11.53%) Unix
3 47 (2.20%) Windows
4 24 (1.12%) Red Hat Enterprise Linux
5 16 (0.75%) CentOS
6 13 (0.61%) Ubuntu
7 10 (0.47%) zOS
8 8 (0.37%) Windows Server
9 5 (0.23%) Debian
10 4 (0.19%) Android
11 3 (0.14%) Apple iOS
12 2 (0.094%) AIX
12 2 (0.094%) CoreOS
12 2 (0.094%) Mac OS X
12 2 (0.094%) SUSE Linux Enterprise Server
12 2 (0.094%) VME
13 1 (0.047%) HPUX
13 1 (0.047%) Oracle Linux
Processes & Methodologies
1 1,042 (48.83%) Agile Software Development
2 891 (41.75%) Analytics
3 683 (32.01%) Data Science
4 530 (24.84%) Machine Learning
5 454 (21.27%) ETL
6 419 (19.63%) Business Intelligence
7 407 (19.07%) DevOps
8 405 (18.98%) Software Engineering
9 362 (16.96%) Data Modelling
10 320 (15.00%) Visualisation
11 319 (14.95%) Data Engineering
12 288 (13.50%) Data Analytics
13 241 (11.29%) Artificial Intelligence
14 236 (11.06%) Data Analysis
15 219 (10.26%) Scrum
16 216 (10.12%) Open Source
17 205 (9.61%) Mathematics
18 201 (9.42%) MapReduce
19 185 (8.67%) TDD
20 169 (7.92%) Microservices
Programming Languages
1 1,229 (57.59%) Python
2 966 (45.27%) Java
3 941 (44.10%) SQL
4 723 (33.88%) Scala
5 356 (16.68%) R
6 174 (8.15%) JavaScript
7 155 (7.26%) C#
8 153 (7.17%) C
9 93 (4.36%) PL/SQL
10 92 (4.31%) C++
11 73 (3.42%) Java 8
12 61 (2.86%) Bash Shell
13 55 (2.58%) Shell Script
14 48 (2.25%) Ruby
15 37 (1.73%) HiveQL
15 37 (1.73%) Perl
16 33 (1.55%) Go
17 32 (1.50%) PHP
18 27 (1.27%) TypeScript
19 22 (1.03%) PowerShell
Qualifications
1 320 (15.00%) Degree
2 126 (5.90%) Security Cleared
3 88 (4.12%) PhD
4 51 (2.39%) SC Cleared
5 48 (2.25%) Computer Science Degree
6 44 (2.06%) Master's Degree
7 42 (1.97%) DV Cleared
8 31 (1.45%) Numerate Degree
9 9 (0.42%) BPSS Clearance
10 8 (0.37%) AWS Certification
11 6 (0.28%) ISTQB
11 6 (0.28%) ISTQB Foundation Certification
12 5 (0.23%) Microsoft Certification
13 2 (0.094%) MBA
13 2 (0.094%) MCSA
14 1 (0.047%) APMP
14 1 (0.047%) ITIL Certification
14 1 (0.047%) SAS Certification
14 1 (0.047%) Six Sigma Black Belt
14 1 (0.047%) Sun Certification
Quality Assurance & Compliance
1 106 (4.97%) Data Quality
2 90 (4.22%) QA
3 14 (0.66%) ISO/IEC 27001
4 13 (0.61%) GDPR
5 2 (0.094%) Basel III
5 2 (0.094%) COREP
5 2 (0.094%) CRD IV
6 1 (0.047%) GxP
6 1 (0.047%) IFRS
6 1 (0.047%) IFRS 9
6 1 (0.047%) MiFID
6 1 (0.047%) PMO
System Software
1 324 (15.18%) Docker
2 191 (8.95%) HDFS
3 95 (4.45%) Apache ZooKeeper
4 65 (3.05%) Apache Flume
5 20 (0.94%) VirtualBox
6 10 (0.47%) Cronacle
7 4 (0.19%) Varnish
7 4 (0.19%) VMware Infrastructure
7 4 (0.19%) vSphere
8 2 (0.094%) Active Directory
8 2 (0.094%) Tivoli Workload Scheduler
8 2 (0.094%) Virtual Machines
8 2 (0.094%) Virtual Servers
9 1 (0.047%) Hyper-V
Systems Management
1 191 (8.95%) Ansible
2 188 (8.81%) Kubernetes
3 160 (7.50%) Terraform
4 139 (6.51%) Hadoop YARN
5 116 (5.44%) Oozie
6 92 (4.31%) Puppet
7 58 (2.72%) Opscode Chef
8 38 (1.78%) Docker Swarm
9 27 (1.27%) Ambari
10 19 (0.89%) logstash
11 18 (0.84%) Kibana
11 18 (0.84%) Mesos
12 16 (0.75%) Consul
13 10 (0.47%) Control-M
13 10 (0.47%) Nagios
13 10 (0.47%) Salt
14 9 (0.42%) Prometheus
15 8 (0.37%) Packer
16 7 (0.33%) Marathon
16 7 (0.33%) Single Sign-On
Vendors
1 268 (12.56%) Oracle
2 257 (12.04%) Cloudera
3 244 (11.43%) Google
4 237 (11.11%) Tableau
5 182 (8.53%) Microsoft
6 136 (6.37%) Teradata
7 135 (6.33%) Hortonworks
8 126 (5.90%) Informatica
9 105 (4.92%) Databricks
10 75 (3.51%) Talend
11 67 (3.14%) IBM
12 45 (2.11%) Splunk
13 41 (1.92%) Netezza
14 35 (1.64%) Red Hat
14 35 (1.64%) SAP
15 32 (1.50%) Ab Initio
16 31 (1.45%) MapR
17 24 (1.12%) Atlassian
18 20 (0.94%) Dataiku
19 16 (0.75%) Vertica