Period
to 27 October 2021

The following table provides summary statistics for permanent job vacancies with a requirement for Hadoop skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Hadoop over the 6 months to 27 October 2021 with a comparison to the same period in the previous 2 years.

Apache Hadoop
UK
6 months to
27 Oct 2021
Same period 2020 Same period 2019
Rank 311 249 234
Rank change year-on-year -62 -15 -30
Permanent jobs citing Hadoop 1,429 664 1,994
As % of all permanent jobs advertised in the UK 1.13% 1.34% 1.47%
As % of the Database & Business Intelligence category 4.04% 4.68% 5.38%
Number of salaries quoted 833 480 1,504
10th Percentile £46,250 £42,500 £43,750
25th Percentile £54,250 £52,500 £52,500
Median annual salary (50th Percentile) £75,000 £70,000 £70,000
Median % change year-on-year +7.14% - +3.70%
75th Percentile £95,000 £95,000 £85,000
90th Percentile £110,000 £112,500 £107,050
UK excluding London median annual salary £61,900 £57,500 £57,500
% change year-on-year +7.65% - -4.17%

Hadoop is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Permanent vacancies with a requirement for database or business intelligence skills 35,355 14,197 37,090
As % of all permanent IT jobs advertised in the UK 27.84% 28.67% 27.27%
Number of salaries quoted 22,535 12,067 30,787
10th Percentile £35,000 £33,750 £31,250
25th Percentile £42,500 £42,000 £38,250
Median annual salary (50th Percentile) £57,500 £55,000 £52,500
Median % change year-on-year +4.55% +4.76% +5.00%
75th Percentile £77,500 £72,500 £70,000
90th Percentile £95,000 £88,750 £86,250
UK excluding London median annual salary £50,000 £47,500 £45,000
% change year-on-year +5.26% +5.56% -

Hadoop
Job Vacancy Trend

Job postings citing Hadoop as a proportion of all IT jobs advertised.

Job vacancy trend for Hadoop in the UK

Hadoop
Salary Trend

3-month moving average salary quoted in jobs citing Hadoop.

Salary trend for Hadoop in the UK

Hadoop
Salary Histogram

Salary distribution for jobs citing Hadoop over the 6 months to 27 October 2021.

Salary histogram for Hadoop in the UK

Hadoop
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Hadoop within the UK over the 6 months to 27 October 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -61 1,260 £80,000 +10.34% 195
London -62 812 £80,000 +3.23% 114
UK excluding London -52 534 £61,900 +7.65% 88
Work from Home -75 259 £83,750 +34.00% 28
South East -64 160 £75,000 +25.00% 22
Midlands -99 86 £60,000 -11.11% 30
Scotland -49 82 £57,500 -4.17% 6
North of England -81 71 £57,500 +6.98% 16
South West -62 64 £68,750 +10.00% 6
West Midlands -59 48 £58,750 +2.17% 22
Yorkshire -68 46 £57,500 +6.98% 2
East of England -72 44 £105,000 +50.00% 5
East Midlands -66 38 £65,000 -7.14% 8
North West -86 23 £57,500 +4.55% 11
Wales -66 18 £45,000 - 1
Northern Ireland - 9 £50,750 - 2
North East -28 2 £56,914 +35.51% 3

For the 6 months to 27 October 2021, IT jobs citing Hadoop also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Hadoop.

1 885 (61.93%) Apache Spark
2 789 (55.21%) Big Data
3 757 (52.97%) Python
4 738 (51.64%) AWS
5 680 (47.59%) SQL
6 668 (46.75%) Java
7 582 (40.73%) Analytics
8 581 (40.66%) Agile Software Development
9 569 (39.82%) Kafka
10 485 (33.94%) Azure
11 420 (29.39%) GCP
12 398 (27.85%) Software Engineering
13 385 (26.94%) DevOps
14 374 (26.17%) NoSQL
15 364 (25.47%) Data Science
16 359 (25.12%) Kubernetes
17 341 (23.86%) ETL
18 334 (23.37%) Jenkins
19 332 (23.23%) Finance
20 313 (21.90%) Docker
21 305 (21.34%) Social Skills
22 294 (20.57%) Apache Hive
23 284 (19.87%) Continuous Integration
24 283 (19.80%) Data Modelling
25 265 (18.54%) Business Intelligence
25 265 (18.54%) Ansible
25 265 (18.54%) Microservices
26 261 (18.26%) Scala
27 259 (18.12%) Data Warehouse
28 255 (17.84%) Degree

Hadoop
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 885 (61.93%) Apache Spark
2 228 (15.96%) Apache
3 180 (12.60%) Elasticsearch
4 104 (7.28%) Apache Airflow
5 89 (6.23%) SAS
6 52 (3.64%) Cloud Foundry
7 42 (2.94%) Jupyter
8 30 (2.10%) Apache Pig
8 30 (2.10%) Luigi
9 27 (1.89%) Confluence
10 21 (1.47%) Tomcat
11 15 (1.05%) Apache Solr
12 10 (0.70%) Oracle SOA Suite
13 8 (0.56%) GemFire
14 7 (0.49%) Apache Synapse
14 7 (0.49%) Oracle Workflow
15 6 (0.42%) CMS
16 5 (0.35%) Appian
17 4 (0.28%) Apache Pulsar
17 4 (0.28%) SharePoint
Applications
1 22 (1.54%) Microsoft Excel
2 13 (0.91%) Microsoft PowerPoint
3 8 (0.56%) MS Visio
4 6 (0.42%) Chatbot
5 5 (0.35%) InVision
6 4 (0.28%) Microsoft Office
7 3 (0.21%) Spreadsheet
8 2 (0.14%) SPSS
Business Applications
1 19 (1.33%) Oracle BPM
2 4 (0.28%) Unit4 Business World
3 2 (0.14%) Oracle EBS
3 2 (0.14%) SAP S/4HANA
4 1 (0.070%) SAP Oil and Gas
Cloud Services
1 738 (51.64%) AWS
2 485 (33.94%) Azure
3 420 (29.39%) GCP
4 151 (10.57%) Amazon Kinesis
5 136 (9.52%) Serverless
6 127 (8.89%) Power Platform
7 92 (6.44%) Amazon EC2
8 89 (6.23%) Snowflake
9 78 (5.46%) AWS Lambda
10 75 (5.25%) OpenShift
11 73 (5.11%) Amazon S3
12 59 (4.13%) PaaS
13 56 (3.92%) Amazon EMR
13 56 (3.92%) Cloud Computing
14 49 (3.43%) SaaS
15 47 (3.29%) AWS CloudFormation
16 41 (2.87%) Azure Data Factory
17 35 (2.45%) GitHub
18 31 (2.17%) Azure Cosmos DB
18 31 (2.17%) Azure Synapse Analytics
Communications & Networking
1 62 (4.34%) HTTP
2 25 (1.75%) Wireless
3 20 (1.40%) Internet
4 14 (0.98%) NetScaler
5 13 (0.91%) Firewall
6 12 (0.84%) SSL
7 10 (0.70%) Broadband
8 9 (0.63%) SMS
9 5 (0.35%) DNS
9 5 (0.35%) TCP/IP
10 4 (0.28%) Kerberos
10 4 (0.28%) Sourcefire
10 4 (0.28%) VLAN
11 2 (0.14%) Network Security
Database & Business Intelligence
1 789 (55.21%) Big Data
2 374 (26.17%) NoSQL
3 294 (20.57%) Apache Hive
4 259 (18.12%) Data Warehouse
5 227 (15.89%) MongoDB
6 194 (13.58%) HBase
7 193 (13.51%) Data Lake
7 193 (13.51%) Relational Database
8 189 (13.23%) MySQL
9 177 (12.39%) Amazon Redshift
10 171 (11.97%) Tableau
11 156 (10.92%) PostgreSQL
12 147 (10.29%) Apache Cassandra
13 140 (9.80%) SQL Server
14 127 (8.89%) Power BI
15 112 (7.84%) Flink
16 103 (7.21%) BigQuery
17 97 (6.79%) Amazon DynamoDB
18 69 (4.83%) Impala
19 58 (4.06%) Amazon RDS
Development Applications
1 334 (23.37%) Jenkins
2 124 (8.68%) Git (software)
3 94 (6.58%) JMeter
4 83 (5.81%) Atlassian Bamboo
5 69 (4.83%) JUnit
6 60 (4.20%) Eclipse
6 60 (4.20%) Selenium
7 50 (3.50%) Bitbucket
7 50 (3.50%) JIRA
8 44 (3.08%) Robot Framework
9 40 (2.80%) Gradle
10 34 (2.38%) WebDriver
11 32 (2.24%) Cucumber
12 31 (2.17%) Maven
12 31 (2.17%) SonarQube
13 27 (1.89%) Vagrant
14 21 (1.47%) Postman
15 19 (1.33%) Cypress.io
16 18 (1.26%) GitLab
17 17 (1.19%) ERwin
General
1 332 (23.23%) Finance
2 305 (21.34%) Social Skills
3 143 (10.01%) Banking
4 120 (8.40%) Analytical Skills
5 116 (8.12%) Inclusion and Diversity
6 97 (6.79%) Presentation Skills
7 93 (6.51%) Retail
8 85 (5.95%) Law
9 38 (2.66%) Games
10 36 (2.52%) Pharmaceutical
11 35 (2.45%) Manufacturing
12 33 (2.31%) Advertising
13 28 (1.96%) Public Sector
14 27 (1.89%) Marketing
15 26 (1.82%) Telecoms
16 20 (1.40%) Legal
17 18 (1.26%) Investment Banking
18 16 (1.12%) Influencing Skills
19 15 (1.05%) Retail Banking
20 8 (0.56%) Automotive
Job Titles
1 362 (25.33%) Data Engineer
2 289 (20.22%) Developer
3 165 (11.55%) Architect
4 164 (11.48%) Java Developer
5 140 (9.80%) Software Engineer
6 120 (8.40%) Data Architect
7 103 (7.21%) Big Data Engineer
8 94 (6.58%) Senior Data Engineer
9 88 (6.16%) Data Scientist
10 66 (4.62%) Senior Developer
11 52 (3.64%) Consultant
12 49 (3.43%) Senior Software Engineer
13 48 (3.36%) Principal Engineer
14 47 (3.29%) Senior Java Developer
15 45 (3.15%) DevOps Engineer
16 43 (3.01%) Data Manager
17 41 (2.87%) Java Engineer
18 37 (2.59%) Scala Engineer
19 35 (2.45%) Platform Engineer
20 34 (2.38%) Big Data Architect
Libraries, Frameworks & Software Standards
1 569 (39.82%) Kafka
2 182 (12.74%) Spring
3 166 (11.62%) Spring Boot
4 156 (10.92%) REST
5 141 (9.87%) Web Services
6 125 (8.75%) RabbitMQ
7 89 (6.23%) JSON
8 83 (5.81%) React
9 73 (5.11%) Node.js
10 66 (4.62%) RESTful
11 61 (4.27%) AngularJS
11 61 (4.27%) CSS
11 61 (4.27%) Elastic Stack
12 60 (4.20%) PySpark
13 59 (4.13%) HTML
14 58 (4.06%) Apache Beam
14 58 (4.06%) Spark Streaming
15 55 (3.85%) Apache NiFi
15 55 (3.85%) Apache Thrift
16 49 (3.43%) Middleware
Miscellaneous
1 130 (9.10%) Algorithms
2 74 (5.18%) Data Structures
3 70 (4.90%) User Experience
4 59 (4.13%) Virtual Team
5 57 (3.99%) Self-Motivation
6 52 (3.64%) Cloud Native
7 48 (3.36%) Cyberattack
7 48 (3.36%) Public Cloud
8 42 (2.94%) Management Information System
9 38 (2.66%) Hybrid Cloud
10 32 (2.24%) Life Science
11 26 (1.82%) Social Media
12 25 (1.75%) Greenfield Project
13 19 (1.33%) Data Centre
14 17 (1.19%) Linux Command Line
15 15 (1.05%) Equities
15 15 (1.05%) Social Network
16 14 (0.98%) Hedge funds
17 13 (0.91%) Credit Risk
18 11 (0.77%) Enterprise Software
Operating Systems
1 242 (16.93%) Linux
2 178 (12.46%) Unix
3 68 (4.76%) Windows
4 40 (2.80%) CentOS
5 27 (1.89%) Windows Server
6 24 (1.68%) Red Hat Enterprise Linux
7 8 (0.56%) Solaris
8 4 (0.28%) DC/OS
9 2 (0.14%) VMS
10 1 (0.070%) CoreOS
10 1 (0.070%) Windows Server 2012
Processes & Methodologies
1 582 (40.73%) Analytics
2 581 (40.66%) Agile Software Development
3 398 (27.85%) Software Engineering
4 385 (26.94%) DevOps
5 364 (25.47%) Data Science
6 341 (23.86%) ETL
7 284 (19.87%) Continuous Integration
8 283 (19.80%) Data Modelling
9 265 (18.54%) Business Intelligence
9 265 (18.54%) Microservices
10 254 (17.77%) Data Engineering
10 254 (17.77%) Machine Learning
11 247 (17.28%) Data Pipeline
12 204 (14.28%) Computer Science
13 196 (13.72%) Visualisation
14 190 (13.30%) Artificial Intelligence
15 161 (11.27%) Data Management
16 160 (11.20%) Mentoring
17 159 (11.13%) Open Source
18 147 (10.29%) Scrum
Programming Languages
1 757 (52.97%) Python
2 680 (47.59%) SQL
3 668 (46.75%) Java
4 261 (18.26%) Scala
5 149 (10.43%) R
6 120 (8.40%) JavaScript
7 78 (5.46%) C#
8 69 (4.83%) Shell Script
9 68 (4.76%) Go
10 53 (3.71%) Bash Shell
11 50 (3.50%) PL/SQL
12 47 (3.29%) PowerShell
13 45 (3.15%) TypeScript
14 34 (2.38%) Groovy
15 31 (2.17%) C++
15 31 (2.17%) Java 8
16 23 (1.61%) C
17 22 (1.54%) HiveQL
18 19 (1.33%) Perl
19 18 (1.26%) Ruby
Qualifications
1 255 (17.84%) Degree
2 108 (7.56%) Security Cleared
3 83 (5.81%) Computer Science Degree
4 82 (5.74%) SC Cleared
5 38 (2.66%) PhD
6 36 (2.52%) Master's Degree
7 31 (2.17%) DV Cleared
8 14 (0.98%) RHCE
9 12 (0.84%) SAS Certification
10 7 (0.49%) BPSS Clearance
11 6 (0.42%) AWS Certification
11 6 (0.42%) Microsoft Certification
12 5 (0.35%) TOGAF Certification
13 4 (0.28%) MCSA
14 3 (0.21%) CFA
15 2 (0.14%) CCA
15 2 (0.14%) MCDBA
15 2 (0.14%) MCT
16 1 (0.070%) PMI Certification
16 1 (0.070%) PMP
Quality Assurance & Compliance
1 87 (6.09%) Data Quality
2 46 (3.22%) QA
3 37 (2.59%) GDPR
4 16 (1.12%) GxP
5 8 (0.56%) Accessibility
6 5 (0.35%) Actionable Recommendations
7 4 (0.28%) PMO
7 4 (0.28%) SLA
8 3 (0.21%) Sarbanes-Oxley
9 1 (0.070%) COBIT
9 1 (0.070%) IFRS
9 1 (0.070%) IFRS 9
9 1 (0.070%) ISO/IEC 27001
9 1 (0.070%) NIST
System Software
1 313 (21.90%) Docker
2 73 (5.11%) HDFS
3 43 (3.01%) Apache ZooKeeper
4 27 (1.89%) VirtualBox
5 25 (1.75%) VMware Infrastructure
6 17 (1.19%) Active Directory
7 14 (0.98%) vSphere
7 14 (0.98%) XenApp
7 14 (0.98%) XenDesktop
8 11 (0.77%) Apache Flume
9 7 (0.49%) Virtual Machines
10 1 (0.070%) Hyper-V
Systems Management
1 359 (25.12%) Kubernetes
2 265 (18.54%) Ansible
3 208 (14.56%) Terraform
4 62 (4.34%) Puppet
5 43 (3.01%) Opscode Chef
6 30 (2.10%) Docker Swarm
7 17 (1.19%) Single Sign-On
8 16 (1.12%) Prometheus
9 14 (0.98%) Mesos
9 14 (0.98%) Oozie
10 13 (0.91%) Control-M
10 13 (0.91%) Hadoop YARN
11 8 (0.56%) Consul
12 7 (0.49%) Istio
12 7 (0.49%) Kubeflow
13 6 (0.42%) HAProxy
14 5 (0.35%) Grafana
14 5 (0.35%) Salt
15 4 (0.28%) Kibana
15 4 (0.28%) Tivoli
Vendors
1 247 (17.28%) Oracle
2 194 (13.58%) Google
3 139 (9.73%) Microsoft
4 134 (9.38%) Databricks
5 96 (6.72%) Informatica
6 77 (5.39%) Talend
7 76 (5.32%) Cloudera
8 75 (5.25%) Teradata
9 70 (4.90%) Splunk
10 56 (3.92%) Atlassian
11 47 (3.29%) Hortonworks
12 42 (2.94%) IBM
13 30 (2.10%) StreamSets
14 26 (1.82%) VMware
15 21 (1.47%) Facebook
15 21 (1.47%) Samsung
16 16 (1.12%) Dataiku
16 16 (1.12%) F5
16 16 (1.12%) Red Hat
17 14 (0.98%) MicroStrategy