Period
to 29 July 2021

The following table provides summary statistics for permanent job vacancies with a requirement for Hadoop skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Hadoop over the 6 months to 29 July 2021 with a comparison to the same period in the previous 2 years.

Apache Hadoop
UK
6 months to
29 Jul 2021
Same period 2020 Same period 2019
Rank 282 237 208
Rank change year-on-year -45 -29 -19
Permanent jobs citing Hadoop 1,323 983 2,520
As % of all permanent jobs advertised in the UK 1.17% 1.42% 1.75%
As % of the Database & Business Intelligence category 4.18% 5.16% 6.29%
Number of salaries quoted 854 661 1,909
10th Percentile £46,250 £45,000 £44,900
25th Percentile £53,094 £55,000 £53,250
Median annual salary (50th Percentile) £75,000 £72,500 £70,000
Median % change year-on-year +3.45% +3.57% +7.69%
75th Percentile £95,000 £91,250 £87,500
90th Percentile £110,000 £115,000 £107,500
UK excluding London median annual salary £60,000 £60,000 £57,500
% change year-on-year - +4.35% +4.55%

Hadoop is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Permanent vacancies with a requirement for database or business intelligence skills 31,672 19,060 40,041
As % of all permanent IT jobs advertised in the UK 28.05% 27.62% 27.81%
Number of salaries quoted 21,204 15,650 33,146
10th Percentile £34,750 £32,500 £31,250
25th Percentile £42,500 £41,250 £37,500
Median annual salary (50th Percentile) £55,000 £55,000 £52,000
Median % change year-on-year - +5.77% +4.00%
75th Percentile £75,000 £72,500 £70,000
90th Percentile £92,500 £88,750 £85,000
UK excluding London median annual salary £48,500 £47,500 £45,000
% change year-on-year +2.11% +5.56% -

Hadoop
Job Vacancy Trend

Job postings citing Hadoop as a proportion of all IT jobs advertised.

Job vacancy trend for Hadoop in the UK

Hadoop
Salary Trend

3-month moving average salary quoted in jobs citing Hadoop.

Salary trend for Hadoop in the UK

Hadoop
Salary Histogram

Salary distribution for jobs citing Hadoop over the 6 months to 29 July 2021.

Salary histogram for Hadoop in the UK

Hadoop
Top 18 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Hadoop within the UK over the 6 months to 29 July 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -56 1,195 £77,500 +3.33% 180
London -44 810 £80,000 +5.26% 119
UK excluding London -68 457 £60,000 - 74
Work from Home -55 238 £82,500 +3.13% 32
South East -78 139 £72,500 +7.41% 19
South West -59 76 £73,870 +31.32% 5
Midlands -83 63 £55,500 -20.71% 9
North of England -40 62 £58,750 +6.82% 24
Scotland -57 58 £56,250 +2.27% 6
East Midlands -58 32 £50,000 -33.33% 2
North West -45 31 £60,000 +20.00% 12
West Midlands -55 31 £60,000 +9.09% 7
Yorkshire -49 29 £58,750 +6.82% 9
Wales -23 26 £47,500 +5.56% 6
East of England -33 26 £106,250 +63.46% 2
Northern Ireland -32 6 £50,750 - 3
North East -26 2 £56,914 -1.02% 3
Isle of Man - 1 £37,500 -

For the 6 months to 29 July 2021, IT jobs citing Hadoop also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Hadoop.

1 831 (62.81%) Apache Spark
2 789 (59.64%) Big Data
3 705 (53.29%) AWS
4 694 (52.46%) Python
5 631 (47.69%) Java
6 593 (44.82%) SQL
7 555 (41.95%) Agile Software Development
8 531 (40.14%) Kafka
9 527 (39.83%) Azure
10 479 (36.21%) Analytics
11 429 (32.43%) GCP
11 429 (32.43%) Kubernetes
12 393 (29.71%) Software Engineering
13 376 (28.42%) Docker
14 366 (27.66%) NoSQL
15 344 (26.00%) DevOps
16 341 (25.77%) ETL
17 323 (24.41%) Jenkins
18 318 (24.04%) Finance
19 306 (23.13%) Data Science
20 289 (21.84%) Continuous Integration
21 284 (21.47%) Ansible
22 279 (21.09%) Data Engineering
23 278 (21.01%) Data Pipeline
24 251 (18.97%) Microservices
25 249 (18.82%) Scala
26 246 (18.59%) Social Skills
27 245 (18.52%) Apache Hive
28 237 (17.91%) Terraform
29 229 (17.31%) Linux

Hadoop
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 831 (62.81%) Apache Spark
2 165 (12.47%) Apache
3 164 (12.40%) Elasticsearch
4 96 (7.26%) Apache Airflow
5 60 (4.54%) SAS
6 49 (3.70%) Cloud Foundry
7 35 (2.65%) Apache Pig
8 34 (2.57%) Luigi
9 24 (1.81%) Confluence
10 18 (1.36%) Jupyter
11 15 (1.13%) Oracle Workflow
11 15 (1.13%) Tomcat
12 10 (0.76%) OpenStack
12 10 (0.76%) Oracle SOA Suite
13 7 (0.53%) GemFire
13 7 (0.53%) WebLogic
14 6 (0.45%) Apache Solr
14 6 (0.45%) CMS
14 6 (0.45%) IIS
14 6 (0.45%) nginx
Applications
1 15 (1.13%) Microsoft Excel
2 11 (0.83%) Microsoft PowerPoint
3 8 (0.60%) Microsoft Office
3 8 (0.60%) MS Visio
4 4 (0.30%) Chatbot
5 3 (0.23%) SPSS
6 1 (0.076%) Microsoft Project
6 1 (0.076%) Spreadsheet
Business Applications
1 11 (0.83%) Oracle BPM
2 3 (0.23%) Oracle EBS
3 2 (0.15%) Unit4 Business World
4 1 (0.076%) Dynamics CRM
Cloud Services
1 705 (53.29%) AWS
2 527 (39.83%) Azure
3 429 (32.43%) GCP
4 135 (10.20%) Serverless
5 120 (9.07%) Power Platform
6 109 (8.24%) Amazon Kinesis
7 102 (7.71%) Snowflake
8 93 (7.03%) Amazon EMR
9 81 (6.12%) AWS Lambda
10 68 (5.14%) IBM Cloud
11 66 (4.99%) Amazon EC2
12 62 (4.69%) OpenShift
13 60 (4.54%) SaaS
14 54 (4.08%) Cloud Functions
15 51 (3.85%) Cloud Computing
16 48 (3.63%) PaaS
17 45 (3.40%) AWS CloudFormation
18 39 (2.95%) Azure Data Factory
19 34 (2.57%) Amazon S3
19 34 (2.57%) Google Compute Engine
Communications & Networking
1 53 (4.01%) HTTP
2 28 (2.12%) Internet
3 19 (1.44%) Wireless
4 18 (1.36%) Firewall
5 10 (0.76%) NetScaler
6 9 (0.68%) FTP
6 9 (0.68%) SSL
7 8 (0.60%) SMS
8 7 (0.53%) TCP/IP
9 6 (0.45%) VLAN
10 4 (0.30%) DNS
10 4 (0.30%) Sourcefire
11 2 (0.15%) Broadband
12 1 (0.076%) Skype
Database & Business Intelligence
1 789 (59.64%) Big Data
2 366 (27.66%) NoSQL
3 245 (18.52%) Apache Hive
4 200 (15.12%) Data Warehouse
5 196 (14.81%) MongoDB
6 177 (13.38%) Amazon Redshift
7 166 (12.55%) Data Lake
8 154 (11.64%) Relational Database
9 143 (10.81%) HBase
10 133 (10.05%) SQL Server
11 129 (9.75%) MySQL
12 122 (9.22%) Tableau
13 120 (9.07%) Apache Cassandra
13 120 (9.07%) Power BI
14 89 (6.73%) PostgreSQL
15 85 (6.42%) Neo4j
16 83 (6.27%) Amazon Athena
17 72 (5.44%) Electronic Patient Records
18 69 (5.22%) Flink
19 64 (4.84%) Amazon DynamoDB
Development Applications
1 323 (24.41%) Jenkins
2 122 (9.22%) Atlassian Bamboo
3 113 (8.54%) Git (software)
4 98 (7.41%) JMeter
5 74 (5.59%) Vagrant
6 56 (4.23%) Selenium
7 50 (3.78%) Eclipse
7 50 (3.78%) JUnit
8 49 (3.70%) Gradle
9 46 (3.48%) JIRA
10 38 (2.87%) SonarQube
11 37 (2.80%) Robot Framework
12 35 (2.65%) Maven
13 33 (2.49%) Postman
14 31 (2.34%) Bitbucket
14 31 (2.34%) Cucumber
15 30 (2.27%) Cypress.io
15 30 (2.27%) WebDriver
16 23 (1.74%) ERwin
17 20 (1.51%) TeamCity
General
1 318 (24.04%) Finance
2 246 (18.59%) Social Skills
3 151 (11.41%) Banking
4 129 (9.75%) Analytical Skills
5 83 (6.27%) Law
6 80 (6.05%) Presentation Skills
7 75 (5.67%) Inclusion and Diversity
8 69 (5.22%) Retail
9 35 (2.65%) Advertising
10 31 (2.34%) Telecoms
11 30 (2.27%) Games
12 27 (2.04%) Manufacturing
12 27 (2.04%) Marketing
13 23 (1.74%) Retail Banking
14 22 (1.66%) Pharmaceutical
15 19 (1.44%) Investment Banking
16 17 (1.28%) Influencing Skills
16 17 (1.28%) Public Sector
17 13 (0.98%) Front Office
17 13 (0.98%) Legal
Job Titles
1 384 (29.02%) Data Engineer
2 258 (19.50%) Developer
3 162 (12.24%) Architect
4 151 (11.41%) Java Developer
5 149 (11.26%) Software Engineer
6 121 (9.15%) Big Data Engineer
7 118 (8.92%) Data Architect
8 95 (7.18%) Senior Data Engineer
9 65 (4.91%) Platform Engineer
10 62 (4.69%) Data Scientist
11 58 (4.38%) Java Engineer
12 52 (3.93%) AWS Engineer
12 52 (3.93%) DevOps Engineer
13 51 (3.85%) Senior Developer
14 45 (3.40%) Consultant
15 43 (3.25%) Senior Software Engineer
16 41 (3.10%) Java Software Engineer
17 36 (2.72%) Big Data Architect
18 34 (2.57%) Platform Architect
19 33 (2.49%) Senior Java Developer
Libraries, Frameworks & Software Standards
1 531 (40.14%) Kafka
2 157 (11.87%) Spring Boot
3 153 (11.56%) Spring
4 142 (10.73%) Web Services
5 133 (10.05%) REST
6 111 (8.39%) Spark Streaming
7 104 (7.86%) React
8 95 (7.18%) RabbitMQ
9 91 (6.88%) AngularJS
10 89 (6.73%) Node.js
11 69 (5.22%) JSON
12 65 (4.91%) HTML
13 59 (4.46%) CSS
14 56 (4.23%) RESTful
15 53 (4.01%) Apache NiFi
16 50 (3.78%) Apache Thrift
16 50 (3.78%) Elastic Stack
16 50 (3.78%) Middleware
17 46 (3.48%) PySpark
18 44 (3.33%) HTML5
Miscellaneous
1 74 (5.59%) Algorithms
2 59 (4.46%) Self-Motivation
3 57 (4.31%) Hybrid Cloud
4 55 (4.16%) Data Structures
5 50 (3.78%) User Experience
6 47 (3.55%) Cyberattack
7 32 (2.42%) Virtual Team
8 27 (2.04%) Cloud Native
8 27 (2.04%) Equities
8 27 (2.04%) Hedge funds
9 24 (1.81%) Greenfield Project
10 21 (1.59%) Data Centre
10 21 (1.59%) Public Cloud
11 20 (1.51%) Management Information System
12 15 (1.13%) Social Media
13 13 (0.98%) Internet of Things
13 13 (0.98%) Private Cloud
14 12 (0.91%) Legacy Systems
14 12 (0.91%) Linux Command Line
15 11 (0.83%) PKI
Operating Systems
1 229 (17.31%) Linux
2 128 (9.67%) Unix
3 63 (4.76%) Windows
4 37 (2.80%) CentOS
5 31 (2.34%) Red Hat Enterprise Linux
6 20 (1.51%) Windows Server
7 8 (0.60%) Solaris
8 1 (0.076%) AIX
8 1 (0.076%) Android
8 1 (0.076%) Apple iOS
8 1 (0.076%) CoreOS
8 1 (0.076%) Windows Server 2012
Processes & Methodologies
1 555 (41.95%) Agile Software Development
2 479 (36.21%) Analytics
3 393 (29.71%) Software Engineering
4 344 (26.00%) DevOps
5 341 (25.77%) ETL
6 306 (23.13%) Data Science
7 289 (21.84%) Continuous Integration
8 279 (21.09%) Data Engineering
9 278 (21.01%) Data Pipeline
10 251 (18.97%) Microservices
11 224 (16.93%) Artificial Intelligence
12 206 (15.57%) Business Intelligence
13 205 (15.50%) Visualisation
14 200 (15.12%) Data Modelling
15 188 (14.21%) Machine Learning
16 161 (12.17%) Computer Science
17 156 (11.79%) Open Source
18 138 (10.43%) Fintech
19 125 (9.45%) Data Architecture
20 123 (9.30%) Unit Testing
Programming Languages
1 694 (52.46%) Python
2 631 (47.69%) Java
3 593 (44.82%) SQL
4 249 (18.82%) Scala
5 121 (9.15%) JavaScript
6 110 (8.31%) C#
7 93 (7.03%) R
8 57 (4.31%) TypeScript
9 55 (4.16%) Bash Shell
10 52 (3.93%) Shell Script
11 46 (3.48%) Java 8
12 45 (3.40%) Go
13 42 (3.17%) PowerShell
14 28 (2.12%) C++
15 26 (1.97%) PHP
16 23 (1.74%) Kotlin
17 22 (1.66%) C
18 19 (1.44%) PL/SQL
19 17 (1.28%) Perl
20 16 (1.21%) ES6
Qualifications
1 208 (15.72%) Degree
2 95 (7.18%) Security Cleared
3 72 (5.44%) Computer Science Degree
4 66 (4.99%) SC Cleared
5 41 (3.10%) DV Cleared
6 23 (1.74%) PhD
7 21 (1.59%) Master's Degree
8 10 (0.76%) RHCE
9 6 (0.45%) BPSS Clearance
10 4 (0.30%) SAS Certification
10 4 (0.30%) TOGAF Certification
11 3 (0.23%) AWS Certification
11 3 (0.23%) CFA
11 3 (0.23%) Microsoft Certification
11 3 (0.23%) Numerate Degree
12 2 (0.15%) MCDBA
12 2 (0.15%) MCT
13 1 (0.076%) DBS Check
13 1 (0.076%) MCSA
Quality Assurance & Compliance
1 44 (3.33%) Data Quality
2 42 (3.17%) QA
3 32 (2.42%) GDPR
4 7 (0.53%) Accessibility
5 6 (0.45%) GxP
6 4 (0.30%) SLA
7 3 (0.23%) ISO/IEC 27001
8 2 (0.15%) COBIT
8 2 (0.15%) NIST
System Software
1 376 (28.42%) Docker
2 74 (5.59%) VirtualBox
3 71 (5.37%) HDFS
4 40 (3.02%) Apache ZooKeeper
5 19 (1.44%) VMware Infrastructure
6 17 (1.28%) Apache Flume
7 12 (0.91%) Active Directory
8 10 (0.76%) vSphere
8 10 (0.76%) XenApp
8 10 (0.76%) XenDesktop
9 6 (0.45%) Oracle RAC
10 3 (0.23%) Virtual Machines
11 2 (0.15%) Firmware
12 1 (0.076%) Hyper-V
Systems Management
1 429 (32.43%) Kubernetes
2 284 (21.47%) Ansible
3 237 (17.91%) Terraform
4 75 (5.67%) Docker Swarm
5 63 (4.76%) Puppet
6 33 (2.49%) Opscode Chef
7 16 (1.21%) Prometheus
8 12 (0.91%) Hadoop YARN
8 12 (0.91%) Oozie
9 9 (0.68%) Single Sign-On
10 7 (0.53%) ITRS Geneos
11 6 (0.45%) Data Guard
11 6 (0.45%) Mesos
11 6 (0.45%) Oracle RMAN
12 4 (0.30%) Control-M
12 4 (0.30%) Istio
12 4 (0.30%) Tivoli
13 3 (0.23%) Kubeflow
14 2 (0.15%) Kibana
14 2 (0.15%) Salt
Vendors
1 195 (14.74%) Google
2 176 (13.30%) Oracle
3 133 (10.05%) Cloudera
4 115 (8.69%) Talend
5 112 (8.47%) Microsoft
6 109 (8.24%) Databricks
7 108 (8.16%) Informatica
8 89 (6.73%) Hortonworks
9 86 (6.50%) IBM
10 65 (4.91%) Teradata
11 53 (4.01%) Splunk
12 48 (3.63%) Atlassian
13 20 (1.51%) VMware
14 19 (1.44%) SAP
15 16 (1.21%) Salesforce.com
16 15 (1.13%) Alteryx
16 15 (1.13%) Red Hat
17 14 (1.06%) Samsung
18 12 (0.91%) AppDynamics
19 11 (0.83%) F5