Big Data
UK

The following table provides summary statistics for permanent job vacancies with a requirement for Big Data skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Big Data over the 6 months to 23 May 2024 with a comparison to the same period in the previous 2 years.

6 months to
23 May 2024
Same period 2023 Same period 2022
Rank 241 132 113
Rank change year-on-year -109 -19 +21
Permanent jobs citing Big Data 1,193 2,589 5,310
As % of all permanent jobs advertised in the UK 1.18% 2.63% 3.24%
As % of the Database & Business Intelligence category 6.53% 12.25% 11.53%
Number of salaries quoted 487 1,023 1,876
10th Percentile £43,750 £51,250 £46,250
25th Percentile £57,500 £62,500 £60,000
Median annual salary (50th Percentile) £80,000 £70,000 £80,000
Median % change year-on-year +14.29% -12.50% +23.08%
75th Percentile £110,000 £92,500 £95,000
90th Percentile £122,500 £110,000 £110,000
UK excluding London median annual salary £64,250 £70,000 £69,500
% change year-on-year -8.21% +0.72% +26.36%

All Database and Business Intelligence Skills
UK

Big Data is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for database or business intelligence skills.

Permanent vacancies with a requirement for database or business intelligence skills 18,281 21,129 46,064
As % of all permanent jobs advertised in the UK 18.11% 21.45% 28.12%
Number of salaries quoted 12,536 13,165 21,039
10th Percentile £35,000 £37,500 £36,250
25th Percentile £42,725 £47,500 £45,000
Median annual salary (50th Percentile) £57,500 £65,000 £60,000
Median % change year-on-year -11.54% +8.33% +9.09%
75th Percentile £77,500 £82,500 £80,000
90th Percentile £97,500 £103,750 £95,000
UK excluding London median annual salary £52,500 £55,000 £52,500
% change year-on-year -4.55% +4.76% +10.53%

Big Data
Job Vacancy Trend

Job postings citing Big Data as a proportion of all IT jobs advertised.

Job vacancy trend for Big Data in the UK

Big Data
Salary Trend

3-month moving average salary quoted in jobs citing Big Data.

Salary trend for Big Data in the UK

Big Data
Salary Histogram

Salary distribution for jobs citing Big Data over the 6 months to 23 May 2024.

Salary histogram for Big Data in the UK

Big Data
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Big Data within the UK over the 6 months to 23 May 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
UK excluding London -82 845 £64,250 -8.21% 171
England -238 596 £80,000 +14.29% 258
Scotland +70 554 £70,576 +0.82% 20
London -138 317 £95,000 +18.32% 157
Work from Home -187 294 £87,500 +25.00% 112
South West -37 107 £80,000 +14.29% 19
North of England -215 62 £55,000 -21.43% 27
South East -172 45 £54,500 -22.14% 65
North West -140 44 £55,000 -21.43% 15
East of England -40 35 £60,000 -7.69% 10
Midlands -129 28 £58,750 -6.00% 20
West Midlands -82 18 £57,500 -4.17% 16
North East -5 11 £42,500 -37.04% 6
Wales -16 11 £46,131 -34.10% 2
East Midlands -40 10 £60,000 -20.00% 4
Yorkshire -117 7 £92,778 +32.54% 6
Northern Ireland +14 3 £115,000 +31.43% 8

Big Data
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 713 (59.77%) Apache Spark
2 231 (19.36%) Apache
3 180 (15.09%) Apache Airflow
4 41 (3.44%) SAS
5 19 (1.59%) Jupyter
6 18 (1.51%) nginx
7 8 (0.67%) Apache Pig
8 6 (0.50%) Luigi
8 6 (0.50%) OpenStack
9 5 (0.42%) Kinematics
10 3 (0.25%) Confluence
10 3 (0.25%) Exchange Server 2013
10 3 (0.25%) Microsoft Exchange
11 1 (0.084%) Apache Solr
11 1 (0.084%) CMS
Applications
1 30 (2.51%) Microsoft Excel
2 28 (2.35%) Microsoft Office
3 15 (1.26%) SPSS
4 6 (0.50%) Trello
5 5 (0.42%) Microsoft PowerPoint
6 4 (0.34%) MS Visio
7 1 (0.084%) Microsoft Project
7 1 (0.084%) Spreadsheet
Business Applications
1 7 (0.59%) Dynamics CRM
2 6 (0.50%) Payment Gateway
3 5 (0.42%) Sparx Enterprise Architect
4 4 (0.34%) Primavera
4 4 (0.34%) Relativity
5 3 (0.25%) SAP Business One
6 2 (0.17%) Veritas eDiscovery
7 1 (0.084%) Magento
7 1 (0.084%) Ringtail
Cloud Services
1 650 (54.48%) AWS
2 264 (22.13%) Azure
3 155 (12.99%) GCP
4 128 (10.73%) Serverless
5 95 (7.96%) AWS Lambda
6 92 (7.71%) Snowflake
7 86 (7.21%) Power Platform
8 83 (6.96%) Amazon EKS
9 79 (6.62%) Amazon SQS
10 71 (5.95%) SaaS
11 68 (5.70%) Amazon EMR
12 67 (5.62%) dbt
13 54 (4.53%) Amazon S3
14 52 (4.36%) Amazon EC2
14 52 (4.36%) OpenShift
15 51 (4.27%) Amazon ECS
16 47 (3.94%) Azure Synapse Analytics
17 41 (3.44%) Azure Data Factory
18 40 (3.35%) Cloud Computing
19 32 (2.68%) AWS CloudFormation
Communications & Networking
1 28 (2.35%) AMQP
2 20 (1.68%) Firewall
3 19 (1.59%) Wireless
4 18 (1.51%) DSL
5 13 (1.09%) Internet
6 5 (0.42%) Network Security
6 5 (0.42%) RFID
7 4 (0.34%) RS-232
7 4 (0.34%) SD-WAN
8 3 (0.25%) DNS
8 3 (0.25%) Fibre Channel
8 3 (0.25%) iSCSI
8 3 (0.25%) LAN
8 3 (0.25%) MQTT
8 3 (0.25%) SAN
8 3 (0.25%) VoIP
8 3 (0.25%) WAN
8 3 (0.25%) WebEx
9 2 (0.17%) 4G
9 2 (0.17%) Intrusion Detection
Database & Business Intelligence
1 382 (32.02%) NoSQL
2 329 (27.58%) Hadoop
3 193 (16.18%) Data Lake
4 127 (10.65%) Data Warehouse
5 92 (7.71%) Apache Hive
6 75 (6.29%) Power BI
7 69 (5.78%) Amazon Athena
7 69 (5.78%) Tableau
8 61 (5.11%) Dremio
9 60 (5.03%) Apache Arrow
10 59 (4.95%) SQL Server
11 53 (4.44%) Electronic Patient Records
12 49 (4.11%) Relational Database
13 41 (3.44%) Data Mining
13 41 (3.44%) MongoDB
14 39 (3.27%) HBase
15 33 (2.77%) Elasticsearch
15 33 (2.77%) Flink
16 30 (2.51%) Amazon Redshift
17 28 (2.35%) Cloudera CDH
Development Applications
1 102 (8.55%) Git
2 84 (7.04%) Jenkins
3 37 (3.10%) Bitbucket Server
4 17 (1.42%) Atlassian Bamboo
5 16 (1.34%) JIRA
6 10 (0.84%) CircleCI
7 9 (0.75%) JUnit
7 9 (0.75%) SonarQube
8 8 (0.67%) Oracle Forms
9 7 (0.59%) Checkstyle
10 6 (0.50%) GitLab
10 6 (0.50%) Jasmine
10 6 (0.50%) Karma Test Runner
10 6 (0.50%) MLflow
10 6 (0.50%) Visual Studio Code
11 4 (0.34%) Cucumber
11 4 (0.34%) Octopus Deploy
11 4 (0.34%) Subversion
11 4 (0.34%) Visual Studio
12 3 (0.25%) SQL Server Management Studio (SSMS)
General
1 675 (56.58%) Finance
2 560 (46.94%) Inclusion and Diversity
3 513 (43.00%) Law
4 285 (23.89%) Marketing
5 258 (21.63%) Legal
6 183 (15.34%) Social Skills
7 165 (13.83%) Influencing Skills
8 115 (9.64%) Banking
9 108 (9.05%) Analytical Skills
10 78 (6.54%) Financial Institution
11 45 (3.77%) Aerospace
12 39 (3.27%) Retail
13 30 (2.51%) Presentation Skills
14 29 (2.43%) Investment Banking
15 19 (1.59%) Private Banking
16 18 (1.51%) Public Sector
17 14 (1.17%) Retail Banking
18 13 (1.09%) Games
18 13 (1.09%) Pharmaceutical
19 12 (1.01%) Electronics
Job Titles
1 490 (41.07%) Software Engineer
2 468 (39.23%) Big Data Engineer
3 390 (32.69%) Big Data Software Engineer
4 354 (29.67%) Data Engineer
5 209 (17.52%) Scala Engineer
5 209 (17.52%) Scala Software Engineer
6 202 (16.93%) AWS Engineer
7 191 (16.01%) Lead
7 191 (16.01%) Senior
8 181 (15.17%) Java Engineer
9 178 (14.92%) Java Software Engineer
10 150 (12.57%) Lead Big Data Engineer
11 149 (12.49%) Lead Software Engineer
12 144 (12.07%) AWS Software Engineer
13 119 (9.97%) Java Scala Software Engineer
14 109 (9.14%) Python Engineer
15 97 (8.13%) Architect
16 96 (8.05%) Lead Java Engineer
17 95 (7.96%) Python Software Engineer
18 69 (5.78%) Developer
Libraries, Frameworks & Software Standards
1 156 (13.08%) Kafka
2 135 (11.32%) PySpark
3 116 (9.72%) Spring
4 106 (8.89%) Spring Boot
5 87 (7.29%) gRPC
6 58 (4.86%) AngularJS
7 54 (4.53%) Protocol Buffers
8 47 (3.94%) RabbitMQ
8 47 (3.94%) React
9 40 (3.35%) Node.js
10 38 (3.19%) REST
11 34 (2.85%) OpenAPI
11 34 (2.85%) Swagger
12 33 (2.77%) .NET
13 28 (2.35%) Material-UI
14 27 (2.26%) Apache Beam
15 25 (2.10%) Apache NiFi
16 20 (1.68%) ActiveMQ
16 20 (1.68%) GraphQL
16 20 (1.68%) Pandas
Miscellaneous
1 260 (21.79%) Cloud Native
2 222 (18.61%) Algorithms
3 69 (5.78%) Data Structures
4 64 (5.36%) Hedge funds
5 61 (5.11%) Management Information System
6 56 (4.69%) Product Ownership
7 55 (4.61%) Clustering
8 53 (4.44%) Public Cloud
9 45 (3.77%) Onboarding
10 43 (3.60%) Legacy Code
11 29 (2.43%) Self-Motivation
12 22 (1.84%) IoT
13 21 (1.76%) Distributed Systems
13 21 (1.76%) Enterprise Software
14 19 (1.59%) Greenfield Project
14 19 (1.59%) Hybrid Cloud
14 19 (1.59%) Legacy Systems
15 15 (1.26%) Cyber Defence
15 15 (1.26%) Cybercrime
16 13 (1.09%) Distributed Applications
Operating Systems
1 100 (8.38%) Linux
2 38 (3.19%) Unix
3 30 (2.51%) CentOS
3 30 (2.51%) Debian
3 30 (2.51%) Red Hat Enterprise Linux
3 30 (2.51%) Ubuntu
4 22 (1.84%) Windows
5 13 (1.09%) Windows Server
6 6 (0.50%) Mesosphere
7 4 (0.34%) FreeRTOS
8 3 (0.25%) Windows Server 2012
9 1 (0.084%) Android
9 1 (0.084%) Apple iOS
Processes & Methodologies
1 728 (61.02%) Agile
2 621 (52.05%) Software Engineering
3 548 (45.93%) CI/CD
4 500 (41.91%) SDLC
5 403 (33.78%) Operational Stability
6 361 (30.26%) Machine Learning
7 311 (26.07%) Artificial Intelligence
8 255 (21.37%) Risk Management
9 244 (20.45%) Visualisation
10 237 (19.87%) Performance Tuning
11 234 (19.61%) Problem-Solving
12 231 (19.36%) Community of Practice
13 227 (19.03%) Analytics
14 213 (17.85%) DevOps
15 202 (16.93%) Data Modelling
16 199 (16.68%) Continuous Improvement
17 192 (16.09%) Data Science
18 155 (12.99%) Business Intelligence
19 154 (12.91%) Mentoring
20 145 (12.15%) Decision-Making
Programming Languages
1 664 (55.66%) SQL
2 580 (48.62%) Java
3 577 (48.37%) Python
4 373 (31.27%) Scala
5 121 (10.14%) JavaScript
6 98 (8.21%) Go
7 58 (4.86%) R
8 55 (4.61%) PL/SQL
9 54 (4.53%) C#
10 37 (3.10%) Bash
11 21 (1.76%) T-SQL
12 20 (1.68%) Shell Script
13 16 (1.34%) C
14 11 (0.92%) C++
14 11 (0.92%) Data Analysis Expressions
15 10 (0.84%) TypeScript
16 8 (0.67%) Ruby
17 6 (0.50%) PHP
18 5 (0.42%) VB
18 5 (0.42%) XAML
Qualifications
1 207 (17.35%) Degree
2 100 (8.38%) Security Cleared
3 68 (5.70%) SC Cleared
4 59 (4.95%) DV Cleared
5 56 (4.69%) Computer Science Degree
6 51 (4.27%) Master's Degree
7 17 (1.42%) PhD
8 16 (1.34%) Microsoft Certification
9 14 (1.17%) AWS Certification
9 14 (1.17%) Azure Certification
10 11 (0.92%) DBS Check
11 10 (0.84%) AWS Certified Solutions Architect
12 8 (0.67%) BPSS Clearance
13 6 (0.50%) IIBA
14 5 (0.42%) CCNA
14 5 (0.42%) Cisco Certification
14 5 (0.42%) CISSP
15 4 (0.34%) CIMA
15 4 (0.34%) Network+ Certification
16 3 (0.25%) MBA
Quality Assurance & Compliance
1 61 (5.11%) Data Quality
2 34 (2.85%) SLA
3 27 (2.26%) GDPR
3 27 (2.26%) QA
4 17 (1.42%) Accessibility
5 6 (0.50%) WAI
6 4 (0.34%) ISO/IEC 27001
7 3 (0.25%) SEPA
8 1 (0.084%) PMO
System Software
1 82 (6.87%) Docker
2 55 (4.61%) HDFS
3 8 (0.67%) VMware Infrastructure
4 6 (0.50%) Hyper-V
5 5 (0.42%) Firmware
6 3 (0.25%) Active Directory
6 3 (0.25%) Virtual Machines
6 3 (0.25%) vSphere
7 2 (0.17%) IAG
8 1 (0.084%) Apache Flume
8 1 (0.084%) KVM
8 1 (0.084%) Slurm Workload Manager
8 1 (0.084%) VMware ESXi
Systems Management
1 254 (21.29%) Kubernetes
2 177 (14.84%) Terraform
3 78 (6.54%) Ansible
4 73 (6.12%) Prometheus
5 43 (3.60%) Puppet
6 18 (1.51%) Grafana
7 17 (1.42%) HAProxy
8 8 (0.67%) Progress Chef
9 6 (0.50%) Docker Swarm
10 4 (0.34%) Microsoft Intune
11 3 (0.25%) Perfmon
11 3 (0.25%) Podman
12 2 (0.17%) Argo
12 2 (0.17%) Kibana
12 2 (0.17%) Kubeflow
12 2 (0.17%) Kustomize
13 1 (0.084%) HP ALM
13 1 (0.084%) Nagios
13 1 (0.084%) vCenter Server
13 1 (0.084%) vRealize
Vendors
1 204 (17.10%) Databricks
2 116 (9.72%) Microsoft
3 77 (6.45%) Google
4 38 (3.19%) Informatica
5 33 (2.77%) Oracle
6 28 (2.35%) Red Hat
6 28 (2.35%) SaltStack
7 23 (1.93%) Alteryx
7 23 (1.93%) Cloudera
8 20 (1.68%) Cognos
8 20 (1.68%) Dell
9 18 (1.51%) Sybase
10 14 (1.17%) VMware
11 12 (1.01%) Teradata
12 11 (0.92%) Salesforce
12 11 (0.92%) SAP
13 8 (0.67%) YouTube
14 7 (0.59%) Citrix
14 7 (0.59%) Sparx
15 5 (0.42%) Atlassian