Big Data
UK

The following table provides summary statistics for permanent job vacancies with a requirement for Big Data skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Big Data over the 6 months to 7 May 2024 with a comparison to the same period in the previous 2 years.

6 months to
7 May 2024
Same period 2023 Same period 2022
Rank 246 134 115
Rank change year-on-year -112 -19 +17
Permanent jobs citing Big Data 1,182 2,633 5,071
As % of all permanent jobs advertised in the UK 1.20% 2.59% 3.22%
As % of the Database & Business Intelligence category 6.55% 12.15% 11.53%
Number of salaries quoted 476 1,078 1,909
10th Percentile £43,750 £51,250 £46,150
25th Percentile £57,500 £62,500 £60,000
Median annual salary (50th Percentile) £80,000 £70,000 £80,000
Median % change year-on-year +14.29% -12.50% +23.08%
75th Percentile £110,000 £92,500 £95,000
90th Percentile £122,500 £112,500 £110,000
UK excluding London median annual salary £65,000 £70,000 £65,000
% change year-on-year -7.14% +7.69% +18.18%

All Database and Business Intelligence Skills
UK

Big Data is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for database or business intelligence skills.

Permanent vacancies with a requirement for database or business intelligence skills 18,050 21,669 43,969
As % of all permanent jobs advertised in the UK 18.33% 21.34% 27.93%
Number of salaries quoted 12,498 13,515 21,192
10th Percentile £35,000 £37,500 £36,250
25th Percentile £42,725 £47,500 £45,000
Median annual salary (50th Percentile) £57,500 £65,000 £60,000
Median % change year-on-year -11.54% +8.33% +9.09%
75th Percentile £76,250 £83,750 £79,000
90th Percentile £95,000 £102,500 £95,000
UK excluding London median annual salary £52,500 £55,000 £52,500
% change year-on-year -4.55% +4.76% +8.25%

Big Data
Job Vacancy Trend

Job postings citing Big Data as a proportion of all IT jobs advertised.

Job vacancy trend for Big Data in the UK

Big Data
Salary Trend

3-month moving average salary quoted in jobs citing Big Data.

Salary trend for Big Data in the UK

Big Data
Salary Histogram

Salary distribution for jobs citing Big Data over the 6 months to 7 May 2024.

Salary histogram for Big Data in the UK

Big Data
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Big Data within the UK over the 6 months to 7 May 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
UK excluding London -76 847 £65,000 -7.14% 100
England -251 576 £80,000 +14.29% 201
Scotland +81 561 £70,576 +0.82% 13
London -140 303 £95,000 +18.32% 114
Work from Home -192 291 £87,500 +25.00% 84
South West -56 100 £80,000 +14.29% 15
North of England -216 65 £55,000 -21.43% 18
North West -134 45 £55,000 -21.43% 11
South East -171 42 £54,500 -22.14% 18
East of England -43 37 £57,500 -11.54% 12
Midlands -123 27 £60,000 -9.77% 18
West Midlands -74 17 £61,250 +2.08% 16
North East -5 11 £42,500 -37.04% 2
Wales -8 10 £44,316 -36.69% 1
East Midlands -42 10 £60,000 -20.00% 2
Yorkshire -114 9 £87,500 +25.00% 5
Northern Ireland +29 5 £115,000 +31.43% 7

Big Data
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 718 (60.74%) Apache Spark
2 220 (18.61%) Apache
3 166 (14.04%) Apache Airflow
4 42 (3.55%) SAS
5 19 (1.61%) Jupyter
6 17 (1.44%) nginx
7 9 (0.76%) Apache Pig
8 6 (0.51%) OpenStack
9 5 (0.42%) Kinematics
10 3 (0.25%) Exchange Server 2013
10 3 (0.25%) Microsoft Exchange
11 2 (0.17%) Confluence
11 2 (0.17%) Luigi
12 1 (0.085%) CMS
Applications
1 32 (2.71%) Microsoft Excel
2 26 (2.20%) Microsoft Office
3 15 (1.27%) SPSS
4 6 (0.51%) Trello
5 5 (0.42%) Microsoft PowerPoint
6 4 (0.34%) MS Visio
7 1 (0.085%) Microsoft Project
7 1 (0.085%) Spreadsheet
Business Applications
1 7 (0.59%) Dynamics CRM
2 6 (0.51%) Payment Gateway
3 5 (0.42%) Sparx Enterprise Architect
4 4 (0.34%) Primavera
4 4 (0.34%) Relativity
5 3 (0.25%) SAP Business One
6 2 (0.17%) Veritas eDiscovery
7 1 (0.085%) Magento
7 1 (0.085%) Ringtail
Cloud Services
1 636 (53.81%) AWS
2 266 (22.50%) Azure
3 150 (12.69%) GCP
4 130 (11.00%) Serverless
5 96 (8.12%) AWS Lambda
6 81 (6.85%) Amazon EKS
6 81 (6.85%) Power Platform
7 80 (6.77%) Amazon SQS
8 79 (6.68%) Snowflake
9 78 (6.60%) SaaS
10 69 (5.84%) Amazon EMR
11 55 (4.65%) Amazon S3
12 53 (4.48%) Amazon EC2
13 52 (4.40%) Amazon ECS
13 52 (4.40%) OpenShift
14 50 (4.23%) dbt
15 45 (3.81%) Azure Synapse Analytics
16 41 (3.47%) Cloud Computing
17 39 (3.30%) Azure Data Factory
18 30 (2.54%) AWS CloudFormation
Communications & Networking
1 26 (2.20%) AMQP
2 23 (1.95%) Wireless
3 18 (1.52%) DSL
3 18 (1.52%) Firewall
4 13 (1.10%) Internet
5 5 (0.42%) Network Security
5 5 (0.42%) RFID
6 4 (0.34%) RS-232
6 4 (0.34%) SD-WAN
7 3 (0.25%) DNS
7 3 (0.25%) Fibre Channel
7 3 (0.25%) iSCSI
7 3 (0.25%) LAN
7 3 (0.25%) MQTT
7 3 (0.25%) SAN
7 3 (0.25%) VoIP
7 3 (0.25%) WAN
7 3 (0.25%) WebEx
8 2 (0.17%) 4G
8 2 (0.17%) Intrusion Detection
Database & Business Intelligence
1 385 (32.57%) NoSQL
2 332 (28.09%) Hadoop
3 177 (14.97%) Data Lake
4 127 (10.74%) Data Warehouse
5 96 (8.12%) Apache Hive
6 70 (5.92%) Power BI
7 61 (5.16%) Tableau
8 54 (4.57%) SQL Server
9 52 (4.40%) Amazon Athena
9 52 (4.40%) Electronic Patient Records
10 46 (3.89%) Relational Database
11 43 (3.64%) Dremio
12 42 (3.55%) Apache Arrow
12 42 (3.55%) Data Mining
13 41 (3.47%) MongoDB
14 40 (3.38%) HBase
15 35 (2.96%) Flink
16 34 (2.88%) Elasticsearch
17 28 (2.37%) Amazon Redshift
18 26 (2.20%) BigQuery
Development Applications
1 98 (8.29%) Git
2 82 (6.94%) Jenkins
3 37 (3.13%) Bitbucket Server
4 18 (1.52%) JIRA
5 17 (1.44%) Atlassian Bamboo
6 11 (0.93%) JUnit
7 10 (0.85%) CircleCI
7 10 (0.85%) SonarQube
8 8 (0.68%) Checkstyle
8 8 (0.68%) Oracle Forms
9 7 (0.59%) GitLab
10 6 (0.51%) Jasmine
10 6 (0.51%) Karma Test Runner
10 6 (0.51%) Visual Studio Code
11 5 (0.42%) MLflow
12 4 (0.34%) Cucumber
12 4 (0.34%) Octopus Deploy
12 4 (0.34%) Subversion
12 4 (0.34%) Visual Studio
13 3 (0.25%) Selenium
General
1 663 (56.09%) Finance
2 568 (48.05%) Inclusion and Diversity
3 515 (43.57%) Law
4 288 (24.37%) Marketing
5 260 (22.00%) Legal
6 181 (15.31%) Social Skills
7 165 (13.96%) Influencing Skills
8 113 (9.56%) Banking
9 104 (8.80%) Analytical Skills
10 79 (6.68%) Financial Institution
11 41 (3.47%) Aerospace
11 41 (3.47%) Retail
12 31 (2.62%) Presentation Skills
13 25 (2.12%) Investment Banking
14 19 (1.61%) Private Banking
15 17 (1.44%) Public Sector
16 15 (1.27%) Games
17 14 (1.18%) Retail Banking
18 12 (1.02%) Electronics
19 11 (0.93%) Pharmaceutical
Job Titles
1 494 (41.79%) Software Engineer
2 456 (38.58%) Big Data Engineer
3 390 (32.99%) Big Data Software Engineer
4 355 (30.03%) Data Engineer
5 209 (17.68%) Scala Engineer
5 209 (17.68%) Scala Software Engineer
6 193 (16.33%) Lead
7 190 (16.07%) AWS Engineer
8 188 (15.91%) Senior
9 181 (15.31%) Java Engineer
10 178 (15.06%) Java Software Engineer
11 150 (12.69%) Lead Big Data Engineer
12 149 (12.61%) Lead Software Engineer
13 144 (12.18%) AWS Software Engineer
14 119 (10.07%) Java Scala Software Engineer
15 109 (9.22%) Python Engineer
16 103 (8.71%) Architect
17 96 (8.12%) Lead Java Engineer
18 95 (8.04%) Python Software Engineer
19 72 (6.09%) Developer
Libraries, Frameworks & Software Standards
1 165 (13.96%) Kafka
2 132 (11.17%) PySpark
3 117 (9.90%) Spring
4 109 (9.22%) Spring Boot
5 73 (6.18%) gRPC
6 58 (4.91%) AngularJS
7 47 (3.98%) React
8 45 (3.81%) RabbitMQ
9 42 (3.55%) Protocol Buffers
10 38 (3.21%) Node.js
11 36 (3.05%) .NET
12 35 (2.96%) REST
13 32 (2.71%) OpenAPI
13 32 (2.71%) Swagger
14 28 (2.37%) Apache Beam
15 26 (2.20%) Apache NiFi
15 26 (2.20%) Material-UI
16 23 (1.95%) GraphQL
17 20 (1.69%) ActiveMQ
18 19 (1.61%) RESTful
Miscellaneous
1 260 (22.00%) Cloud Native
2 223 (18.87%) Algorithms
3 71 (6.01%) Data Structures
4 62 (5.25%) Management Information System
5 54 (4.57%) Product Ownership
5 54 (4.57%) Public Cloud
6 46 (3.89%) Hedge funds
7 43 (3.64%) Clustering
7 43 (3.64%) Legacy Code
8 42 (3.55%) Onboarding
9 31 (2.62%) Self-Motivation
10 25 (2.12%) Distributed Systems
11 22 (1.86%) IoT
12 21 (1.78%) Enterprise Software
13 19 (1.61%) Greenfield Project
13 19 (1.61%) Legacy Systems
14 18 (1.52%) Hybrid Cloud
15 15 (1.27%) Cybercrime
16 14 (1.18%) Cyber Defence
17 13 (1.10%) Robotics
Operating Systems
1 100 (8.46%) Linux
2 37 (3.13%) Unix
3 28 (2.37%) CentOS
3 28 (2.37%) Debian
3 28 (2.37%) Red Hat Enterprise Linux
3 28 (2.37%) Ubuntu
4 22 (1.86%) Windows
5 13 (1.10%) Windows Server
6 6 (0.51%) Mesosphere
7 4 (0.34%) FreeRTOS
8 3 (0.25%) Windows Server 2012
9 2 (0.17%) Android
9 2 (0.17%) Apple iOS
Processes & Methodologies
1 719 (60.83%) Agile
2 631 (53.38%) Software Engineering
3 546 (46.19%) CI/CD
4 504 (42.64%) SDLC
5 403 (34.09%) Operational Stability
6 352 (29.78%) Machine Learning
7 306 (25.89%) Artificial Intelligence
8 250 (21.15%) Risk Management
9 240 (20.30%) Visualisation
10 239 (20.22%) Problem-Solving
11 238 (20.14%) Performance Tuning
12 231 (19.54%) Community of Practice
13 212 (17.94%) Analytics
14 200 (16.92%) Data Modelling
15 199 (16.84%) Continuous Improvement
16 198 (16.75%) DevOps
17 188 (15.91%) Data Science
18 158 (13.37%) Mentoring
19 153 (12.94%) Business Intelligence
20 145 (12.27%) Decision-Making
Programming Languages
1 655 (55.41%) SQL
2 574 (48.56%) Java
3 562 (47.55%) Python
4 379 (32.06%) Scala
5 108 (9.14%) JavaScript
6 89 (7.53%) Go
7 56 (4.74%) C#
8 55 (4.65%) PL/SQL
9 53 (4.48%) R
10 43 (3.64%) Bash
11 19 (1.61%) Shell Script
11 19 (1.61%) T-SQL
12 16 (1.35%) C
13 13 (1.10%) C++
14 11 (0.93%) Data Analysis Expressions
14 11 (0.93%) TypeScript
15 8 (0.68%) Ruby
16 7 (0.59%) PHP
16 7 (0.59%) VB
17 5 (0.42%) XAML
Qualifications
1 200 (16.92%) Degree
2 96 (8.12%) Security Cleared
3 64 (5.41%) SC Cleared
4 57 (4.82%) DV Cleared
5 50 (4.23%) Computer Science Degree
6 48 (4.06%) Master's Degree
7 16 (1.35%) Microsoft Certification
8 15 (1.27%) AWS Certification
8 15 (1.27%) PhD
9 14 (1.18%) Azure Certification
10 10 (0.85%) AWS Certified Solutions Architect
10 10 (0.85%) DBS Check
11 8 (0.68%) BPSS Clearance
12 6 (0.51%) CISSP
12 6 (0.51%) IIBA
13 5 (0.42%) CCNA
13 5 (0.42%) Cisco Certification
14 4 (0.34%) CIMA
14 4 (0.34%) Network+ Certification
15 3 (0.25%) MBA
Quality Assurance & Compliance
1 51 (4.31%) Data Quality
2 34 (2.88%) SLA
3 26 (2.20%) QA
4 25 (2.12%) GDPR
5 16 (1.35%) Accessibility
6 6 (0.51%) WAI
7 4 (0.34%) ISO/IEC 27001
7 4 (0.34%) SEPA
8 1 (0.085%) GMP
8 1 (0.085%) PMO
System Software
1 92 (7.78%) Docker
2 56 (4.74%) HDFS
3 9 (0.76%) VMware Infrastructure
4 6 (0.51%) Hyper-V
5 5 (0.42%) Firmware
6 3 (0.25%) Active Directory
6 3 (0.25%) Virtual Machines
6 3 (0.25%) vSphere
7 2 (0.17%) IAG
8 1 (0.085%) Apache Flume
8 1 (0.085%) KVM
8 1 (0.085%) Slurm Workload Manager
8 1 (0.085%) VMware ESXi
Systems Management
1 251 (21.24%) Kubernetes
2 162 (13.71%) Terraform
3 74 (6.26%) Ansible
4 61 (5.16%) Prometheus
5 41 (3.47%) Puppet
6 18 (1.52%) Grafana
7 17 (1.44%) HAProxy
8 9 (0.76%) Progress Chef
9 6 (0.51%) Docker Swarm
10 4 (0.34%) Microsoft Intune
11 3 (0.25%) Perfmon
11 3 (0.25%) Podman
12 2 (0.17%) Argo
12 2 (0.17%) Kibana
12 2 (0.17%) Kubeflow
12 2 (0.17%) Kustomize
13 1 (0.085%) HP ALM
13 1 (0.085%) Rancher
13 1 (0.085%) vCenter Server
13 1 (0.085%) vRealize
Vendors
1 200 (16.92%) Databricks
2 112 (9.48%) Microsoft
3 73 (6.18%) Google
4 38 (3.21%) Informatica
5 31 (2.62%) Oracle
6 26 (2.20%) Red Hat
6 26 (2.20%) SaltStack
7 24 (2.03%) Dell
8 22 (1.86%) Alteryx
8 22 (1.86%) Cloudera
9 20 (1.69%) Cognos
10 18 (1.52%) Sybase
11 16 (1.35%) VMware
12 12 (1.02%) Teradata
13 11 (0.93%) Salesforce
13 11 (0.93%) SAP
14 8 (0.68%) YouTube
15 7 (0.59%) Atlassian
15 7 (0.59%) Citrix
16 6 (0.51%) Sparx