Data Lakes
UK

The following table provides summary statistics for permanent job vacancies with a requirement for Data Lake skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Lake over the 6 months to 28 May 2024 with a comparison to the same period in the previous 2 years.

6 months to
28 May 2024
Same period 2023 Same period 2022
Rank 266 235 293
Rank change year-on-year -31 +58 +68
Permanent jobs citing Data Lake 1,109 1,502 2,359
As % of all permanent jobs advertised in the UK 1.08% 1.54% 1.41%
As % of the Database & Business Intelligence category 6.05% 7.15% 5.01%
Number of salaries quoted 850 1,015 1,134
10th Percentile £42,500 £50,000 £46,250
25th Percentile £60,375 £58,750 £54,000
Median annual salary (50th Percentile) £87,500 £80,000 £70,000
Median % change year-on-year +9.38% +14.29% -
75th Percentile £122,500 £100,000 £87,500
90th Percentile £148,750 £130,000 £102,500
UK excluding London median annual salary £60,000 £60,000 £60,000

All Database and Business Intelligence Skills
UK

Data Lake is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for database or business intelligence skills.

Permanent vacancies with a requirement for database or business intelligence skills 18,328 21,007 47,066
As % of all permanent jobs advertised in the UK 17.86% 21.58% 28.19%
Number of salaries quoted 12,523 13,073 21,105
10th Percentile £35,000 £37,500 £36,250
25th Percentile £42,725 £47,500 £45,000
Median annual salary (50th Percentile) £57,500 £65,000 £60,000
Median % change year-on-year -11.54% +8.33% +9.09%
75th Percentile £77,500 £82,500 £80,000
90th Percentile £97,500 £103,750 £95,000
UK excluding London median annual salary £52,500 £55,000 £52,500
% change year-on-year -4.55% +4.76% +10.53%

Data Lake
Job Vacancy Trend

Job postings citing Data Lake as a proportion of all IT jobs advertised.

Job vacancy trend for Data Lake in the UK

Data Lake
Salary Trend

3-month moving average salary quoted in jobs citing Data Lake.

Salary trend for Data Lake in the UK

Data Lake
Salary Histogram

Salary distribution for jobs citing Data Lake over the 6 months to 28 May 2024.

Salary histogram for Data Lake in the UK

Data Lake
Top 18 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Lake within the UK over the 6 months to 28 May 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
England -14 968 £90,000 +12.50% 188
Work from Home +42 675 £110,000 +37.00% 96
London +29 593 £115,000 +27.78% 109
UK excluding London -75 463 £60,000 - 78
North of England -3 123 £55,000 -4.35% 31
South East -41 99 £63,500 -9.29% 8
Scotland -35 82 £66,500 +2.31% 7
Midlands -13 75 £57,500 +6.24% 10
North West -15 70 £45,000 -18.18% 26
South West -70 53 £70,576 +4.56% 16
West Midlands +9 49 £57,500 +7.98% 6
Yorkshire +43 37 £65,000 -2.86% 4
East Midlands -24 26 £47,500 -13.64% 4
North East -7 16 £75,000 +36.36% 1
East of England -21 15 £57,500 +6.98% 5
Wales -8 8 £46,131 -20.46% 1
Northern Ireland +12 6 - - 1
Isle of Man - 2 £82,500 -

Data Lake
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 331 (29.85%) Apache Airflow
2 293 (26.42%) Apache
3 102 (9.20%) Apache Spark
4 16 (1.44%) Confluence
5 7 (0.63%) SharePoint
6 5 (0.45%) Microsoft Exchange
7 2 (0.18%) IBM Domino
7 2 (0.18%) Jupyter
8 1 (0.090%) Apache Pig
8 1 (0.090%) Apache Synapse
8 1 (0.090%) IIS
8 1 (0.090%) SAS
Applications
1 19 (1.71%) Microsoft Excel
1 19 (1.71%) Microsoft Office
2 5 (0.45%) Microsoft PowerPoint
3 1 (0.090%) MS Visio
Business Applications
1 23 (2.07%) Dynamics CRM
2 9 (0.81%) Dynamics NAV
3 6 (0.54%) Dynamics AX
3 6 (0.54%) Primavera
4 3 (0.27%) SAP S/4HANA
5 1 (0.090%) HFM
5 1 (0.090%) Magento
5 1 (0.090%) Salesforce CRM
Cloud Services
1 633 (57.08%) Azure
2 568 (51.22%) AWS
3 391 (35.26%) Snowflake
4 330 (29.76%) dbt
5 290 (26.15%) Power Platform
6 271 (24.44%) GCP
7 189 (17.04%) Azure Data Factory
8 159 (14.34%) Azure Synapse Analytics
9 58 (5.23%) Azure Cosmos DB
10 54 (4.87%) Matillion
11 51 (4.60%) Azure DevOps
12 46 (4.15%) PowerApps
12 46 (4.15%) Serverless
13 42 (3.79%) Dynamics 365
14 27 (2.43%) Power Automate
15 22 (1.98%) Cloud Computing
15 22 (1.98%) GitHub
15 22 (1.98%) SaaS
16 17 (1.53%) Microsoft 365
17 16 (1.44%) Amazon S3
Communications & Networking
1 17 (1.53%) DSL
2 16 (1.44%) Wireless
3 11 (0.99%) Network Security
4 4 (0.36%) DNS
4 4 (0.36%) Firewall
5 3 (0.27%) NetScaler
6 1 (0.090%) HTTP
6 1 (0.090%) SAN
Database & Business Intelligence
1 393 (35.44%) Data Warehouse
2 323 (29.13%) Dremio
3 314 (28.31%) Apache Arrow
4 313 (28.22%) Amazon Athena
5 281 (25.34%) Power BI
6 263 (23.72%) Apache Ignite
7 191 (17.22%) Big Data
8 176 (15.87%) SQL Server
9 79 (7.12%) Tableau
10 76 (6.85%) Azure SQL Database
11 69 (6.22%) Azure SQL Data Warehouse
12 65 (5.86%) Metadata
13 62 (5.59%) NoSQL
14 61 (5.50%) SQL Server Integration Services
15 50 (4.51%) Star Schema
16 46 (4.15%) Qlik Sense
17 42 (3.79%) Elasticsearch
18 41 (3.70%) Relational Database
19 40 (3.61%) Hadoop
20 38 (3.43%) SQL Server Analysis Services
Development Applications
1 39 (3.52%) GitLab
2 25 (2.25%) JIRA
3 23 (2.07%) Git
4 17 (1.53%) Jenkins
5 15 (1.35%) CircleCI
6 12 (1.08%) MLflow
7 10 (0.90%) Selenium
8 9 (0.81%) JUnit
8 9 (0.81%) Oracle Forms
9 8 (0.72%) Cucumber
9 8 (0.72%) SQL Server Management Studio (SSMS)
10 6 (0.54%) RStudio
11 5 (0.45%) Visual Studio
12 4 (0.36%) Octopus Deploy
12 4 (0.36%) Zephyr
13 3 (0.27%) ER Studio
13 3 (0.27%) ERwin
14 2 (0.18%) Bitbucket
15 1 (0.090%) Gradle
15 1 (0.090%) SoapUI
General
1 599 (54.01%) Finance
2 340 (30.66%) Banking
3 280 (25.25%) Investment Banking
4 266 (23.99%) Front Office
5 179 (16.14%) Social Skills
6 141 (12.71%) Analytical Skills
7 100 (9.02%) Inclusion and Diversity
8 88 (7.94%) Law
9 70 (6.31%) Legal
10 57 (5.14%) Marketing
11 53 (4.78%) Presentation Skills
12 35 (3.16%) Retail
13 34 (3.07%) Financial Institution
14 23 (2.07%) Retail Banking
15 16 (1.44%) Billing
15 16 (1.44%) Public Sector
16 15 (1.35%) Aerospace
17 13 (1.17%) Back Office
17 13 (1.17%) Influencing Skills
17 13 (1.17%) Telecoms
Job Titles
1 359 (32.37%) Data Engineer
2 272 (24.53%) Senior
3 209 (18.85%) Architect
4 193 (17.40%) Software Engineer
5 192 (17.31%) Lead
6 152 (13.71%) AWS Engineer
7 125 (11.27%) Developer
8 118 (10.64%) Python Engineer
9 104 (9.38%) Data Architect
10 92 (8.30%) AWS Software Engineer
11 91 (8.21%) Python Software Engineer
12 90 (8.12%) Java Engineer
12 90 (8.12%) Java Software Engineer
13 86 (7.75%) Senior Data Engineer
13 86 (7.75%) Senior Software Engineer
14 85 (7.66%) Solutions Architect
15 79 (7.12%) Azure Engineer
16 74 (6.67%) Azure Data Engineer
17 70 (6.31%) Consultant
18 63 (5.68%) Technical Leader
Libraries, Frameworks & Software Standards
1 306 (27.59%) gRPC
1 306 (27.59%) Protocol Buffers
2 273 (24.62%) Pandas
3 258 (23.26%) NumPy
3 258 (23.26%) SciPy
4 120 (10.82%) PySpark
5 49 (4.42%) Kafka
6 17 (1.53%) Apache Beam
7 16 (1.44%) JSON
8 14 (1.26%) .NET
8 14 (1.26%) RESTful
8 14 (1.26%) XML
9 9 (0.81%) MDX
10 7 (0.63%) Apache Avro
10 7 (0.63%) OpenID
10 7 (0.63%) Spark Streaming
11 6 (0.54%) REST
11 6 (0.54%) Web Services
12 5 (0.45%) CSS
12 5 (0.45%) React
Miscellaneous
1 314 (28.31%) Hedge funds
2 260 (23.44%) Derivative
2 260 (23.44%) Equities
2 260 (23.44%) Fixed Income
2 260 (23.44%) Foreign Exchange (FX)
3 80 (7.21%) Algorithms
4 62 (5.59%) Data Structures
5 55 (4.96%) Clustering
6 50 (4.51%) Product Ownership
7 49 (4.42%) Cloud Native
8 44 (3.97%) Self-Motivation
9 42 (3.79%) Onboarding
10 36 (3.25%) Pricing Data
11 34 (3.07%) Management Information System
12 27 (2.43%) Greenfield Project
13 23 (2.07%) Public Cloud
14 22 (1.98%) Enterprise Software
15 17 (1.53%) NHS
16 16 (1.44%) Legacy Applications
17 14 (1.26%) Data Centre
Operating Systems
1 26 (2.34%) Linux
2 16 (1.44%) Unix
3 7 (0.63%) Windows
4 3 (0.27%) Windows Server
5 1 (0.090%) AIX
5 1 (0.090%) Android
5 1 (0.090%) Solaris
5 1 (0.090%) VMS
Processes & Methodologies
1 541 (48.78%) Agile
2 363 (32.73%) Asset Management
3 352 (31.74%) Software Engineering
4 314 (28.31%) Analytics
5 312 (28.13%) Business Intelligence
6 280 (25.25%) Investment Management
7 258 (23.26%) ETL
8 251 (22.63%) Data Modelling
9 248 (22.36%) Data Engineering
10 241 (21.73%) Data Pipeline
11 223 (20.11%) Machine Learning
12 220 (19.84%) Artificial Intelligence
13 179 (16.14%) CI/CD
14 175 (15.78%) DevOps
15 164 (14.79%) Roadmaps
16 161 (14.52%) Visualisation
17 152 (13.71%) Data Architecture
18 139 (12.53%) Data Strategy
19 133 (11.99%) Data Science
20 116 (10.46%) Continuous Improvement
Programming Languages
1 642 (57.89%) Python
2 495 (44.63%) SQL
3 425 (38.32%) Java
4 315 (28.40%) JavaScript
5 254 (22.90%) TypeScript
6 100 (9.02%) T-SQL
7 64 (5.77%) Go
8 55 (4.96%) Scala
9 47 (4.24%) Data Analysis Expressions
10 30 (2.71%) C#
11 25 (2.25%) R
12 20 (1.80%) PL/SQL
13 11 (0.99%) U-SQL
14 10 (0.90%) Kotlin
15 8 (0.72%) PowerShell
16 4 (0.36%) C
17 3 (0.27%) Bash
18 2 (0.18%) Shell Script
18 2 (0.18%) VBA
19 1 (0.090%) C++
Qualifications
1 89 (8.03%) Azure Certification
2 87 (7.84%) Degree
3 66 (5.95%) AWS Certification
4 41 (3.70%) Microsoft Certification
5 24 (2.16%) Computer Science Degree
6 14 (1.26%) SC Cleared
6 14 (1.26%) Security Cleared
7 13 (1.17%) Master's Degree
8 12 (1.08%) DBS Check
9 8 (0.72%) DV Cleared
10 6 (0.54%) CIMA
11 3 (0.27%) MCP
11 3 (0.27%) MCSE
12 2 (0.18%) MCSA
12 2 (0.18%) Postgraduate
13 1 (0.090%) IBM Certification
13 1 (0.090%) LPIC
13 1 (0.090%) Numerate Degree
Quality Assurance & Compliance
1 252 (22.72%) Data Quality
2 54 (4.87%) GDPR
3 19 (1.71%) SLA
4 13 (1.17%) QA
5 8 (0.72%) PSD2
6 7 (0.63%) Accessibility
7 6 (0.54%) California Consumer Privacy Act
8 3 (0.27%) PMO
9 1 (0.090%) HIPAA
9 1 (0.090%) ISO/IEC 27001
System Software
1 24 (2.16%) Docker
2 7 (0.63%) Active Directory
2 7 (0.63%) HDFS
3 1 (0.090%) Apache Flume
3 1 (0.090%) VMware Infrastructure
Systems Management
1 114 (10.28%) Kubernetes
2 83 (7.48%) Terraform
3 54 (4.87%) Prometheus
4 3 (0.27%) HP Fortify
5 2 (0.18%) Ansible
5 2 (0.18%) Argo
5 2 (0.18%) Progress Chef
Vendors
1 284 (25.61%) Databricks
2 266 (23.99%) Microsoft
3 71 (6.40%) Alteryx
4 55 (4.96%) Google
5 45 (4.06%) Oracle
6 21 (1.89%) Cognos
7 17 (1.53%) Sybase
8 15 (1.35%) SAP
9 13 (1.17%) Teradata
10 10 (0.90%) Cloudera
11 8 (0.72%) Duck Creek
12 7 (0.63%) Adobe
13 6 (0.54%) Apple
13 6 (0.54%) Informatica
13 6 (0.54%) Talend
14 4 (0.36%) OpenAI
15 3 (0.27%) Citrix
15 3 (0.27%) IBM
15 3 (0.27%) SnapLogic
15 3 (0.27%) Veracode