Data Lakes
UK

The following table provides summary statistics for permanent job vacancies with a requirement for Data Lake skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Lake over the 6 months to 6 May 2024 with a comparison to the same period in the previous 2 years.

6 months to
6 May 2024
Same period 2023 Same period 2022
Rank 296 260 303
Rank change year-on-year -36 +43 +44
Permanent jobs citing Data Lake 972 1,446 2,147
As % of all permanent jobs advertised in the UK 0.98% 1.40% 1.36%
As % of the Database & Business Intelligence category 5.35% 6.55% 4.90%
Number of salaries quoted 759 1,046 1,131
10th Percentile £42,500 £50,000 £46,250
25th Percentile £58,125 £58,750 £54,500
Median annual salary (50th Percentile) £82,500 £80,000 £70,000
Median % change year-on-year +3.13% +14.29% -
75th Percentile £118,750 £100,000 £86,250
90th Percentile £148,750 £130,000 £102,500
UK excluding London median annual salary £60,000 £60,000 £60,000

All Database and Business Intelligence Skills
UK

Data Lake is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for database or business intelligence skills.

Permanent vacancies with a requirement for database or business intelligence skills 18,174 22,077 43,855
As % of all permanent jobs advertised in the UK 18.40% 21.35% 27.86%
Number of salaries quoted 12,615 13,788 21,399
10th Percentile £35,000 £37,500 £36,250
25th Percentile £42,725 £47,500 £45,000
Median annual salary (50th Percentile) £57,500 £65,000 £60,000
Median % change year-on-year -11.54% +8.33% +9.09%
75th Percentile £76,250 £83,750 £80,000
90th Percentile £95,000 £102,500 £95,000
UK excluding London median annual salary £52,500 £55,000 £52,500
% change year-on-year -4.55% +4.76% +7.14%

Data Lake
Job Vacancy Trend

Job postings citing Data Lake as a proportion of all IT jobs advertised.

Job vacancy trend for Data Lake in the UK

Data Lake
Salary Trend

3-month moving average salary quoted in jobs citing Data Lake.

Salary trend for Data Lake in the UK

Data Lake
Salary Histogram

Salary distribution for jobs citing Data Lake over the 6 months to 6 May 2024.

Salary histogram for Data Lake in the UK

Data Lake
Top 18 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Lake within the UK over the 6 months to 6 May 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
England -23 844 £85,000 +6.25% 207
Work from Home +12 542 £90,000 +12.09% 123
London +8 476 £110,000 +22.22% 119
UK excluding London -61 440 £60,000 - 100
North of England +19 120 £55,000 -6.38% 26
South East -20 98 £65,000 -6.47% 23
Scotland -30 80 £74,500 +10.82% 11
Midlands -5 69 £55,000 - 14
North West +4 65 £45,000 -23.15% 17
South West -73 49 £72,500 +7.41% 12
West Midlands +10 47 £57,500 +6.24% 9
Yorkshire +66 37 £70,000 +7.69% 9
East Midlands -18 22 £47,500 -13.64% 5
North East -5 18 £72,500 +31.82%
East of England -2 15 £57,500 - 8
Wales +8 5 £46,131 -20.46% 5
Northern Ireland +20 2 - - 3
Isle of Man - 2 £82,500 -

Data Lake
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 230 (23.66%) Apache Airflow
2 200 (20.58%) Apache
3 110 (11.32%) Apache Spark
4 15 (1.54%) Confluence
5 9 (0.93%) SharePoint
6 6 (0.62%) Microsoft Exchange
7 3 (0.31%) IBM Domino
8 2 (0.21%) Jupyter
9 1 (0.10%) Apache Pig
9 1 (0.10%) Apache Synapse
9 1 (0.10%) IIS
9 1 (0.10%) SAS
Applications
1 18 (1.85%) Microsoft Office
2 17 (1.75%) Microsoft Excel
3 5 (0.51%) Microsoft PowerPoint
4 1 (0.10%) MS Visio
Business Applications
1 20 (2.06%) Dynamics CRM
2 8 (0.82%) Dynamics NAV
3 6 (0.62%) Dynamics AX
3 6 (0.62%) Primavera
4 3 (0.31%) SAP S/4HANA
5 2 (0.21%) Magento
6 1 (0.10%) HFM
6 1 (0.10%) Salesforce CRM
Cloud Services
1 615 (63.27%) Azure
2 457 (47.02%) AWS
3 296 (30.45%) Snowflake
4 275 (28.29%) Power Platform
5 260 (26.75%) GCP
6 221 (22.74%) dbt
7 190 (19.55%) Azure Data Factory
8 148 (15.23%) Azure Synapse Analytics
9 57 (5.86%) Matillion
10 56 (5.76%) Azure Cosmos DB
11 51 (5.25%) Azure DevOps
12 49 (5.04%) Serverless
13 46 (4.73%) PowerApps
14 42 (4.32%) Dynamics 365
15 35 (3.60%) SaaS
16 30 (3.09%) PaaS
17 29 (2.98%) IaaS
18 26 (2.67%) Power Automate
19 23 (2.37%) GitHub
20 22 (2.26%) Cloud Computing
Communications & Networking
1 20 (2.06%) Wireless
2 18 (1.85%) DSL
3 9 (0.93%) Network Security
4 6 (0.62%) DNS
5 4 (0.41%) Firewall
6 3 (0.31%) NetScaler
7 1 (0.10%) HTTP
7 1 (0.10%) SAN
Database & Business Intelligence
1 388 (39.92%) Data Warehouse
2 266 (27.37%) Power BI
3 212 (21.81%) Dremio
4 209 (21.50%) Amazon Athena
5 208 (21.40%) Apache Arrow
6 181 (18.62%) Apache Ignite
7 177 (18.21%) Big Data
7 177 (18.21%) SQL Server
8 78 (8.02%) NoSQL
9 75 (7.72%) Azure SQL Database
10 72 (7.41%) Tableau
11 63 (6.48%) Metadata
12 62 (6.38%) SQL Server Integration Services
13 51 (5.25%) Star Schema
14 46 (4.73%) Elasticsearch
14 46 (4.73%) Qlik Sense
15 45 (4.63%) Azure SQL Data Warehouse
15 45 (4.63%) Hadoop
16 44 (4.53%) Relational Database
17 37 (3.81%) SQL Server Analysis Services
Development Applications
1 39 (4.01%) GitLab
2 25 (2.57%) JIRA
3 23 (2.37%) Git
4 17 (1.75%) Jenkins
5 16 (1.65%) CircleCI
6 10 (1.03%) Selenium
7 9 (0.93%) JUnit
7 9 (0.93%) MLflow
7 9 (0.93%) Oracle Forms
8 8 (0.82%) Cucumber
8 8 (0.82%) SQL Server Management Studio (SSMS)
8 8 (0.82%) Visual Studio
9 6 (0.62%) RStudio
10 5 (0.51%) ERwin
11 4 (0.41%) Zephyr
12 3 (0.31%) ER Studio
12 3 (0.31%) Octopus Deploy
13 2 (0.21%) Bitbucket
14 1 (0.10%) Gradle
14 1 (0.10%) SpecFlow
General
1 470 (48.35%) Finance
2 248 (25.51%) Banking
3 186 (19.14%) Investment Banking
4 177 (18.21%) Social Skills
5 166 (17.08%) Front Office
6 146 (15.02%) Analytical Skills
7 98 (10.08%) Inclusion and Diversity
8 90 (9.26%) Law
9 66 (6.79%) Legal
10 55 (5.66%) Marketing
11 38 (3.91%) Retail
12 33 (3.40%) Financial Institution
13 29 (2.98%) Presentation Skills
14 21 (2.16%) Retail Banking
15 19 (1.95%) Public Sector
16 17 (1.75%) Billing
17 13 (1.34%) Telecoms
18 12 (1.23%) Back Office
18 12 (1.23%) Games
18 12 (1.23%) Influencing Skills
Job Titles
1 348 (35.80%) Data Engineer
2 211 (21.71%) Senior
3 204 (20.99%) Architect
4 156 (16.05%) Lead
5 141 (14.51%) Software Engineer
6 113 (11.63%) Developer
7 109 (11.21%) Data Architect
8 101 (10.39%) AWS Engineer
9 85 (8.74%) Senior Data Engineer
10 75 (7.72%) Solutions Architect
11 74 (7.61%) Python Engineer
12 71 (7.30%) Azure Engineer
13 65 (6.69%) Azure Data Engineer
14 56 (5.76%) Python Software Engineer
15 55 (5.66%) Java Engineer
15 55 (5.66%) Java Software Engineer
16 52 (5.35%) AWS Software Engineer
17 50 (5.14%) Consultant
17 50 (5.14%) Senior Software Engineer
18 44 (4.53%) Lead Software Engineer
Libraries, Frameworks & Software Standards
1 208 (21.40%) gRPC
1 208 (21.40%) Protocol Buffers
2 181 (18.62%) Pandas
3 166 (17.08%) NumPy
3 166 (17.08%) SciPy
4 107 (11.01%) PySpark
5 55 (5.66%) Kafka
6 19 (1.95%) Apache Beam
6 19 (1.95%) OpenID
7 16 (1.65%) JSON
8 14 (1.44%) RESTful
9 13 (1.34%) .NET
9 13 (1.34%) XML
10 9 (0.93%) MDX
11 8 (0.82%) Apache Avro
11 8 (0.82%) Spark Streaming
12 6 (0.62%) REST
12 6 (0.62%) Web Services
13 4 (0.41%) SOAP
13 4 (0.41%) YAML
Miscellaneous
1 208 (21.40%) Hedge funds
2 166 (17.08%) Derivative
2 166 (17.08%) Equities
2 166 (17.08%) Fixed Income
2 166 (17.08%) Foreign Exchange (FX)
3 77 (7.92%) Algorithms
4 64 (6.58%) Data Structures
5 43 (4.42%) Clustering
5 43 (4.42%) Self-Motivation
6 42 (4.32%) Onboarding
7 41 (4.22%) Product Ownership
8 36 (3.70%) Cloud Native
9 35 (3.60%) Management Information System
10 28 (2.88%) Greenfield Project
11 24 (2.47%) Public Cloud
12 20 (2.06%) Blockchain
12 20 (2.06%) Enterprise Software
13 19 (1.95%) Mainframe
14 18 (1.85%) Pricing Data
15 14 (1.44%) Data Centre
Operating Systems
1 27 (2.78%) Unix
2 11 (1.13%) Linux
3 6 (0.62%) Windows
4 3 (0.31%) Windows Server
5 1 (0.10%) AIX
5 1 (0.10%) Solaris
5 1 (0.10%) VMS
Processes & Methodologies
1 442 (45.47%) Agile
2 312 (32.10%) Business Intelligence
3 305 (31.38%) Analytics
4 269 (27.67%) ETL
5 259 (26.65%) Software Engineering
6 256 (26.34%) Data Modelling
7 250 (25.72%) Asset Management
8 249 (25.62%) Data Pipeline
9 235 (24.18%) Data Engineering
10 233 (23.97%) Machine Learning
11 206 (21.19%) Artificial Intelligence
12 188 (19.34%) Investment Management
13 179 (18.42%) CI/CD
14 161 (16.56%) DevOps
15 159 (16.36%) Visualisation
16 143 (14.71%) Data Strategy
17 140 (14.40%) Data Science
18 133 (13.68%) Roadmaps
19 123 (12.65%) Data Architecture
20 113 (11.63%) Continuous Improvement
Programming Languages
1 528 (54.32%) Python
2 485 (49.90%) SQL
3 322 (33.13%) Java
4 212 (21.81%) JavaScript
5 168 (17.28%) TypeScript
6 99 (10.19%) T-SQL
7 56 (5.76%) Scala
8 52 (5.35%) Go
9 45 (4.63%) Data Analysis Expressions
10 29 (2.98%) C#
11 25 (2.57%) R
12 21 (2.16%) PL/SQL
13 10 (1.03%) Kotlin
13 10 (1.03%) PowerShell
14 9 (0.93%) U-SQL
15 4 (0.41%) C
16 3 (0.31%) Bash
17 2 (0.21%) C++
18 1 (0.10%) Shell Script
18 1 (0.10%) VBA
Qualifications
1 85 (8.74%) Degree
2 61 (6.28%) Azure Certification
3 43 (4.42%) Microsoft Certification
4 40 (4.12%) AWS Certification
5 18 (1.85%) Computer Science Degree
6 13 (1.34%) SC Cleared
6 13 (1.34%) Security Cleared
7 11 (1.13%) DBS Check
7 11 (1.13%) Master's Degree
8 8 (0.82%) DV Cleared
9 6 (0.62%) CIMA
10 3 (0.31%) MCP
10 3 (0.31%) MCSE
11 2 (0.21%) MCSA
11 2 (0.21%) Postgraduate
12 1 (0.10%) AWS Certified Solutions Architect
12 1 (0.10%) HND
12 1 (0.10%) IBM Certification
12 1 (0.10%) LPIC
Quality Assurance & Compliance
1 198 (20.37%) Data Quality
2 53 (5.45%) GDPR
3 20 (2.06%) SLA
4 13 (1.34%) QA
5 8 (0.82%) PSD2
6 6 (0.62%) Accessibility
6 6 (0.62%) California Consumer Privacy Act
7 3 (0.31%) PMO
8 1 (0.10%) GMP
8 1 (0.10%) HIPAA
8 1 (0.10%) ISO/IEC 27001
System Software
1 27 (2.78%) Docker
2 8 (0.82%) HDFS
3 7 (0.72%) Active Directory
4 1 (0.10%) Apache Flume
4 1 (0.10%) Virtual Machines
4 1 (0.10%) VMware Infrastructure
Systems Management
1 104 (10.70%) Kubernetes
2 70 (7.20%) Terraform
3 42 (4.32%) Prometheus
4 3 (0.31%) HP Fortify
5 2 (0.21%) Argo
5 2 (0.21%) Progress Chef
6 1 (0.10%) Ansible
Vendors
1 287 (29.53%) Databricks
2 272 (27.98%) Microsoft
3 71 (7.30%) Alteryx
4 58 (5.97%) Google
5 44 (4.53%) Oracle
6 22 (2.26%) Cognos
7 18 (1.85%) Sybase
8 16 (1.65%) SAP
9 13 (1.34%) Teradata
10 11 (1.13%) Cloudera
11 8 (0.82%) Duck Creek
11 8 (0.82%) Talend
12 7 (0.72%) Adobe
12 7 (0.72%) Informatica
13 6 (0.62%) Apple
14 5 (0.51%) IBM
15 4 (0.41%) OpenAI
15 4 (0.41%) SnapLogic
16 3 (0.31%) Citrix
16 3 (0.31%) Veracode