Data Lakes
UK

The following table provides summary statistics for permanent job vacancies with a requirement for Data Lake skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Lake over the 6 months to 23 April 2024 with a comparison to the same period in the previous 2 years.

6 months to
23 Apr 2024
Same period 2023 Same period 2022
Rank 298 261 297
Rank change year-on-year -37 +36 +48
Permanent jobs citing Data Lake 934 1,478 2,090
As % of all permanent jobs advertised in the UK 0.96% 1.40% 1.36%
As % of the Database & Business Intelligence category 5.17% 6.45% 4.92%
Number of salaries quoted 708 1,065 1,123
10th Percentile £42,250 £50,844 £46,250
25th Percentile £57,500 £59,999 £55,000
Median annual salary (50th Percentile) £77,500 £80,000 £70,000
Median % change year-on-year -3.13% +14.29% +3.70%
75th Percentile £110,000 £100,000 £85,625
90th Percentile £143,750 £130,000 £102,500
UK excluding London median annual salary £62,500 £62,500 £60,000
% change year-on-year - +4.17% -

All Database and Business Intelligence Skills
UK

Data Lake is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for database or business intelligence skills.

Permanent vacancies with a requirement for database or business intelligence skills 18,054 22,905 42,485
As % of all permanent jobs advertised in the UK 18.47% 21.67% 27.60%
Number of salaries quoted 12,611 14,133 21,464
10th Percentile £35,000 £37,500 £36,250
25th Percentile £42,725 £47,500 £45,000
Median annual salary (50th Percentile) £57,500 £65,000 £60,000
Median % change year-on-year -11.54% +8.33% +9.09%
75th Percentile £76,250 £83,750 £78,750
90th Percentile £95,000 £103,750 £95,000
UK excluding London median annual salary £52,500 £55,000 £52,000
% change year-on-year -4.55% +5.77% +6.12%

Data Lake
Job Vacancy Trend

Job postings citing Data Lake as a proportion of all IT jobs advertised.

Job vacancy trend for Data Lake in the UK

Data Lake
Salary Trend

3-month moving average salary quoted in jobs citing Data Lake.

Salary trend for Data Lake in the UK

Data Lake
Salary Histogram

Salary distribution for jobs citing Data Lake over the 6 months to 23 April 2024.

Salary histogram for Data Lake in the UK

Data Lake
Top 18 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Lake within the UK over the 6 months to 23 April 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
England -40 813 £78,613 -1.73% 168
Work from Home -6 498 £87,500 +8.98% 94
UK excluding London -60 442 £62,500 - 77
London -4 438 £110,000 +22.22% 96
North of England +1 116 £55,000 -15.38% 23
South East -9 104 £70,000 - 19
Scotland -35 76 £82,500 +22.72% 10
Midlands -10 68 £47,500 -13.64% 10
North West -13 61 £45,000 -28.00% 17
South West -48 54 £72,500 +7.41% 6
West Midlands +2 44 £56,250 +3.93% 6
Yorkshire +54 37 £70,000 +1.71% 6
East Midlands -15 24 £47,500 -13.64% 4
North East +7 18 £72,500 +31.82%
East of England +13 15 £57,500 - 5
Wales -5 5 £46,131 -20.46% 5
Northern Ireland +18 2 - -
Isle of Man - 2 £82,500 -

Data Lake
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 173 (18.52%) Apache Airflow
2 168 (17.99%) Apache
3 128 (13.70%) Apache Spark
4 15 (1.61%) Confluence
5 11 (1.18%) SharePoint
6 8 (0.86%) Microsoft Exchange
7 3 (0.32%) IBM Domino
7 3 (0.32%) SAS
8 2 (0.21%) Jupyter
9 1 (0.11%) Apache Pig
9 1 (0.11%) Apache Synapse
9 1 (0.11%) IIS
Applications
1 17 (1.82%) Microsoft Excel
1 17 (1.82%) Microsoft Office
2 5 (0.54%) Microsoft PowerPoint
3 1 (0.11%) MS Visio
Business Applications
1 21 (2.25%) Dynamics CRM
2 9 (0.96%) Dynamics NAV
3 7 (0.75%) Dynamics AX
4 6 (0.64%) Primavera
5 3 (0.32%) SAP S/4HANA
6 2 (0.21%) Magento
7 1 (0.11%) HFM
7 1 (0.11%) Salesforce CRM
Cloud Services
1 621 (66.49%) Azure
2 400 (42.83%) AWS
3 288 (30.84%) Power Platform
4 249 (26.66%) GCP
5 240 (25.70%) Snowflake
6 185 (19.81%) Azure Data Factory
7 165 (17.67%) dbt
8 147 (15.74%) Azure Synapse Analytics
9 59 (6.32%) Matillion
10 56 (6.00%) Azure Cosmos DB
11 54 (5.78%) Serverless
12 49 (5.25%) Azure DevOps
13 48 (5.14%) PowerApps
14 45 (4.82%) Dynamics 365
15 35 (3.75%) SaaS
16 33 (3.53%) GitHub
17 29 (3.10%) IaaS
17 29 (3.10%) PaaS
18 25 (2.68%) Power Automate
19 21 (2.25%) Cloud Computing
Communications & Networking
1 24 (2.57%) Wireless
2 18 (1.93%) DSL
3 8 (0.86%) Network Security
4 6 (0.64%) DNS
5 4 (0.43%) Firewall
6 3 (0.32%) NetScaler
7 1 (0.11%) HTTP
7 1 (0.11%) SAN
Database & Business Intelligence
1 388 (41.54%) Data Warehouse
2 278 (29.76%) Power BI
3 178 (19.06%) Big Data
4 172 (18.42%) SQL Server
5 157 (16.81%) Dremio
6 153 (16.38%) Amazon Athena
7 152 (16.27%) Apache Arrow
8 141 (15.10%) Apache Ignite
9 87 (9.31%) NoSQL
10 81 (8.67%) Tableau
11 74 (7.92%) Azure SQL Database
12 64 (6.85%) Metadata
13 58 (6.21%) SQL Server Integration Services
14 52 (5.57%) Relational Database
15 51 (5.46%) Star Schema
16 50 (5.35%) Elasticsearch
16 50 (5.35%) Hadoop
17 45 (4.82%) Azure SQL Data Warehouse
18 44 (4.71%) BigQuery
19 43 (4.60%) Qlik Sense
Development Applications
1 39 (4.18%) GitLab
2 29 (3.10%) Jenkins
3 25 (2.68%) JIRA
4 24 (2.57%) Git
5 16 (1.71%) CircleCI
6 10 (1.07%) Selenium
7 9 (0.96%) JUnit
7 9 (0.96%) Oracle Forms
8 8 (0.86%) Cucumber
8 8 (0.86%) MLflow
8 8 (0.86%) Visual Studio
9 7 (0.75%) SQL Server Management Studio (SSMS)
10 6 (0.64%) ERwin
10 6 (0.64%) RStudio
11 4 (0.43%) Zephyr
12 3 (0.32%) ER Studio
12 3 (0.32%) Octopus Deploy
13 2 (0.21%) Bitbucket
14 1 (0.11%) Gradle
14 1 (0.11%) SoapUI
General
1 429 (45.93%) Finance
2 213 (22.81%) Banking
3 176 (18.84%) Social Skills
4 154 (16.49%) Analytical Skills
5 142 (15.20%) Investment Banking
6 122 (13.06%) Front Office
7 95 (10.17%) Inclusion and Diversity
8 92 (9.85%) Law
9 63 (6.75%) Legal
10 57 (6.10%) Marketing
11 50 (5.35%) Retail
12 32 (3.43%) Financial Institution
12 32 (3.43%) Presentation Skills
13 21 (2.25%) Public Sector
14 19 (2.03%) Retail Banking
15 18 (1.93%) Influencing Skills
16 15 (1.61%) Billing
17 12 (1.28%) Games
18 11 (1.18%) Telecoms
19 10 (1.07%) Back Office
Job Titles
1 342 (36.62%) Data Engineer
2 204 (21.84%) Architect
3 191 (20.45%) Senior
4 148 (15.85%) Lead
5 113 (12.10%) Data Architect
6 110 (11.78%) Software Engineer
7 103 (11.03%) Developer
8 82 (8.78%) Senior Data Engineer
9 72 (7.71%) AWS Engineer
10 70 (7.49%) Azure Engineer
11 67 (7.17%) Solutions Architect
12 62 (6.64%) Azure Data Engineer
13 54 (5.78%) Consultant
13 54 (5.78%) Python Engineer
14 40 (4.28%) Data Consultant
14 40 (4.28%) Lead Data Engineer
15 37 (3.96%) Enterprise Architect
16 36 (3.85%) Lead Software Engineer
16 36 (3.85%) Python Software Engineer
17 35 (3.75%) Java Engineer
Libraries, Frameworks & Software Standards
1 152 (16.27%) gRPC
1 152 (16.27%) Protocol Buffers
2 137 (14.67%) Pandas
3 122 (13.06%) NumPy
3 122 (13.06%) SciPy
4 102 (10.92%) PySpark
5 71 (7.60%) Kafka
6 19 (2.03%) Apache Beam
6 19 (2.03%) OpenID
7 18 (1.93%) JSON
8 15 (1.61%) RESTful
8 15 (1.61%) XML
9 14 (1.50%) .NET
10 9 (0.96%) MDX
11 8 (0.86%) Apache Avro
11 8 (0.86%) Spark Streaming
12 6 (0.64%) REST
13 5 (0.54%) React
13 5 (0.54%) Web Services
14 4 (0.43%) SOAP
Miscellaneous
1 152 (16.27%) Hedge funds
2 122 (13.06%) Derivative
2 122 (13.06%) Equities
2 122 (13.06%) Fixed Income
2 122 (13.06%) Foreign Exchange (FX)
3 73 (7.82%) Algorithms
4 64 (6.85%) Data Structures
5 43 (4.60%) Self-Motivation
6 40 (4.28%) Product Ownership
7 39 (4.18%) Onboarding
8 35 (3.75%) Cloud Native
8 35 (3.75%) Management Information System
9 31 (3.32%) Clustering
10 29 (3.10%) Greenfield Project
11 24 (2.57%) Public Cloud
12 19 (2.03%) Blockchain
12 19 (2.03%) Mainframe
13 17 (1.82%) Enterprise Software
14 14 (1.50%) Data Centre
14 14 (1.50%) Distributed Systems
Operating Systems
1 27 (2.89%) Unix
2 11 (1.18%) Linux
3 6 (0.64%) Windows
4 3 (0.32%) Windows Server
5 1 (0.11%) AIX
5 1 (0.11%) Solaris
5 1 (0.11%) VMS
Processes & Methodologies
1 411 (44.00%) Agile
2 321 (34.37%) Business Intelligence
3 312 (33.40%) Analytics
4 281 (30.09%) ETL
5 260 (27.84%) Data Pipeline
6 256 (27.41%) Data Modelling
7 242 (25.91%) Data Engineering
8 241 (25.80%) Machine Learning
9 214 (22.91%) Artificial Intelligence
10 212 (22.70%) Software Engineering
11 191 (20.45%) Asset Management
12 185 (19.81%) CI/CD
13 167 (17.88%) Visualisation
14 147 (15.74%) DevOps
15 146 (15.63%) Data Strategy
15 146 (15.63%) Investment Management
16 142 (15.20%) Data Science
17 132 (14.13%) Roadmaps
18 124 (13.28%) Data Architecture
19 119 (12.74%) Data Integration
Programming Languages
1 483 (51.71%) SQL
2 473 (50.64%) Python
3 259 (27.73%) Java
4 157 (16.81%) JavaScript
5 125 (13.38%) TypeScript
6 96 (10.28%) T-SQL
7 55 (5.89%) Scala
8 44 (4.71%) Data Analysis Expressions
9 40 (4.28%) Go
10 32 (3.43%) C#
11 26 (2.78%) R
12 21 (2.25%) PL/SQL
13 10 (1.07%) PowerShell
14 8 (0.86%) Kotlin
14 8 (0.86%) U-SQL
15 5 (0.54%) C
16 3 (0.32%) Bash
16 3 (0.32%) C++
17 1 (0.11%) Kusto Query Language
17 1 (0.11%) Shell Script
Qualifications
1 88 (9.42%) Degree
2 57 (6.10%) Azure Certification
3 43 (4.60%) Microsoft Certification
4 38 (4.07%) AWS Certification
5 19 (2.03%) Computer Science Degree
6 13 (1.39%) SC Cleared
6 13 (1.39%) Security Cleared
7 11 (1.18%) DBS Check
7 11 (1.18%) Master's Degree
8 8 (0.86%) DV Cleared
9 6 (0.64%) CIMA
10 3 (0.32%) MCP
10 3 (0.32%) MCSE
11 2 (0.21%) MCSA
11 2 (0.21%) Postgraduate
12 1 (0.11%) AWS Certified Solutions Architect
12 1 (0.11%) BPSS Clearance
12 1 (0.11%) HND
12 1 (0.11%) IBM Certification
12 1 (0.11%) LPIC
Quality Assurance & Compliance
1 178 (19.06%) Data Quality
2 52 (5.57%) GDPR
3 20 (2.14%) SLA
4 13 (1.39%) QA
5 8 (0.86%) PSD2
6 6 (0.64%) Accessibility
7 5 (0.54%) California Consumer Privacy Act
8 3 (0.32%) PMO
9 1 (0.11%) GMP
9 1 (0.11%) HIPAA
9 1 (0.11%) ISO/IEC 27001
System Software
1 27 (2.89%) Docker
2 8 (0.86%) HDFS
3 7 (0.75%) Active Directory
4 1 (0.11%) Apache Flume
4 1 (0.11%) Virtual Machines
4 1 (0.11%) VMware Infrastructure
Systems Management
1 90 (9.64%) Kubernetes
2 66 (7.07%) Terraform
3 30 (3.21%) Prometheus
4 3 (0.32%) HP Fortify
5 2 (0.21%) Argo
5 2 (0.21%) Progress Chef
6 1 (0.11%) Ansible
Vendors
1 292 (31.26%) Databricks
2 273 (29.23%) Microsoft
3 71 (7.60%) Google
4 68 (7.28%) Alteryx
5 44 (4.71%) Oracle
6 22 (2.36%) Cognos
7 18 (1.93%) Sybase
8 16 (1.71%) SAP
9 14 (1.50%) Teradata
10 12 (1.28%) Cloudera
11 8 (0.86%) Duck Creek
11 8 (0.86%) Informatica
11 8 (0.86%) Talend
12 7 (0.75%) Adobe
13 6 (0.64%) Apple
13 6 (0.64%) IBM
14 4 (0.43%) OpenAI
14 4 (0.43%) SnapLogic
15 3 (0.32%) CheckPoint
15 3 (0.32%) Veracode