Period
to 12 May 2021

The following table provides summary statistics for contract job vacancies with a requirement for Data Lake skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Data Lake over the 6 months to 12 May 2021 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Data Lakes
UK
6 months to
12 May 2021
Same period 2020 Same period 2019
Rank 196 235 335
Rank change year-on-year +39 +100 +145
Contract jobs citing Data Lake 731 577 735
As % of all contract jobs advertised in the UK 1.30% 1.13% 0.80%
As % of the Database & Business Intelligence category 6.18% 5.34% 3.55%
Number of daily rates quoted 475 422 555
10th Percentile £395 £396 £413
25th Percentile £429 £450 £463
Median daily rate (50th Percentile) £510 £525 £525
Median % change year-on-year -2.86% - +5.00%
75th Percentile £613 £625 £613
90th Percentile £688 £688 £695
UK excluding London median daily rate £480 £500 £500
% change year-on-year -4.00% - +2.56%
Number of hourly rates quoted 5 3 1
10th Percentile £27.05 £26.00 "
25th Percentile £32.00 £37.63 £24.25
Median hourly rate £44.50 £59.00 £24.50
Median % change year-on-year -24.58% +140.82% -33.78%
75th Percentile £52.25 £66.13 £24.75
90th Percentile £60.80 £69.20 "
UK excluding London median hourly rate - £43.00 -

Data Lake is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Contract vacancies with a requirement for database or business intelligence skills 11,822 10,808 20,708
As % of all contract IT jobs advertised in the UK 21.01% 21.16% 22.50%
Number of daily rates quoted 8,079 7,418 14,391
10th Percentile £325 £325 £325
25th Percentile £406 £400 £400
Median daily rate (50th Percentile) £500 £475 £475
Median % change year-on-year +5.26% - +2.15%
75th Percentile £595 £581 £575
90th Percentile £675 £675 £650
UK excluding London median daily rate £440 £450 £430
% change year-on-year -2.22% +4.65% +3.61%
Number of hourly rates quoted 207 162 233
10th Percentile £11.71 £13.52 £13.40
25th Percentile £19.00 £18.06 £18.00
Median hourly rate £26.50 £24.70 £30.00
Median % change year-on-year +7.29% -17.67% -
75th Percentile £45.00 £46.00 £47.50
90th Percentile £60.65 £59.35 £57.50
UK excluding London median hourly rate £24.30 £22.50 £30.00
% change year-on-year +8.00% -25.00% -1.64%

Data Lake
Job Vacancy Trend

Job postings citing Data Lake as a proportion of all IT jobs advertised.

Job vacancy trend for Data Lake in the UK

Data Lake
Contractor Daily Rate Trend

3-month moving average daily rate quoted in jobs citing Data Lake.

Daily rate trend for Data Lake in the UK

Data Lake
Daily Rate Histogram

Daily rate distribution for jobs citing Data Lake over the 6 months to 12 May 2021.

Daily rate histogram for Data Lake in the UK

Data Lake
Contractor Hourly Rate Trend

3-month moving average hourly rates quoted in jobs citing Data Lake.

Hourly rate trend for Data Lake in the UK

Data Lake
Hourly Rate Histogram

Hourly rate distribution of jobs citing Data Lake over the 6 months to 12 May 2021.

Hourly rate histogram for Data Lake in the UK

Data Lake
Top 15 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Data Lake within the UK over the 6 months to 12 May 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +16 577 £515 -1.89% 181
London +18 343 £538 -2.27% 79
Work from Home -57 263 £503 -13.36% 49
UK excluding London -11 191 £480 -4.00% 119
North of England +59 53 £525 -4.55% 25
South East -58 52 £500 - 35
North West +31 30 £525 - 10
Midlands +19 29 £425 -8.11% 25
South West +37 24 £555 -2.42% 6
Yorkshire +39 23 £469 -14.77% 14
East Midlands +25 15 £404 -4.88% 7
West Midlands +12 14 £500 -23.08% 18
Scotland -8 14 £413 -8.33% 13
East of England +10 13 £450 -5.26% 10
Wales +10 6 £415 -17.00% 3

For the 6 months to 12 May 2021, IT contractor jobs citing Data Lake also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for Data Lake.

1 460 (62.93%) Azure
2 331 (45.28%) SQL
3 274 (37.48%) Data Warehouse
4 250 (34.20%) Agile Software Development
5 245 (33.52%) Azure Data Factory
6 239 (32.69%) Databricks
7 234 (32.01%) Analytics
8 226 (30.92%) Data Modelling
9 222 (30.37%) DevOps
10 221 (30.23%) Business Intelligence
11 201 (27.50%) Python
12 196 (26.81%) ETL
13 185 (25.31%) Microsoft
14 173 (23.67%) Power Platform
15 171 (23.39%) Big Data
16 163 (22.30%) Power BI
17 149 (20.38%) AWS
17 149 (20.38%) Azure SQL Database
18 136 (18.60%) Azure Synapse Analytics
19 134 (18.33%) Data Pipeline
20 133 (18.19%) Apache Spark
21 125 (17.10%) Finance
22 112 (15.32%) Serverless
23 110 (15.05%) SQL Server
24 109 (14.91%) Data Engineering
24 109 (14.91%) Social Skills
25 103 (14.09%) Hadoop
26 102 (13.95%) Migration
27 96 (13.13%) Data Architecture
28 86 (11.76%) Scrum

Data Lake
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 133 (18.19%) Apache Spark
2 62 (8.48%) Apache
3 44 (6.02%) Apache Airflow
4 26 (3.56%) SharePoint
5 20 (2.74%) Confluence
6 13 (1.78%) SAS
7 10 (1.37%) Elasticsearch
8 8 (1.09%) Oracle Workflow
8 8 (1.09%) WebLogic
9 3 (0.41%) ArcGIS
9 3 (0.41%) MQSeries
9 3 (0.41%) Oracle Coherence
10 2 (0.27%) Apache Pig
10 2 (0.27%) Apache Pulsar
10 2 (0.27%) Apache Solr
10 2 (0.27%) Camunda
10 2 (0.27%) Luigi
11 1 (0.14%) MS Exchange
11 1 (0.14%) OpenStack
Applications
1 8 (1.09%) Microsoft Office
2 7 (0.96%) Microsoft Excel
3 4 (0.55%) SPSS
4 3 (0.41%) Microsoft PowerPoint
5 2 (0.27%) Microsoft Project
5 2 (0.27%) MS Visio
Business Applications
1 9 (1.23%) Dynamics CRM
2 4 (0.55%) Temenos T24
3 3 (0.41%) Aligne
3 3 (0.41%) SAP CRM
3 3 (0.41%) SAP ERP
4 1 (0.14%) Maximo
4 1 (0.14%) NetSuite
4 1 (0.14%) Oracle EBS
4 1 (0.14%) SAP APO
4 1 (0.14%) SAP IBP
4 1 (0.14%) SAP S/4HANA
4 1 (0.14%) SAP SD
4 1 (0.14%) Traffica
4 1 (0.14%) Workfront
Cloud Services
1 460 (62.93%) Azure
2 245 (33.52%) Azure Data Factory
3 173 (23.67%) Power Platform
4 149 (20.38%) AWS
5 136 (18.60%) Azure Synapse Analytics
6 112 (15.32%) Serverless
7 77 (10.53%) Azure Cosmos DB
8 59 (8.07%) Azure Functions
9 42 (5.75%) PaaS
10 41 (5.61%) Amazon S3
11 40 (5.47%) AWS Lambda
11 40 (5.47%) Azure Storage
12 39 (5.34%) GCP
13 27 (3.69%) Azure Key Vault
14 26 (3.56%) Azure Logic Apps
15 18 (2.46%) PowerApps
16 17 (2.33%) Microsoft 365
16 17 (2.33%) SaaS
17 16 (2.19%) Amazon EC2
17 16 (2.19%) GitHub
Communications & Networking
1 10 (1.37%) Firewall
1 10 (1.37%) HTTP
2 2 (0.27%) MQTT
2 2 (0.27%) NAS
3 1 (0.14%) DNS
3 1 (0.14%) FTP
3 1 (0.14%) Internet
3 1 (0.14%) Unified Communications
3 1 (0.14%) Wi-Fi
3 1 (0.14%) X.21
Database & Business Intelligence
1 274 (37.48%) Data Warehouse
2 171 (23.39%) Big Data
3 163 (22.30%) Power BI
4 149 (20.38%) Azure SQL Database
5 110 (15.05%) SQL Server
6 103 (14.09%) Hadoop
7 64 (8.76%) Azure SQL Data Warehouse
8 62 (8.48%) SQL Server Integration Services
9 61 (8.34%) Apache Hive
10 57 (7.80%) SQL Server Analysis Services
11 45 (6.16%) NoSQL
12 44 (6.02%) SQL Server Reporting Services
13 41 (5.61%) Relational Database
14 32 (4.38%) Tableau
15 28 (3.83%) Amazon Redshift
16 27 (3.69%) PostgreSQL
16 27 (3.69%) Snowflake Schema
17 26 (3.56%) Enterprise Data Warehouse
17 26 (3.56%) Microsoft Analysis Services
18 24 (3.28%) HBase
Development Applications
1 61 (8.34%) JIRA
2 60 (8.21%) Git (software)
3 50 (6.84%) Jenkins
4 25 (3.42%) kappa
5 22 (3.01%) GitLab
5 22 (3.01%) PowerDesigner
6 19 (2.60%) Visual Studio
7 12 (1.64%) Maven
8 10 (1.37%) Sonatype Nexus
9 9 (1.23%) ERwin
9 9 (1.23%) Visual Studio Team System
10 7 (0.96%) IBM UrbanCode
11 6 (0.82%) ER Studio
11 6 (0.82%) Gradle
12 4 (0.55%) Bitbucket
12 4 (0.55%) Octopus Deploy
13 3 (0.41%) SQL Server Management Studio (SSMS)
13 3 (0.41%) Subversion
13 3 (0.41%) TeamCity
14 2 (0.27%) ReSharper
General
1 125 (17.10%) Finance
2 109 (14.91%) Social Skills
3 56 (7.66%) Analytical Skills
4 37 (5.06%) Banking
5 32 (4.38%) Public Sector
6 29 (3.97%) Retail
7 19 (2.60%) Presentation Skills
8 14 (1.92%) Legal
9 12 (1.64%) Documentation Skills
10 11 (1.50%) Marketing
11 10 (1.37%) Games
11 10 (1.37%) Influencing Skills
12 9 (1.23%) Front Office
13 8 (1.09%) Manufacturing
14 5 (0.68%) Publishing
15 4 (0.55%) Automotive
16 3 (0.41%) Investment Banking
16 3 (0.41%) Organisational Skills
16 3 (0.41%) Telecoms
17 2 (0.27%) Advertising
Job Titles
1 238 (32.56%) Data Engineer
2 165 (22.57%) Architect
3 126 (17.24%) Data Architect
4 111 (15.18%) Azure Engineer
5 96 (13.13%) Developer
6 64 (8.76%) Azure Architect
7 52 (7.11%) Analyst
8 50 (6.84%) AWS Engineer
9 44 (6.02%) Solutions Architect
10 38 (5.20%) Consultant
11 37 (5.06%) Project Manager
12 28 (3.83%) Data Analyst
13 23 (3.15%) DevOps Engineer
14 22 (3.01%) Big Data Engineer
15 21 (2.87%) Azure Developer
16 19 (2.60%) Azure DevOps
16 19 (2.60%) Python Engineer
16 19 (2.60%) Senior Data Engineer
17 18 (2.46%) Azure DevOps Engineer
17 18 (2.46%) Scala Engineer
Libraries, Frameworks & Software Standards
1 83 (11.35%) Kafka
2 51 (6.98%) PySpark
3 30 (4.10%) .NET
4 24 (3.28%) Web Services
5 23 (3.15%) XML
6 17 (2.33%) HTML
6 17 (2.33%) RESTful
7 13 (1.78%) YAML
8 11 (1.50%) JSON
8 11 (1.50%) Spark Streaming
8 11 (1.50%) Velocity
9 10 (1.37%) .NET Framework
9 10 (1.37%) Pandas
9 10 (1.37%) PostGIS
10 9 (1.23%) Spring
11 8 (1.09%) Flask
11 8 (1.09%) MDX
11 8 (1.09%) pytest
12 6 (0.82%) NumPy
13 5 (0.68%) REST
Miscellaneous
1 40 (5.47%) Data Structures
2 33 (4.51%) Greenfield Project
3 31 (4.24%) Algorithms
4 22 (3.01%) Virtual Team
5 18 (2.46%) Cloud Native
6 16 (2.19%) Self-Motivation
7 11 (1.50%) Reference Data
8 10 (1.37%) NHS
8 10 (1.37%) Taxonomies
9 9 (1.23%) Cyber Defence
10 8 (1.09%) Linux Command Line
11 7 (0.96%) Management Information System
12 6 (0.82%) SCADA
13 5 (0.68%) BYOD
13 5 (0.68%) Public Cloud
13 5 (0.68%) Reinsurance
13 5 (0.68%) User Experience
14 4 (0.55%) Derivative
14 4 (0.55%) Foreign Exchange (FX)
15 3 (0.41%) Market Risk
Operating Systems
1 31 (4.24%) Linux
2 12 (1.64%) Red Hat Enterprise Linux
3 10 (1.37%) Windows
4 8 (1.09%) CentOS
4 8 (1.09%) SUSE Linux Enterprise Server
5 7 (0.96%) Unix
6 3 (0.41%) Windows Server
7 2 (0.27%) Ubuntu
8 1 (0.14%) Android
8 1 (0.14%) Apple iOS
8 1 (0.14%) Debian
8 1 (0.14%) Mac OS X
Processes & Methodologies
1 250 (34.20%) Agile Software Development
2 234 (32.01%) Analytics
3 226 (30.92%) Data Modelling
4 222 (30.37%) DevOps
5 221 (30.23%) Business Intelligence
6 196 (26.81%) ETL
7 134 (18.33%) Data Pipeline
8 109 (14.91%) Data Engineering
9 102 (13.95%) Migration
10 96 (13.13%) Data Architecture
11 86 (11.76%) Scrum
12 85 (11.63%) Data Integration
13 81 (11.08%) Data Management
14 79 (10.81%) Data Science
15 76 (10.40%) Data Analysis
16 72 (9.85%) Machine Learning
17 61 (8.34%) Visualisation
18 58 (7.93%) Data Analytics
19 52 (7.11%) Artificial Intelligence
20 51 (6.98%) Data Ingestion
Programming Languages
1 331 (45.28%) SQL
2 201 (27.50%) Python
3 66 (9.03%) Java
4 65 (8.89%) Scala
5 46 (6.29%) C#
6 39 (5.34%) T-SQL
7 34 (4.65%) PowerShell
8 23 (3.15%) JavaScript
9 18 (2.46%) R
10 15 (2.05%) PL/SQL
11 13 (1.78%) Data Analysis Expressions (DAX)
12 10 (1.37%) Bash Shell
13 9 (1.23%) Shell Script
14 8 (1.09%) Java 8
15 6 (0.82%) C++
15 6 (0.82%) Elm
15 6 (0.82%) Haskell
15 6 (0.82%) TypeScript
16 3 (0.41%) C
16 3 (0.41%) SAS Macro
Qualifications
1 47 (6.43%) Security Cleared
2 40 (5.47%) SC Cleared
3 22 (3.01%) Degree
4 6 (0.82%) DV Cleared
4 6 (0.82%) Microsoft Certification
5 5 (0.68%) Computer Science Degree
6 4 (0.55%) AWS Certification
6 4 (0.55%) PMI Certification
6 4 (0.55%) PMP
7 3 (0.41%) CISSP
7 3 (0.41%) Master's Degree
7 3 (0.41%) TOGAF Certification
8 2 (0.27%) BPSS Clearance
8 2 (0.27%) GIAC
8 2 (0.27%) OSCP
8 2 (0.27%) Tableau Certification
9 1 (0.14%) CEH
9 1 (0.14%) CompTIA Security+
9 1 (0.14%) ISTQB
9 1 (0.14%) MBCS
Quality Assurance & Compliance
1 82 (11.22%) Data Quality
2 35 (4.79%) QA
3 13 (1.78%) GDPR
4 5 (0.68%) IFRS
5 3 (0.41%) PCI DSS
5 3 (0.41%) PMO
6 2 (0.27%) NIST
6 2 (0.27%) SLA
7 1 (0.14%) Basel II
7 1 (0.14%) Cyber Essentials
7 1 (0.14%) HL7
7 1 (0.14%) NCSC
7 1 (0.14%) PSD2
7 1 (0.14%) Sarbanes-Oxley
7 1 (0.14%) WCAG
System Software
1 21 (2.87%) Docker
1 21 (2.87%) HDFS
2 14 (1.92%) Active Directory
3 8 (1.09%) iptables
4 4 (0.55%) ISA Server
5 2 (0.27%) Hyper-V
5 2 (0.27%) Virtual Machines
5 2 (0.27%) VMware Infrastructure
6 1 (0.14%) Apache Flume
Systems Management
1 46 (6.29%) Terraform
2 31 (4.24%) Kubernetes
3 22 (3.01%) Ansible
4 15 (2.05%) Oozie
4 15 (2.05%) Opscode Chef
5 8 (1.09%) Hadoop YARN
5 8 (1.09%) Puppet
6 3 (0.41%) Single Sign-On
7 2 (0.27%) Kubeflow
8 1 (0.14%) Computer Emergency Response Teams
8 1 (0.14%) Control-M
8 1 (0.14%) Grafana
8 1 (0.14%) Oracle EDQ
8 1 (0.14%) Oracle RMAN
8 1 (0.14%) Prometheus
Vendors
1 239 (32.69%) Databricks
2 185 (25.31%) Microsoft
3 45 (6.16%) Oracle
4 25 (3.42%) SAP
5 22 (3.01%) BiZZdesign
6 19 (2.60%) Google
6 19 (2.60%) MuleSoft
6 19 (2.60%) Salesforce.com
7 16 (2.19%) Talend
8 12 (1.64%) Alteryx
8 12 (1.64%) Atlassian
9 11 (1.50%) Cloudera
9 11 (1.50%) Informatica
9 11 (1.50%) Red Hat
10 10 (1.37%) Teradata
11 9 (1.23%) Cisco
11 9 (1.23%) Remedy
12 8 (1.09%) CA Agile Central
12 8 (1.09%) SaltStack
12 8 (1.09%) Serena