Period
to 15 November 2018

The following table provides summary statistics for permanent job vacancies with a requirement for Data Lake skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Lake over the 6 months to 15 November 2018 with a comparison to the same period in the previous 2 years.

Data Lakes
UK
6 months to
15 Nov 2018
Same period 2017 Same period 2016
Rank 585 883 1128
Rank change year-on-year +298 +245 +178
Permanent jobs citing Data Lake 678 321 116
As % of all permanent IT jobs advertised in the UK 0.42% 0.18% 0.061%
As % of the Database & Business Intelligence category 1.43% 0.59% 0.21%
Number of salaries quoted 524 283 93
UK median annual salary £65,000 £60,000 £75,000
Median salary % change year-on-year +8.33% -20.00% +25.00%
10th Percentile £42,500 £42,750 £42,500
90th Percentile £99,250 £100,000 £113,750
UK excluding London median annual salary £62,138 £55,000 £50,000
% change year-on-year +12.98% +10.00% +5.26%

Data Lake is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Permanent vacancies with a requirement for database or business intelligence skills 47,444 54,600 56,580
As % of all permanent IT jobs advertised in the UK 29.11% 30.24% 29.61%
Number of salaries quoted 39,046 44,939 48,318
UK median annual salary £50,000 £47,500 £47,500
Median salary % change year-on-year +5.26% - +5.56%
10th Percentile £31,250 £30,000 £28,750
90th Percentile £83,750 £77,500 £76,413
UK excluding London median annual salary £45,000 £42,500 £42,500
% change year-on-year +5.88% - +6.25%

Data Lake
Job Vacancy Trend

Job postings citing Data Lake as a percentage of all IT jobs advertised.

Job vacancy trend for Data Lake in the UK

Data Lake
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Data Lake.

Salary trend for Data Lake in the UK

Data Lake
Salary Histogram

The salary distribution of IT jobs citing Data Lake over the 6 months to 15 November 2018.

Salary histogram for Data Lake in the UK

Data Lake
Top 13 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Lake within the UK over the 6 months to 15 November 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +299 665 £65,000 +8.33% 102
London +263 432 £67,500 -3.57% 58
UK excluding London +125 244 £62,138 +12.98% 46
South East +32 95 £65,000 +23.81% 11
South West +53 58 £62,138 +10.47% 9
East of England +55 33 £52,500 -6.67% 7
Midlands +40 28 £58,750 +6.82% 6
North of England +38 26 £51,500 -10.43% 11
West Midlands +50 17 £64,733 -7.52% 5
North West +40 17 £60,000 +4.35% 7
East Midlands +9 11 £50,000 -9.09% 1
Yorkshire +33 9 £50,000 -28.57% 3
Scotland +2 8 £46,500 -7.00%

For the 6 months to 15 November 2018, IT jobs citing Data Lake also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Data Lake.

1 409 (60.32%) SQL
2 402 (59.29%) Data Warehouse
3 394 (58.11%) Business Intelligence
3 394 (58.11%) Analytics
4 375 (55.31%) Microsoft Azure
5 330 (48.67%) Big Data
6 299 (44.10%) Microsoft
7 295 (43.51%) SQL Server
8 238 (35.10%) SQL Server Integration Services
9 237 (34.96%) Agile Software Development
9 237 (34.96%) Power BI
10 224 (33.04%) Hadoop
11 221 (32.60%) ETL
11 221 (32.60%) Data Modelling
12 205 (30.24%) Finance
13 188 (27.73%) SQL Server Analysis Services
14 180 (26.55%) SQL Server Reporting Services
15 178 (26.25%) Python
16 172 (25.37%) Azure Data Factory
17 160 (23.60%) Azure SQL Database
18 144 (21.24%) Apache Spark
19 142 (20.94%) Azure SQL Data Warehouse
20 141 (20.80%) Machine Learning
21 132 (19.47%) Data Analytics
22 131 (19.32%) T-SQL
23 128 (18.88%) Amazon AWS
24 117 (17.26%) Visualisation
24 117 (17.26%) Data Science
25 115 (16.96%) C#
26 102 (15.04%) Oracle

Data Lake
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 144 (21.24%) Apache Spark
2 52 (7.67%) Apache
3 33 (4.87%) Apache Pig
4 28 (4.13%) Apache Solr
5 27 (3.98%) SAS
6 21 (3.10%) Elasticsearch
7 17 (2.51%) SharePoint
8 16 (2.36%) Confluence
9 5 (0.74%) Tomcat
10 4 (0.59%) IIS
11 3 (0.44%) Apache Airflow
11 3 (0.44%) BizTalk Server
11 3 (0.44%) JBoss
11 3 (0.44%) Luigi
12 2 (0.29%) Oracle SOA Suite
12 2 (0.29%) Tornado
12 2 (0.29%) WebLogic
13 1 (0.15%) CMS
13 1 (0.15%) Kentico
13 1 (0.15%) SharePoint 2013
Applications
1 34 (5.01%) Microsoft Excel
2 2 (0.29%) Microsoft Office
2 2 (0.29%) SPSS
3 1 (0.15%) Chatbot
Business Applications
1 7 (1.03%) Dynamics CRM
2 2 (0.29%) Dynamics AX
2 2 (0.29%) Maximo
3 1 (0.15%) Markit EDM
3 1 (0.15%) NetSuite
Cloud Services
1 375 (55.31%) Microsoft Azure
2 172 (25.37%) Azure Data Factory
3 128 (18.88%) Amazon AWS
4 79 (11.65%) PaaS
5 76 (11.21%) Data Lake Analytics
6 44 (6.49%) Serverless
7 40 (5.90%) Azure Machine Learning
8 36 (5.31%) Google Cloud Platform
9 31 (4.57%) Cortana Intelligence Suite
10 21 (3.10%) Azure Functions
10 21 (3.10%) SaaS
11 20 (2.95%) IaaS
12 18 (2.65%) Cloud Computing
13 17 (2.51%) Amazon Kinesis
14 15 (2.21%) Amazon S3
15 11 (1.62%) AWS Lambda
16 8 (1.18%) Azure Active Directory
17 7 (1.03%) Amazon EMR
18 6 (0.88%) BlazeMeter
19 5 (0.74%) Amazon SageMaker
Communications & Networking
1 8 (1.18%) Firewall
2 7 (1.03%) HTTP
3 5 (0.74%) Network Security
4 4 (0.59%) Kerberos
5 3 (0.44%) GoToMeeting
5 3 (0.44%) Internet
5 3 (0.44%) SNMP
5 3 (0.44%) SSL
5 3 (0.44%) WebEx
6 2 (0.29%) VPN
Database & Business Intelligence
1 402 (59.29%) Data Warehouse
2 330 (48.67%) Big Data
3 295 (43.51%) SQL Server
4 238 (35.10%) SQL Server Integration Services
5 237 (34.96%) Power BI
6 224 (33.04%) Hadoop
7 188 (27.73%) SQL Server Analysis Services
8 180 (26.55%) SQL Server Reporting Services
9 160 (23.60%) Azure SQL Database
10 142 (20.94%) Azure SQL Data Warehouse
11 99 (14.60%) NoSQL
12 91 (13.42%) Cosmos DB
13 82 (12.09%) Apache Hive
14 72 (10.62%) Relational Database
15 58 (8.55%) Microsoft Analysis Services
15 58 (8.55%) MongoDB
16 57 (8.41%) MySQL
17 53 (7.82%) Apache Cassandra
18 51 (7.52%) HBase
19 38 (5.60%) Amazon Redshift
Development Applications
1 50 (7.37%) Git (software)
2 47 (6.93%) Visual Studio
3 36 (5.31%) Jenkins
4 31 (4.57%) Visual Studio Team System
5 26 (3.83%) JIRA
6 16 (2.36%) ERwin
7 14 (2.06%) Team Foundation Server
8 13 (1.92%) TeamCity
9 10 (1.47%) Octopus Deploy
10 9 (1.33%) SpecFlow
10 9 (1.33%) Subversion
11 8 (1.18%) Eclipse
11 8 (1.18%) tSQLt
12 7 (1.03%) ER Studio
13 6 (0.88%) Atlassian Bamboo
13 6 (0.88%) SQL Server Management Studio (SSMS)
14 5 (0.74%) JUnit
14 5 (0.74%) Maven
14 5 (0.74%) SoapUI
15 4 (0.59%) IntelliJ
General
1 205 (30.24%) Finance
2 80 (11.80%) Retail
3 63 (9.29%) Banking
4 49 (7.23%) Advertising
5 30 (4.42%) Investment Banking
6 20 (2.95%) Marketing
7 18 (2.65%) Games
8 14 (2.06%) Financial Institution
9 12 (1.77%) Corporate Banking
10 9 (1.33%) Telecoms
11 6 (0.88%) Online Betting
12 4 (0.59%) Automotive
12 4 (0.59%) Front Office
12 4 (0.59%) Manufacturing
12 4 (0.59%) Publishing
13 3 (0.44%) General Ledger
14 2 (0.29%) Digital Economy
15 1 (0.15%) Electronics
15 1 (0.15%) Law
15 1 (0.15%) Legal
Job Titles
1 204 (30.09%) Developer
2 131 (19.32%) Architect
3 103 (15.19%) Data Architect
4 99 (14.60%) Data Engineer
5 94 (13.86%) Business Intelligence Developer
6 88 (12.98%) Consultant
7 61 (9.00%) SQL Developer
8 49 (7.23%) Data Consultant
9 48 (7.08%) SSIS Developer
10 45 (6.64%) Data Warehouse Developer
11 44 (6.49%) Analyst
12 40 (5.90%) SQL BI Developer
13 37 (5.46%) Azure Consultant
13 37 (5.46%) Business Intelligence Consultant
14 34 (5.01%) Cloud Architect
15 33 (4.87%) Big Data Developer
15 33 (4.87%) Big Data Engineer
16 32 (4.72%) Senior Developer
17 30 (4.42%) SSRS Developer
18 29 (4.28%) T-SQL Developer
Libraries, Frameworks & Software Standards
1 92 (13.57%) Kafka
2 89 (13.13%) .NET
3 49 (7.23%) JSON
4 32 (4.72%) MDX
5 28 (4.13%) Web Services
6 24 (3.54%) Velocity
7 18 (2.65%) AngularJS
7 18 (2.65%) HTML
8 17 (2.51%) RESTful
9 16 (2.36%) ASP.NET
10 15 (2.21%) HTML5
10 15 (2.21%) RabbitMQ
11 13 (1.92%) Natural Language Processing
11 13 (1.92%) REST
11 13 (1.92%) SAML
12 12 (1.77%) CSS
13 11 (1.62%) Spark Streaming
13 11 (1.62%) XML
14 10 (1.47%) .NET Core
14 10 (1.47%) Twitter Bootstrap
Miscellaneous
1 78 (11.50%) Analytical Skills
2 36 (5.31%) Computer Science
3 20 (2.95%) Greenfield Project
4 19 (2.80%) Cyberthreat
5 16 (2.36%) Reference Data
6 15 (2.21%) Algorithms
6 15 (2.21%) Enterprise Software
6 15 (2.21%) Management Information System
7 13 (1.92%) Equities
8 12 (1.77%) Credit Risk
9 11 (1.62%) Driving Licence
10 10 (1.47%) Self-Motivation
11 9 (1.33%) Clustering
12 8 (1.18%) Cloud Native
12 8 (1.18%) Data Structures
12 8 (1.18%) NHS
12 8 (1.18%) Public Cloud
12 8 (1.18%) Virtual Team
13 7 (1.03%) Fintech
13 7 (1.03%) Reinsurance
Operating Systems
1 38 (5.60%) Linux
2 22 (3.24%) Unix
2 22 (3.24%) Windows
3 16 (2.36%) Red Hat Enterprise Linux
4 13 (1.92%) CentOS
5 8 (1.18%) Ubuntu
6 5 (0.74%) Windows Server
7 3 (0.44%) Windows Server 2008
8 1 (0.15%) Android
8 1 (0.15%) Apple iOS
8 1 (0.15%) Windows 10
Processes & Methodologies
1 394 (58.11%) Analytics
1 394 (58.11%) Business Intelligence
2 237 (34.96%) Agile Software Development
3 221 (32.60%) Data Modelling
3 221 (32.60%) ETL
4 141 (20.80%) Machine Learning
5 132 (19.47%) Data Analytics
6 117 (17.26%) Data Science
6 117 (17.26%) Visualisation
7 99 (14.60%) Data Architecture
8 93 (13.72%) Data Engineering
9 81 (11.95%) Data Strategy
10 72 (10.62%) Data Governance
11 70 (10.32%) Problem-Solving
12 69 (10.18%) Data Integration
13 65 (9.59%) Continuous Integration
14 62 (9.14%) Data Management
15 61 (9.00%) Mentoring
16 57 (8.41%) Data Analysis
16 57 (8.41%) Scrum
Programming Languages
1 409 (60.32%) SQL
2 178 (26.25%) Python
3 131 (19.32%) T-SQL
4 115 (16.96%) C#
5 94 (13.86%) Java
6 82 (12.09%) Data Analysis Expressions (DAX)
7 74 (10.91%) Scala
8 69 (10.18%) PowerShell
9 68 (10.03%) R
10 58 (8.55%) JavaScript
11 54 (7.96%) U-SQL
12 20 (2.95%) Shell Script
13 18 (2.65%) Perl
14 14 (2.06%) C++
15 10 (1.47%) C
16 7 (1.03%) Ruby
17 5 (0.74%) Bash Shell
18 4 (0.59%) VB.NET
19 3 (0.44%) HiveQL
19 3 (0.44%) PL/SQL
Qualifications
1 74 (10.91%) Degree
2 26 (3.83%) Microsoft Certification
3 9 (1.33%) Computer Science Degree
4 8 (1.18%) Master's Degree
5 6 (0.88%) PhD
6 4 (0.59%) AWS Certification
6 4 (0.59%) MCSA
6 4 (0.59%) MCSE
7 3 (0.44%) GSEC
7 3 (0.44%) MBA
7 3 (0.44%) MCAD
7 3 (0.44%) MCDBA
7 3 (0.44%) MCSD
8 1 (0.15%) CISM
8 1 (0.15%) CISSP
8 1 (0.15%) CompTIA Security+
8 1 (0.15%) Numerate Degree
8 1 (0.15%) SC Cleared
8 1 (0.15%) Security Cleared
Quality Assurance & Compliance
1 48 (7.08%) Data Quality
2 41 (6.05%) QA
3 12 (1.77%) SLA
4 9 (1.33%) GDPR
4 9 (1.33%) ISO/IEC 27001
5 3 (0.44%) PMO
6 1 (0.15%) Actionable Recommendations
6 1 (0.15%) PCI DSS
System Software
1 45 (6.64%) HDFS
2 23 (3.39%) Apache Flume
2 23 (3.39%) Docker
3 14 (2.06%) Apache ZooKeeper
4 13 (1.92%) Active Directory
4 13 (1.92%) Virtual Machines
5 5 (0.74%) Hyper-V
5 5 (0.74%) VMware Infrastructure
6 3 (0.44%) XenApp
Systems Management
1 23 (3.39%) Oozie
2 22 (3.24%) Terraform
3 20 (2.95%) Opscode Chef
3 20 (2.95%) Puppet
4 15 (2.21%) Kubernetes
5 13 (1.92%) Kibana
6 6 (0.88%) Grafana
7 4 (0.59%) Ansible
7 4 (0.59%) Rundeck
8 2 (0.29%) Hadoop YARN
9 1 (0.15%) ArcSight ESM
9 1 (0.15%) logstash
9 1 (0.15%) Microsoft Intune
9 1 (0.15%) Salt
Vendors
1 299 (44.10%) Microsoft
2 102 (15.04%) Oracle
3 100 (14.75%) Tableau
4 56 (8.26%) Cloudera
5 54 (7.96%) Informatica
6 47 (6.93%) Talend
7 45 (6.64%) Databricks
7 45 (6.64%) Teradata
8 36 (5.31%) Google
9 35 (5.16%) Hortonworks
10 32 (4.72%) Netezza
11 24 (3.54%) Ab Initio
11 24 (3.54%) MapR
12 18 (2.65%) Salesforce.com
13 17 (2.51%) SAP
14 12 (1.77%) Cognizant
14 12 (1.77%) SnapLogic
15 11 (1.62%) Sybase
15 11 (1.62%) WorldPay
16 9 (1.33%) IBM