Period
to 19 October 2020

The following table provides summary statistics for permanent job vacancies with a requirement for Data Engineering skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Engineering over the 6 months to 19 October 2020 with a comparison to the same period in the previous 2 years.

Data Engineering
UK
6 months to
19 Oct 2020
Same period 2019 Same period 2018
Rank 314 448 603
Rank change year-on-year +134 +155 +331
Permanent jobs citing Data Engineering 457 901 671
As % of all permanent jobs advertised in the UK 0.94% 0.66% 0.41%
As % of the Processes & Methodologies category 1.02% 0.72% 0.44%
Number of salaries quoted 438 731 501
Median annual salary £60,000 £70,000 £67,500
Median salary % change year-on-year -14.29% +3.70% -3.57%
10th Percentile £42,500 £40,000 £46,250
90th Percentile £95,000 £95,000 £100,000
UK excluding London median annual salary £55,000 £52,500 £62,138
% change year-on-year +4.76% -15.51% +24.28%

Data Engineering is in the Processes and Methodologies category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for process or methodology skills.

All Process and Methodology Skills
UK
Permanent vacancies with a requirement for process or methodology skills 44,727 125,652 151,469
As % of all permanent IT jobs advertised in the UK 92.45% 92.23% 91.92%
Number of salaries quoted 36,813 98,822 120,565
Median annual salary £55,000 £52,500 £50,000
Median salary % change year-on-year +4.76% +5.00% -
10th Percentile £33,000 £30,000 £29,000
90th Percentile £90,000 £86,250 £83,750
UK excluding London median annual salary £49,000 £45,000 £45,000
% change year-on-year +8.89% - +5.88%

Data Engineering
Job Vacancy Trend

Job postings citing Data Engineering as a proportion of all IT jobs advertised.

Job vacancy trend for Data Engineering in the UK

Data Engineering
Salary Trend

3-month moving average salary quoted in jobs citing Data Engineering.

Salary trend for Data Engineering in the UK

Data Engineering
Salary Histogram

Salary distribution for jobs citing Data Engineering over the 6 months to 19 October 2020.

Salary histogram for Data Engineering in the UK

Data Engineering
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Engineering within the UK over the 6 months to 19 October 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +115 416 £60,000 -14.29% 121
London +64 263 £67,500 -10.00% 64
UK excluding London +188 183 £55,000 +4.76% 76
Work from Home +85 65 £60,000 +9.09% 27
North of England +130 47 £55,000 -3.97% 32
South East +192 40 £57,500 -8.00% 7
Midlands +128 37 £55,000 +7.32% 6
North West +124 34 £56,250 -19.64% 16
West Midlands +91 23 £55,000 +10.00% 2
Scotland +85 20 £50,000 +4.71% 13
South West +71 17 £45,000 -5.26% 1
East Midlands +93 14 £50,000 -4.76% 4
Yorkshire +103 12 £50,000 -12.70% 10
Northern Ireland - 12 £46,250 - 5
East of England +129 8 £55,000 -26.67% 10
Wales +45 2 £40,253 +0.63% 2
North East +8 1 £42,500 -3.80% 6

For the 6 months to 19 October 2020, IT jobs citing Data Engineering also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Data Engineering.

1 273 (59.74%) Python
2 241 (52.74%) SQL
3 221 (48.36%) Azure
4 200 (43.76%) AWS
5 185 (40.48%) Business Intelligence
6 177 (38.73%) Analytics
7 175 (38.29%) Data Warehouse
8 156 (34.14%) ETL
9 154 (33.70%) Agile Software Development
10 121 (26.48%) Data Modelling
11 111 (24.29%) Big Data
12 109 (23.85%) Data Science
13 108 (23.63%) Data Lake
13 108 (23.63%) Power Platform
13 108 (23.63%) Power BI
14 106 (23.19%) Finance
15 91 (19.91%) Apache Spark
16 87 (19.04%) NoSQL
17 85 (18.60%) Machine Learning
17 85 (18.60%) GCP
17 85 (18.60%) DevOps
18 82 (17.94%) Java
19 80 (17.51%) SQL Server
20 76 (16.63%) Kafka
21 75 (16.41%) Artificial Intelligence
22 71 (15.54%) Snowflake Schema
22 71 (15.54%) Software Engineering
23 70 (15.32%) Amazon Redshift
24 68 (14.88%) Tableau
25 67 (14.66%) Hadoop

Data Engineering
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 91 (19.91%) Apache Spark
2 34 (7.44%) Apache
3 18 (3.94%) SAS
4 17 (3.72%) Apache Airflow
5 9 (1.97%) Cloud Foundry
6 6 (1.31%) Jupyter
7 5 (1.09%) Confluence
7 5 (1.09%) Elasticsearch
7 5 (1.09%) SharePoint
8 3 (0.66%) Apache Synapse
9 2 (0.44%) SharePoint 2010
9 2 (0.44%) SharePoint 2013
10 1 (0.22%) Apache Solr
10 1 (0.22%) Luigi
10 1 (0.22%) WebSphere
10 1 (0.22%) WebSphere Application Server
Applications
1 5 (1.09%) Microsoft Excel
Business Applications
1 2 (0.44%) Markit EDM
2 1 (0.22%) Dynamics CRM
Cloud Services
1 221 (48.36%) Azure
2 200 (43.76%) AWS
3 108 (23.63%) Power Platform
4 85 (18.60%) GCP
5 49 (10.72%) Azure Data Factory
6 37 (8.10%) Amazon S3
7 25 (5.47%) SaaS
8 19 (4.16%) Serverless
9 14 (3.06%) Amazon EMR
9 14 (3.06%) PaaS
10 11 (2.41%) Amazon EC2
10 11 (2.41%) Amazon SQS
11 9 (1.97%) AWS Lambda
11 9 (1.97%) GitHub
12 8 (1.75%) AWS CloudFormation
12 8 (1.75%) Cloud Computing
12 8 (1.75%) PowerApps
13 7 (1.53%) AdWords
13 7 (1.53%) Amazon SageMaker
13 7 (1.53%) Google Analytics
Communications & Networking
1 2 (0.44%) Intranet
2 1 (0.22%) DNS
2 1 (0.22%) Firewall
2 1 (0.22%) HTTP
2 1 (0.22%) HTTPS
2 1 (0.22%) Kerberos
2 1 (0.22%) SSH
2 1 (0.22%) SSL
Database & Business Intelligence
1 175 (38.29%) Data Warehouse
2 111 (24.29%) Big Data
3 108 (23.63%) Data Lake
3 108 (23.63%) Power BI
4 87 (19.04%) NoSQL
5 80 (17.51%) SQL Server
6 71 (15.54%) Snowflake Schema
7 70 (15.32%) Amazon Redshift
8 67 (14.66%) Hadoop
9 50 (10.94%) Azure SQL Database
10 45 (9.85%) SQL Server Integration Services
11 38 (8.32%) Looker
12 34 (7.44%) Amazon Athena
12 34 (7.44%) BigQuery
13 31 (6.78%) Azure SQL Data Warehouse
13 31 (6.78%) MongoDB
13 31 (6.78%) Relational Database
14 30 (6.56%) SQL Server Analysis Services
15 23 (5.03%) Apache Hive
16 20 (4.38%) PostgreSQL
Development Applications
1 56 (12.25%) Git (software)
2 39 (8.53%) Jenkins
3 26 (5.69%) JIRA
4 16 (3.50%) Bitbucket
5 14 (3.06%) Artifactory
5 14 (3.06%) Maven
6 5 (1.09%) GitLab
7 4 (0.88%) GoCD
8 3 (0.66%) Team Foundation Server
9 2 (0.44%) CircleCI
9 2 (0.44%) Cucumber
9 2 (0.44%) JUnit
10 1 (0.22%) Atlassian Bamboo
10 1 (0.22%) Flyway
10 1 (0.22%) Liquibase
10 1 (0.22%) Mockito
10 1 (0.22%) SonarQube
10 1 (0.22%) Vagrant
10 1 (0.22%) Visual Studio
General
1 106 (23.19%) Finance
2 34 (7.44%) Marketing
3 16 (3.50%) Banking
4 15 (3.28%) Advertising
5 12 (2.63%) Retail
6 11 (2.41%) Front Office
7 9 (1.97%) Pharmaceutical
8 8 (1.75%) Public Sector
9 6 (1.31%) Law
10 4 (0.88%) Digital Economy
10 4 (0.88%) Manufacturing
11 2 (0.44%) Games
11 2 (0.44%) Retail Banking
Job Titles
1 248 (54.27%) Data Engineer
2 74 (16.19%) Developer
3 33 (7.22%) Python Engineer
4 32 (7.00%) Senior Data Engineer
5 31 (6.78%) Azure Engineer
6 26 (5.69%) Architect
7 20 (4.38%) Data Scientist
8 19 (4.16%) AWS Engineer
9 18 (3.94%) Data Architect
9 18 (3.94%) SQL Developer
10 15 (3.28%) Data Analytics Engineer
11 14 (3.06%) Cloud Engineer
12 13 (2.84%) Big Data Engineer
12 13 (2.84%) Data Warehouse Developer
13 12 (2.63%) Software Engineer
14 11 (2.41%) Data Manager
15 10 (2.19%) Business Intelligence Developer
16 9 (1.97%) Consultant
16 9 (1.97%) Senior Data Scientist
16 9 (1.97%) Senior Developer
Libraries, Frameworks & Software Standards
1 76 (16.63%) Kafka
2 20 (4.38%) .NET
3 18 (3.94%) Natural Language Processing
3 18 (3.94%) PySpark
4 17 (3.72%) React
5 13 (2.84%) CSS
6 12 (2.63%) HTML
6 12 (2.63%) REST
7 11 (2.41%) HTML5
7 11 (2.41%) NumPy
7 11 (2.41%) Pandas
8 10 (2.19%) ASP.NET
8 10 (2.19%) ASP.NET Web API
8 10 (2.19%) Web Services
9 9 (1.97%) TensorFlow
10 8 (1.75%) ASP.NET MVC
10 8 (1.75%) Django
11 7 (1.53%) Spark Streaming
12 6 (1.31%) Flask
12 6 (1.31%) scikit-learn
Miscellaneous
1 56 (12.25%) Analytical Skills
2 53 (11.60%) Algorithms
3 26 (5.69%) Greenfield Project
4 24 (5.25%) Data Structures
5 17 (3.72%) Clustering
6 10 (2.19%) Management Information System
6 10 (2.19%) User Experience
7 8 (1.75%) Social Media
8 6 (1.31%) Self-Motivation
9 5 (1.09%) Data Centre
9 5 (1.09%) Hedge funds
9 5 (1.09%) Public Cloud
10 4 (0.88%) Mobile App
10 4 (0.88%) Pricing Data
11 3 (0.66%) Blog
11 3 (0.66%) Life Science
12 2 (0.44%) Linux Command Line
12 2 (0.44%) PKI
12 2 (0.44%) Private Cloud
12 2 (0.44%) Reinsurance
Operating Systems
1 18 (3.94%) Linux
2 10 (2.19%) Unix
3 6 (1.31%) Windows
4 1 (0.22%) CentOS
4 1 (0.22%) Windows Server
Processes & Methodologies
1 185 (40.48%) Business Intelligence
2 177 (38.73%) Analytics
3 156 (34.14%) ETL
4 154 (33.70%) Agile Software Development
5 121 (26.48%) Data Modelling
6 109 (23.85%) Data Science
7 85 (18.60%) DevOps
7 85 (18.60%) Machine Learning
8 75 (16.41%) Artificial Intelligence
9 71 (15.54%) Software Engineering
10 63 (13.79%) Visualisation
11 58 (12.69%) Data Architecture
12 49 (10.72%) Data Analytics
13 42 (9.19%) Data Management
14 36 (7.88%) Data Visualisation
14 36 (7.88%) Microservices
15 35 (7.66%) Fintech
16 34 (7.44%) Computer Science
17 32 (7.00%) Data Analysis
17 32 (7.00%) Mentoring
Programming Languages
1 273 (59.74%) Python
2 241 (52.74%) SQL
3 82 (17.94%) Java
4 58 (12.69%) C#
5 51 (11.16%) R
6 47 (10.28%) Scala
7 33 (7.22%) JavaScript
8 22 (4.81%) C++
9 17 (3.72%) C
10 15 (3.28%) Data Analysis Expressions (DAX)
10 15 (3.28%) PL/SQL
11 13 (2.84%) T-SQL
12 12 (2.63%) Go
13 7 (1.53%) TypeScript
14 6 (1.31%) Bash Shell
15 4 (0.88%) Ruby
16 3 (0.66%) Julia
16 3 (0.66%) PowerShell
16 3 (0.66%) Shell Script
16 3 (0.66%) VBA
Qualifications
1 51 (11.16%) Degree
2 18 (3.94%) Computer Science Degree
3 15 (3.28%) Security Cleared
4 12 (2.63%) Numerate Degree
4 12 (2.63%) SC Cleared
5 9 (1.97%) AWS Certification
6 8 (1.75%) PhD
7 7 (1.53%) Microsoft Certification
8 2 (0.44%) DV Cleared
8 2 (0.44%) Master's Degree
Quality Assurance & Compliance
1 35 (7.66%) Data Quality
2 18 (3.94%) QA
3 2 (0.44%) GDPR
System Software
1 32 (7.00%) Docker
2 9 (1.97%) HDFS
3 6 (1.31%) Apache Flume
3 6 (1.31%) Apache ZooKeeper
4 2 (0.44%) Active Directory
5 1 (0.22%) Virtual Machines
5 1 (0.22%) VirtualBox
5 1 (0.22%) zsh
Systems Management
1 53 (11.60%) Kubernetes
2 43 (9.41%) Terraform
3 27 (5.91%) Ansible
4 9 (1.97%) Hadoop YARN
5 4 (0.88%) Nagios
6 3 (0.66%) Ambari
6 3 (0.66%) Oozie
7 2 (0.44%) Single Sign-On
7 2 (0.44%) WebSphere Cast Iron
8 1 (0.22%) Docker Swarm
8 1 (0.22%) Opscode Chef
8 1 (0.22%) Puppet
Vendors
1 68 (14.88%) Tableau
2 62 (13.57%) Microsoft
3 56 (12.25%) Databricks
4 36 (7.88%) Cloudera
4 36 (7.88%) Google
5 29 (6.35%) Oracle
6 15 (3.28%) Talend
7 8 (1.75%) Alteryx
7 8 (1.75%) Palantir
8 7 (1.53%) DoubleClick
8 7 (1.53%) Splunk
9 5 (1.09%) Hortonworks
10 4 (0.88%) Foundry
10 4 (0.88%) Percona
10 4 (0.88%) StreamSets
11 3 (0.66%) Confluent
11 3 (0.66%) Dataiku
11 3 (0.66%) Informatica
11 3 (0.66%) Spotify
11 3 (0.66%) Sun