Period
to 3 April 2020

The following table provides summary statistics for permanent job vacancies with a requirement for Data Engineering skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Engineering over the 6 months to 3 April 2020 with a comparison to the same period in the previous 2 years.

Data Engineering
UK
6 months to
3 Apr 2020
Same period 2019 Same period 2018
Rank 387 435 709
Rank change year-on-year +48 +274 +267
Permanent jobs citing Data Engineering 872 1,018 553
As % of all permanent jobs advertised in the UK 0.75% 0.69% 0.31%
As % of the Processes & Methodologies category 0.81% 0.74% 0.34%
Number of salaries quoted 734 790 471
Median annual salary £72,500 £67,500 £67,500
Median salary % change year-on-year +7.41% - -3.57%
10th Percentile £43,750 £45,000 £45,000
90th Percentile £101,250 £102,500 £100,000
UK excluding London median annual salary £59,000 £60,000 £55,000
% change year-on-year -1.67% +9.09% -4.35%

Data Engineering is in the Processes and Methodologies category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for process or methodology skills.

All Process and Methodology Skills
UK
Permanent vacancies with a requirement for process or methodology skills 107,190 136,662 163,231
As % of all permanent IT jobs advertised in the UK 91.92% 92.19% 92.19%
Number of salaries quoted 84,977 107,554 132,868
Median annual salary £55,000 £52,500 £50,000
Median salary % change year-on-year +4.76% +5.00% +5.26%
10th Percentile £31,250 £30,000 £29,000
90th Percentile £88,750 £85,000 £81,250
UK excluding London median annual salary £47,500 £45,000 £45,000
% change year-on-year +5.56% - +5.88%

Data Engineering
Job Vacancy Trend

Job postings citing Data Engineering as a proportion of all IT jobs advertised.

Job vacancy trend for Data Engineering in the UK

Data Engineering
Salary Trend

3-month moving average salary quoted in jobs citing Data Engineering.

Salary trend for Data Engineering in the UK

Data Engineering
Salary Histogram

Salary distribution for jobs citing Data Engineering over the 6 months to 3 April 2020.

Salary histogram for Data Engineering in the UK

Data Engineering
Top 18 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Engineering within the UK over the 6 months to 3 April 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +54 826 £75,000 +11.11% 160
London +55 567 £77,500 +6.90% 98
UK excluding London -11 300 £59,000 -1.67% 68
North of England +27 62 £55,000 -15.38% 13
South East +13 62 £63,000 -6.67% 13
Work from Home +43 55 £75,000 -14.29% 12
East of England +87 53 £75,000 +25.00% 18
Midlands +57 49 £60,000 +12.15% 12
West Midlands +60 34 £61,250 -5.38% 10
Scotland +22 33 £50,000 - 4
South West -22 31 £45,000 -22.41% 6
Yorkshire +43 29 £55,000 -13.19% 2
North West -1 23 £48,750 -30.36% 9
East Midlands +27 15 £55,000 +22.22% 2
North East +6 10 £56,085 +13.68% 2
Wales -6 7 £32,000 -28.89% 2
Channel Islands - 2 £60,000 -
Northern Ireland - 1 £35,000 -

For the 6 months to 3 April 2020, IT jobs citing Data Engineering also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Data Engineering.

1 555 (63.65%) Python
2 447 (51.26%) SQL
3 388 (44.50%) AWS
4 371 (42.55%) Analytics
5 342 (39.22%) Agile Software Development
6 334 (38.30%) Azure
7 324 (37.16%) Data Science
8 310 (35.55%) ETL
9 306 (35.09%) Big Data
10 305 (34.98%) Apache Spark
11 277 (31.77%) Business Intelligence
12 251 (28.78%) Data Warehouse
13 238 (27.29%) Java
14 235 (26.95%) Hadoop
15 215 (24.66%) Machine Learning
15 215 (24.66%) Data Modelling
16 199 (22.82%) Finance
17 184 (21.10%) Kafka
18 181 (20.76%) Scala
18 181 (20.76%) GCP
19 175 (20.07%) NoSQL
20 172 (19.72%) DevOps
21 154 (17.66%) SQL Server
22 150 (17.20%) Power BI
22 150 (17.20%) Power Platform
23 145 (16.63%) Software Engineering
24 135 (15.48%) Data Analytics
24 135 (15.48%) Visualisation
25 134 (15.37%) Data Lake
26 131 (15.02%) Docker

Data Engineering
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 305 (34.98%) Apache Spark
2 63 (7.22%) Apache Airflow
3 56 (6.42%) Apache
4 34 (3.90%) Elasticsearch
5 29 (3.33%) SAS
6 19 (2.18%) Confluence
7 14 (1.61%) Luigi
8 11 (1.26%) Apache Pig
9 6 (0.69%) Jupyter
10 4 (0.46%) Apache Solr
11 3 (0.34%) Apache Synapse
12 2 (0.23%) Adobe Experience Manager
13 1 (0.11%) BizTalk Server
13 1 (0.11%) OpenWhisk
13 1 (0.11%) Tomcat
Applications
1 11 (1.26%) Microsoft Excel
2 1 (0.11%) Microsoft Office
2 1 (0.11%) Spreadsheet
2 1 (0.11%) SPSS
Business Applications
1 5 (0.57%) Odoo
2 2 (0.23%) Dynamics CRM
2 2 (0.23%) Raiser's Edge
3 1 (0.11%) Oracle Financials
3 1 (0.11%) Primavera
3 1 (0.11%) R3 Corda
3 1 (0.11%) SAP Oil and Gas
Cloud Services
1 388 (44.50%) AWS
2 334 (38.30%) Azure
3 181 (20.76%) GCP
4 150 (17.20%) Power Platform
5 86 (9.86%) Serverless
6 78 (8.94%) Amazon EMR
7 75 (8.60%) Azure Data Factory
8 65 (7.45%) Amazon S3
9 64 (7.34%) AWS Lambda
10 55 (6.31%) Amazon Kinesis
11 35 (4.01%) SaaS
12 28 (3.21%) PaaS
13 21 (2.41%) Amazon EC2
14 19 (2.18%) IBM Cloud
15 13 (1.49%) Azure Stack
16 12 (1.38%) Amazon Glacier
16 12 (1.38%) Azure Machine Learning
17 11 (1.26%) Amazon SageMaker
17 11 (1.26%) AWS CloudFormation
17 11 (1.26%) Google Analytics
Communications & Networking
1 20 (2.29%) HTTP
2 6 (0.69%) Internet
3 4 (0.46%) MQTT
3 4 (0.46%) SSL
4 3 (0.34%) HTTPS
4 3 (0.34%) Kerberos
4 3 (0.34%) VPN
5 2 (0.23%) Firewall
5 2 (0.23%) FTP
6 1 (0.11%) Broadband
6 1 (0.11%) IPFS
6 1 (0.11%) VoIP
Database & Business Intelligence
1 306 (35.09%) Big Data
2 251 (28.78%) Data Warehouse
3 235 (26.95%) Hadoop
4 175 (20.07%) NoSQL
5 154 (17.66%) SQL Server
6 150 (17.20%) Power BI
7 134 (15.37%) Data Lake
8 125 (14.33%) Amazon Redshift
9 112 (12.84%) Apache Hive
10 103 (11.81%) SQL Server Integration Services
11 90 (10.32%) PostgreSQL
12 78 (8.94%) Relational Database
13 70 (8.03%) MongoDB
14 66 (7.57%) Snowflake Schema
15 65 (7.45%) Neo4j
16 62 (7.11%) SQL Server Analysis Services
17 61 (7.00%) Apache Cassandra
17 61 (7.00%) Azure SQL Database
18 56 (6.42%) Amazon Athena
19 53 (6.08%) Amazon DynamoDB
Development Applications
1 104 (11.93%) Git (software)
2 93 (10.67%) Jenkins
3 39 (4.47%) Maven
4 34 (3.90%) JIRA
5 32 (3.67%) Atlassian Bamboo
6 27 (3.10%) Vagrant
7 26 (2.98%) Artifactory
7 26 (2.98%) Bitbucket
8 16 (1.83%) Visual Studio
9 14 (1.61%) Visual Studio Team System
10 12 (1.38%) Ant
10 12 (1.38%) TeamCity
11 10 (1.15%) JUnit
12 7 (0.80%) Gradle
13 6 (0.69%) Octopus Deploy
13 6 (0.69%) Sonatype Nexus
14 5 (0.57%) Flyway
14 5 (0.57%) IBM UrbanCode
14 5 (0.57%) kappa
14 5 (0.57%) Liquibase
General
1 199 (22.82%) Finance
2 59 (6.77%) Marketing
3 58 (6.65%) Retail
4 24 (2.75%) Manufacturing
5 23 (2.64%) Banking
6 19 (2.18%) Legal
7 18 (2.06%) Public Sector
8 17 (1.95%) Financial Institution
9 16 (1.83%) Law
10 11 (1.26%) Advertising
11 9 (1.03%) Automotive
11 9 (1.03%) Games
12 8 (0.92%) Front Office
13 6 (0.69%) Aerospace
13 6 (0.69%) Telecoms
14 5 (0.57%) Electronics
15 3 (0.34%) Pharmaceutical
16 2 (0.23%) Military
16 2 (0.23%) Police
16 2 (0.23%) Retail Banking
Job Titles
1 391 (44.84%) Data Engineer
2 138 (15.83%) Architect
3 104 (11.93%) Data Architect
4 92 (10.55%) Developer
5 82 (9.40%) Senior Data Engineer
6 57 (6.54%) Python Engineer
7 43 (4.93%) Big Data Engineer
8 41 (4.70%) Azure Engineer
9 36 (4.13%) Software Engineer
10 35 (4.01%) Analyst
11 33 (3.78%) AWS Engineer
11 33 (3.78%) Data Scientist
12 32 (3.67%) Solutions Architect
13 30 (3.44%) Cloud Engineer
14 26 (2.98%) Java Developer
15 25 (2.87%) Big Data Architect
15 25 (2.87%) Consultant
16 22 (2.52%) Python Developer
17 21 (2.41%) Platform Engineer
18 20 (2.29%) Team Leader
Libraries, Frameworks & Software Standards
1 184 (21.10%) Kafka
2 44 (5.05%) .NET
3 41 (4.70%) React
4 40 (4.59%) Web Services
5 39 (4.47%) RESTful
6 37 (4.24%) Natural Language Processing
7 36 (4.13%) TensorFlow
8 30 (3.44%) Django
9 29 (3.33%) Spark Streaming
10 28 (3.21%) Pandas
11 27 (3.10%) REST
12 26 (2.98%) AngularJS
12 26 (2.98%) MDX
13 23 (2.64%) Node.js
14 20 (2.29%) NumPy
15 19 (2.18%) scikit-learn
16 18 (2.06%) Akka
16 18 (2.06%) Spring
17 17 (1.95%) Flask
17 17 (1.95%) HTML
Miscellaneous
1 131 (15.02%) Analytical Skills
2 54 (6.19%) Algorithms
3 53 (6.08%) Greenfield Project
4 52 (5.96%) Data Structures
5 23 (2.64%) Cloud Native
6 17 (1.95%) Management Information System
7 16 (1.83%) Acoustics
7 16 (1.83%) Blog
8 14 (1.61%) Hedge funds
9 12 (1.38%) Virtual Team
10 11 (1.26%) User Experience
11 10 (1.15%) Public Cloud
12 8 (0.92%) Data Centre
12 8 (0.92%) Hybrid Cloud
13 7 (0.80%) Self-Motivation
14 4 (0.46%) Credit Risk
14 4 (0.46%) Mobile App
14 4 (0.46%) Semantic Web
14 4 (0.46%) Social Media
15 3 (0.34%) Digital Media
Operating Systems
1 106 (12.16%) Linux
2 69 (7.91%) Unix
3 7 (0.80%) Windows
4 2 (0.23%) CentOS
4 2 (0.23%) Red Hat Enterprise Linux
4 2 (0.23%) Windows Server
Processes & Methodologies
1 371 (42.55%) Analytics
2 342 (39.22%) Agile Software Development
3 324 (37.16%) Data Science
4 310 (35.55%) ETL
5 277 (31.77%) Business Intelligence
6 215 (24.66%) Data Modelling
6 215 (24.66%) Machine Learning
7 172 (19.72%) DevOps
8 145 (16.63%) Software Engineering
9 135 (15.48%) Data Analytics
9 135 (15.48%) Visualisation
10 130 (14.91%) Artificial Intelligence
11 105 (12.04%) Continuous Integration
12 96 (11.01%) Data Management
13 93 (10.67%) Computer Science
14 90 (10.32%) Mentoring
15 77 (8.83%) Microservices
16 74 (8.49%) Data Integration
16 74 (8.49%) Scrum
17 70 (8.03%) Data Visualisation
Programming Languages
1 555 (63.65%) Python
2 447 (51.26%) SQL
3 238 (27.29%) Java
4 181 (20.76%) Scala
5 102 (11.70%) C#
6 73 (8.37%) R
7 46 (5.28%) C++
8 45 (5.16%) JavaScript
9 35 (4.01%) C
10 33 (3.78%) PL/SQL
11 27 (3.10%) Shell Script
12 25 (2.87%) T-SQL
13 19 (2.18%) Go
14 15 (1.72%) Bash Shell
15 13 (1.49%) Java 8
16 12 (1.38%) PowerShell
16 12 (1.38%) Ruby
17 9 (1.03%) Data Analysis Expressions (DAX)
17 9 (1.03%) MATLAB
18 7 (0.80%) VBA
Qualifications
1 72 (8.26%) Degree
2 29 (3.33%) PhD
3 21 (2.41%) Computer Science Degree
4 18 (2.06%) Security Cleared
5 17 (1.95%) SC Cleared
6 7 (0.80%) DV Cleared
7 4 (0.46%) AWS Certification
7 4 (0.46%) Microsoft Certification
7 4 (0.46%) Numerate Degree
8 3 (0.34%) Master's Degree
8 3 (0.34%) MCSA
8 3 (0.34%) MCSE
9 2 (0.23%) MBA
9 2 (0.23%) PMI Certification
10 1 (0.11%) MCSD
10 1 (0.11%) OCP
10 1 (0.11%) Oracle Certification
Quality Assurance & Compliance
1 47 (5.39%) Data Quality
2 26 (2.98%) GDPR
3 23 (2.64%) QA
4 8 (0.92%) ISO/IEC 27001
5 4 (0.46%) Actionable Recommendations
6 3 (0.34%) SLA
7 1 (0.11%) COBIT
7 1 (0.11%) NIST
System Software
1 131 (15.02%) Docker
2 48 (5.50%) HDFS
3 28 (3.21%) Apache Flume
4 27 (3.10%) VirtualBox
5 4 (0.46%) Virtual Machines
6 3 (0.34%) Apache ZooKeeper
7 1 (0.11%) Active Directory
7 1 (0.11%) Alluxio
7 1 (0.11%) Ceph
7 1 (0.11%) etcd
7 1 (0.11%) Gluster
Systems Management
1 114 (13.07%) Kubernetes
2 94 (10.78%) Terraform
3 75 (8.60%) Ansible
4 34 (3.90%) Docker Swarm
5 33 (3.78%) Oozie
6 15 (1.72%) Puppet
7 13 (1.49%) Hadoop YARN
8 9 (1.03%) Opscode Chef
8 9 (1.03%) Rancher
9 7 (0.80%) Kibana
10 5 (0.57%) Kubeflow
11 4 (0.46%) Istio
11 4 (0.46%) Prometheus
12 3 (0.34%) Ambari
12 3 (0.34%) Grafana
12 3 (0.34%) linkerd
12 3 (0.34%) Mesos
12 3 (0.34%) Single Sign-On
13 2 (0.23%) Opsview
13 2 (0.23%) Salt
Vendors
1 130 (14.91%) Tableau
2 127 (14.56%) Google
3 110 (12.61%) Databricks
4 104 (11.93%) Microsoft
5 78 (8.94%) Cloudera
6 59 (6.77%) Oracle
7 38 (4.36%) Hortonworks
8 36 (4.13%) Talend
9 33 (3.78%) Informatica
10 22 (2.52%) IBM
11 10 (1.15%) Dataiku
11 10 (1.15%) SnapLogic
12 9 (1.03%) Adobe
13 8 (0.92%) Salesforce.com
13 8 (0.92%) Splunk
14 7 (0.80%) Hitachi
14 7 (0.80%) Vertica
15 6 (0.69%) SAP
16 5 (0.57%) WorldPay
17 4 (0.46%) MuleSoft