Period
to 21 November 2019

The following table provides summary statistics for permanent job vacancies with a requirement for Data Engineering skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Engineering over the 6 months to 21 November 2019 with a comparison to the same period in the previous 2 years.

Data Engineering
UK
6 months to
21 Nov 2019
Same period 2018 Same period 2017
Rank 449 565 904
Rank change year-on-year +116 +339 +168
Permanent jobs citing Data Engineering 889 793 310
As % of all permanent IT jobs advertised in the UK 0.65% 0.49% 0.17%
As % of the Processes & Methodologies category 0.71% 0.53% 0.19%
Number of salaries quoted 694 537 254
UK median annual salary £70,000 £67,500 £68,750
Median salary % change year-on-year +3.70% -1.82% +1.85%
10th Percentile £40,000 £46,250 £42,500
90th Percentile £100,000 £100,000 £110,000
UK excluding London median annual salary £52,500 £62,138 £50,000
% change year-on-year -15.51% +24.28% -4.76%

Data Engineering is in the Processes and Methodologies category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for process or methodology skills.

All Process and Methodology Skills
UK
Permanent vacancies with a requirement for process or methodology skills 125,291 149,786 166,921
As % of all permanent IT jobs advertised in the UK 92.07% 91.83% 92.18%
Number of salaries quoted 99,093 119,110 132,132
UK median annual salary £52,500 £50,000 £50,000
Median salary % change year-on-year +5.00% - +4.17%
10th Percentile £30,000 £29,000 £28,000
90th Percentile £87,500 £85,000 £80,000
UK excluding London median annual salary £45,000 £45,000 £43,000
% change year-on-year - +4.65% +1.18%

Data Engineering
Job Vacancy Trend

Job postings citing Data Engineering as a percentage of all IT jobs advertised.

Job vacancy trend for Data Engineering in the UK

Data Engineering
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Data Engineering.

Salary trend for Data Engineering in the UK

Data Engineering
Salary Histogram

The salary distribution of IT jobs citing Data Engineering over the 6 months to 21 November 2019.

Salary histogram for Data Engineering in the UK

Data Engineering
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Engineering within the UK over the 6 months to 21 November 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +116 841 £70,000 +3.70% 117
London +178 561 £75,000 +7.14% 78
UK excluding London -24 323 £52,500 -15.51% 44
North of England +77 92 £57,275 -18.18% 10
South West -4 64 £47,500 -20.83% 6
Midlands +48 49 £50,000 -21.40% 7
South East -76 49 £62,500 -10.71% 8
Yorkshire +53 44 £55,000 -21.82% 6
West Midlands +37 33 £50,000 -22.76% 3
North West +12 32 £70,000 - 3
Scotland +10 30 £50,000 - 2
East of England +72 24 £75,000 +7.14% 8
North East +39 16 £45,412 -30.89% 1
East Midlands +10 16 £51,250 -4.21% 4
Wales +9 13 £40,000 -11.11% 3
Northern Ireland - 1 £35,000 -
Channel Islands - 1 £60,000 -

For the 6 months to 21 November 2019, IT jobs citing Data Engineering also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Data Engineering.

1 531 (59.73%) Python
2 476 (53.54%) SQL
3 391 (43.98%) Agile Software Development
4 378 (42.52%) Big Data
4 378 (42.52%) Data Science
5 367 (41.28%) AWS
6 357 (40.16%) Apache Spark
7 356 (40.04%) ETL
8 335 (37.68%) Analytics
9 306 (34.42%) Hadoop
10 271 (30.48%) Azure
11 266 (29.92%) Scala
12 256 (28.80%) Java
13 234 (26.32%) Data Warehouse
14 221 (24.86%) Business Intelligence
15 199 (22.38%) Data Modelling
16 197 (22.16%) DevOps
17 184 (20.70%) Machine Learning
18 174 (19.57%) GCP
19 168 (18.90%) NoSQL
20 163 (18.34%) Kafka
21 156 (17.55%) Mentoring
22 155 (17.44%) Apache Hive
23 148 (16.65%) Finance
24 138 (15.52%) Linux
25 136 (15.30%) Software Engineering
26 133 (14.96%) Google
27 129 (14.51%) Docker
28 122 (13.72%) Computer Science
29 120 (13.50%) Analytical Skills

Data Engineering
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 357 (40.16%) Apache Spark
2 82 (9.22%) Apache
3 43 (4.84%) SAS
4 35 (3.94%) Apache Airflow
5 26 (2.92%) Elasticsearch
6 12 (1.35%) Apache Pig
7 10 (1.12%) Confluence
7 10 (1.12%) Jupyter
8 9 (1.01%) Luigi
9 6 (0.67%) Apache Solr
10 5 (0.56%) ArcGIS
11 3 (0.34%) nginx
12 1 (0.11%) BizTalk Server
12 1 (0.11%) Tomcat
12 1 (0.11%) WebLogic Portal
Applications
1 7 (0.79%) Microsoft Excel
2 1 (0.11%) Microsoft Office
2 1 (0.11%) Microsoft PowerPoint
Business Applications
1 4 (0.45%) Odoo
2 1 (0.11%) Cognos Business Intelligence
2 1 (0.11%) FINCAD Analytics
2 1 (0.11%) SAP GTS
Cloud Services
1 367 (41.28%) AWS
2 271 (30.48%) Azure
3 174 (19.57%) GCP
4 80 (9.00%) Serverless
5 77 (8.66%) Amazon S3
6 68 (7.65%) AWS Lambda
7 67 (7.54%) Power Platform
8 66 (7.42%) Amazon EMR
9 65 (7.31%) Amazon Kinesis
10 40 (4.50%) PaaS
11 21 (2.36%) IaaS
12 20 (2.25%) Azure Data Factory
13 18 (2.02%) GitHub
14 15 (1.69%) SaaS
15 12 (1.35%) Amazon Cognito
15 12 (1.35%) CloudFront
16 10 (1.12%) IBM Cloud
17 9 (1.01%) Amazon SQS
17 9 (1.01%) AWS CloudFormation
18 6 (0.67%) Cloud Computing
Communications & Networking
1 38 (4.27%) HTTP
2 6 (0.67%) Internet
3 5 (0.56%) FTP
3 5 (0.56%) HTTPS
3 5 (0.56%) Kerberos
3 5 (0.56%) SSL
3 5 (0.56%) VPN
4 4 (0.45%) Broadband
4 4 (0.45%) Firewall
5 1 (0.11%) VoIP
Database & Business Intelligence
1 378 (42.52%) Big Data
2 306 (34.42%) Hadoop
3 234 (26.32%) Data Warehouse
4 168 (18.90%) NoSQL
5 155 (17.44%) Apache Hive
6 113 (12.71%) Amazon Redshift
7 108 (12.15%) Data Lake
8 106 (11.92%) SQL Server
9 101 (11.36%) Apache Cassandra
10 87 (9.79%) MongoDB
11 77 (8.66%) Relational Database
12 67 (7.54%) Power BI
13 64 (7.20%) PostgreSQL
14 49 (5.51%) Amazon DynamoDB
15 47 (5.29%) BigQuery
16 43 (4.84%) Impala
17 40 (4.50%) Amazon Athena
17 40 (4.50%) SQL Server Integration Services
18 39 (4.39%) HBase
18 39 (4.39%) Neo4j
Development Applications
1 109 (12.26%) Jenkins
2 94 (10.57%) Git (software)
3 54 (6.07%) JIRA
4 41 (4.61%) Maven
5 35 (3.94%) Bitbucket
6 30 (3.37%) Artifactory
7 21 (2.36%) ERwin
7 21 (2.36%) RStudio
8 19 (2.14%) Atlassian Bamboo
9 16 (1.80%) Selenium
10 15 (1.69%) Team Foundation Server
11 14 (1.57%) Vagrant
12 10 (1.12%) Cucumber
13 9 (1.01%) GitLab
14 8 (0.90%) kappa
15 7 (0.79%) Travis CI
16 6 (0.67%) HP UFT
17 5 (0.56%) Ant
17 5 (0.56%) Gradle
17 5 (0.56%) Subversion
General
1 148 (16.65%) Finance
2 51 (5.74%) Retail
3 29 (3.26%) Marketing
4 28 (3.15%) Legal
5 21 (2.36%) Public Sector
6 15 (1.69%) Banking
7 12 (1.35%) Advertising
8 11 (1.24%) Telecoms
9 10 (1.12%) Manufacturing
10 8 (0.90%) Front Office
11 7 (0.79%) Games
11 7 (0.79%) Law
11 7 (0.79%) Pharmaceutical
12 6 (0.67%) Financial Institution
13 5 (0.56%) Electronics
13 5 (0.56%) Publishing
14 3 (0.34%) Retail Banking
15 2 (0.22%) Billing
15 2 (0.22%) Investment Banking
16 1 (0.11%) Multimedia
Job Titles
1 426 (47.92%) Data Engineer
2 131 (14.74%) Developer
3 123 (13.84%) Senior Data Engineer
4 80 (9.00%) Python Engineer
5 79 (8.89%) Architect
6 63 (7.09%) Big Data Engineer
7 57 (6.41%) Data Scientist
8 52 (5.85%) Data Architect
9 39 (4.39%) Senior Developer
10 36 (4.05%) AWS Engineer
11 28 (3.15%) Analyst
12 25 (2.81%) Cloud Engineer
12 25 (2.81%) Senior Data Scientist
13 23 (2.59%) Engineering Manager
13 23 (2.59%) Principal Engineer
13 23 (2.59%) Scala Developer
14 22 (2.47%) Python Developer
15 20 (2.25%) Java Developer
15 20 (2.25%) Junior
15 20 (2.25%) Software Engineer
Libraries, Frameworks & Software Standards
1 163 (18.34%) Kafka
2 45 (5.06%) REST
3 38 (4.27%) Web Services
4 35 (3.94%) Akka
5 30 (3.37%) JSON
5 30 (3.37%) Scalaz
6 27 (3.04%) Pandas
7 26 (2.92%) React
8 24 (2.70%) NumPy
9 23 (2.59%) LAPP Stack
10 22 (2.47%) .NET
10 22 (2.47%) TensorFlow
11 21 (2.36%) ADO
11 21 (2.36%) Node.js
12 20 (2.25%) Natural Language Processing
13 19 (2.14%) Apache Beam
14 18 (2.02%) scikit-learn
15 17 (1.91%) PySpark
16 16 (1.80%) AngularJS
16 16 (1.80%) RESTful
Miscellaneous
1 120 (13.50%) Analytical Skills
2 48 (5.40%) Algorithms
3 42 (4.72%) Management Information System
4 41 (4.61%) Data Structures
5 40 (4.50%) Greenfield Project
6 13 (1.46%) Cloud Native
7 12 (1.35%) User Experience
8 11 (1.24%) Semantic Web
9 9 (1.01%) Virtual Team
10 8 (0.90%) Social Media
11 7 (0.79%) Hedge funds
12 6 (0.67%) Linux Command Line
13 5 (0.56%) Data Centre
13 5 (0.56%) Derivative
13 5 (0.56%) Renewable Energy
13 5 (0.56%) Robotics
14 4 (0.45%) Blog
14 4 (0.45%) Credit Risk
14 4 (0.45%) Fixed Income
14 4 (0.45%) Replication
Operating Systems
1 138 (15.52%) Linux
2 58 (6.52%) Unix
3 7 (0.79%) Windows
4 6 (0.67%) Red Hat Enterprise Linux
5 3 (0.34%) CentOS
5 3 (0.34%) Windows Server
6 1 (0.11%) Apple iOS
Processes & Methodologies
1 391 (43.98%) Agile Software Development
2 378 (42.52%) Data Science
3 356 (40.04%) ETL
4 335 (37.68%) Analytics
5 221 (24.86%) Business Intelligence
6 199 (22.38%) Data Modelling
7 197 (22.16%) DevOps
8 184 (20.70%) Machine Learning
9 156 (17.55%) Mentoring
10 136 (15.30%) Software Engineering
11 122 (13.72%) Computer Science
12 112 (12.60%) Visualisation
13 110 (12.37%) Scrum
14 107 (12.04%) Data Analytics
15 103 (11.59%) Artificial Intelligence
16 95 (10.69%) Data Management
17 93 (10.46%) Coaching
18 79 (8.89%) Test Automation
19 75 (8.44%) Continuous Integration
20 74 (8.32%) Microservices
Programming Languages
1 531 (59.73%) Python
2 476 (53.54%) SQL
3 266 (29.92%) Scala
4 256 (28.80%) Java
5 70 (7.87%) R
6 60 (6.75%) PL/SQL
7 53 (5.96%) C#
8 45 (5.06%) JavaScript
9 22 (2.47%) T-SQL
10 20 (2.25%) Shell Script
11 18 (2.02%) C++
11 18 (2.02%) Java 8
12 14 (1.57%) Bash Shell
12 14 (1.57%) C
13 11 (1.24%) Data Analysis Expressions (DAX)
13 11 (1.24%) Go
14 10 (1.12%) MATLAB
14 10 (1.12%) Ruby
15 9 (1.01%) PHP
16 7 (0.79%) PowerShell
Qualifications
1 85 (9.56%) Degree
2 35 (3.94%) Computer Science Degree
3 26 (2.92%) PhD
4 14 (1.57%) Master's Degree
5 13 (1.46%) Security Cleared
6 12 (1.35%) AWS Certification
6 12 (1.35%) SC Cleared
7 7 (0.79%) PMI Certification
8 6 (0.67%) CEH
8 6 (0.67%) CISSP
9 5 (0.56%) PMP
10 4 (0.45%) Numerate Degree
11 3 (0.34%) CISA
11 3 (0.34%) CISM
11 3 (0.34%) CREST Certified
11 3 (0.34%) CSSLP
11 3 (0.34%) GIAC
11 3 (0.34%) OSCP
11 3 (0.34%) SANS
11 3 (0.34%) SSCP
Quality Assurance & Compliance
1 68 (7.65%) Data Quality
2 37 (4.16%) QA
3 22 (2.47%) GDPR
4 14 (1.57%) ISO/IEC 27001
5 4 (0.45%) SLA
6 1 (0.11%) Actionable Recommendations
System Software
1 129 (14.51%) Docker
2 75 (8.44%) HDFS
3 21 (2.36%) Apache Flume
4 14 (1.57%) VirtualBox
5 4 (0.45%) Apache ZooKeeper
6 2 (0.22%) Active Directory
7 1 (0.11%) Virtual Servers
Systems Management
1 99 (11.14%) Kubernetes
2 64 (7.20%) Terraform
3 52 (5.85%) Ansible
4 31 (3.49%) Oozie
5 17 (1.91%) Hadoop YARN
6 14 (1.57%) Docker Swarm
7 10 (1.12%) Istio
7 10 (1.12%) linkerd
8 8 (0.90%) Puppet
9 6 (0.67%) Rancher
10 5 (0.56%) Ambari
10 5 (0.56%) Opscode Chef
10 5 (0.56%) Single Sign-On
11 4 (0.45%) Kibana
11 4 (0.45%) Mesos
12 1 (0.11%) Chukwa
12 1 (0.11%) Consul
12 1 (0.11%) Packer
Vendors
1 133 (14.96%) Google
2 103 (11.59%) Cloudera
3 98 (11.02%) Oracle
4 78 (8.77%) Microsoft
5 72 (8.10%) Informatica
6 62 (6.97%) Tableau
7 45 (5.06%) Databricks
8 31 (3.49%) Hortonworks
9 28 (3.15%) Talend
10 16 (1.80%) Ab Initio
10 16 (1.80%) Teradata
11 15 (1.69%) SAP
12 11 (1.24%) IBM
13 9 (1.01%) Alteryx
14 8 (0.90%) Atlassian
14 8 (0.90%) Netezza
15 7 (0.79%) Red Hat
16 6 (0.67%) WorldPay
17 4 (0.45%) Dataiku
17 4 (0.45%) MapR