Period
to 23 August 2019

The following table provides summary statistics for permanent job vacancies with a requirement for Data Engineering skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Engineering over the 6 months to 23 August 2019 with a comparison to the same period in the previous 2 years.

Data Engineering
UK
6 months to
23 Aug 2019
Same period 2018 Same period 2017
Rank 443 647 946
Rank change year-on-year +204 +299 +165
Permanent jobs citing Data Engineering 955 617 263
As % of all permanent IT jobs advertised in the UK 0.67% 0.37% 0.15%
As % of the Processes & Methodologies category 0.73% 0.40% 0.16%
Number of salaries quoted 774 493 226
UK median annual salary £70,000 £70,000 £70,000
10th Percentile £42,750 £48,100 £45,000
90th Percentile £102,500 £105,000 £102,500
UK excluding London median annual salary £57,275 £60,000 £50,000
% change year-on-year -4.54% +20.00% -4.76%

Data Engineering is in the Processes and Methodologies category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for process or methodology skills.

All Process and Methodology Skills
UK
Permanent vacancies with a requirement for process or methodology skills 131,420 154,729 162,660
As % of all permanent IT jobs advertised in the UK 92.18% 91.82% 92.29%
Number of salaries quoted 103,282 123,226 129,146
UK median annual salary £52,500 £50,000 £50,000
Median salary % change year-on-year +5.00% - +5.26%
10th Percentile £30,000 £29,000 £28,000
90th Percentile £86,250 £82,500 £80,000
UK excluding London median annual salary £45,000 £45,000 £42,500
% change year-on-year - +5.88% -

Data Engineering
Job Vacancy Trend

Job postings citing Data Engineering as a percentage of all IT jobs advertised.

Job vacancy trend for Data Engineering in the UK

Data Engineering
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Data Engineering.

Salary trend for Data Engineering in the UK

Data Engineering
Salary Histogram

The salary distribution of IT jobs citing Data Engineering over the 6 months to 23 August 2019.

Salary histogram for Data Engineering in the UK

Data Engineering
Top 15 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Engineering within the UK over the 6 months to 23 August 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +187 892 £70,000 - 118
London +174 575 £75,000 -6.25% 74
UK excluding London +159 366 £57,275 -4.54% 58
South West +78 93 £51,250 -14.58% 7
North of England +75 92 £64,177 +14.60% 9
Midlands +62 60 £52,500 -22.22% 10
South East +16 55 £65,000 -5.45% 12
West Midlands +51 42 £52,500 - 8
North West +40 41 £70,963 +32.02% 8
Yorkshire +37 40 £61,000 -18.67% 1
Scotland +22 39 £47,750 -22.04% 10
East of England +73 18 £60,000 +9.09% 6
East Midlands +11 18 £50,000 -23.08% 2
North East +36 11 £45,412 -4.40%
Wales - 11 £40,000 - 4

For the 6 months to 23 August 2019, IT jobs citing Data Engineering also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Data Engineering.

1 575 (60.21%) Python
2 503 (52.67%) SQL
3 452 (47.33%) Big Data
4 441 (46.18%) Agile Software Development
5 384 (40.21%) Apache Spark
6 378 (39.58%) Data Science
7 377 (39.48%) Analytics
8 366 (38.32%) Hadoop
9 361 (37.80%) AWS
10 325 (34.03%) ETL
11 313 (32.77%) Scala
12 306 (32.04%) Java
13 299 (31.31%) Azure
14 254 (26.60%) Machine Learning
15 252 (26.39%) Data Warehouse
16 218 (22.83%) DevOps
17 215 (22.51%) Business Intelligence
18 203 (21.26%) Data Modelling
19 185 (19.37%) Apache Hive
19 185 (19.37%) Kafka
20 182 (19.06%) NoSQL
21 170 (17.80%) Analytical Skills
22 169 (17.70%) GCP
23 146 (15.29%) Cloudera
23 146 (15.29%) Software Engineering
23 146 (15.29%) Google
24 132 (13.82%) Visualisation
25 130 (13.61%) Finance
26 124 (12.98%) Scrum
27 123 (12.88%) Data Lake

Data Engineering
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 384 (40.21%) Apache Spark
2 89 (9.32%) Apache
3 72 (7.54%) SAS
4 42 (4.40%) Apache Pig
5 30 (3.14%) Apache Airflow
6 25 (2.62%) Elasticsearch
7 9 (0.94%) Jupyter
8 8 (0.84%) Apache Solr
9 7 (0.73%) Confluence
9 7 (0.73%) Luigi
10 1 (0.10%) nginx
10 1 (0.10%) WebLogic Portal
Applications
1 25 (2.62%) Microsoft Excel
2 10 (1.05%) SPSS
3 3 (0.31%) Chatbot
3 3 (0.31%) Microsoft PowerPoint
4 2 (0.21%) Microsoft Office
5 1 (0.10%) MS Visio
Business Applications
1 1 (0.10%) PolarLake
1 1 (0.10%) SAP GTS
Cloud Services
1 361 (37.80%) AWS
2 299 (31.31%) Azure
3 169 (17.70%) GCP
4 91 (9.53%) Power Platform
5 48 (5.03%) Amazon S3
6 43 (4.50%) Amazon EMR
7 39 (4.08%) PaaS
8 38 (3.98%) Amazon Kinesis
9 35 (3.66%) Serverless
10 34 (3.56%) Azure Data Factory
11 28 (2.93%) AWS Lambda
12 17 (1.78%) IaaS
13 16 (1.68%) Azure Machine Learning
14 15 (1.57%) Google Analytics
15 12 (1.26%) GitHub
16 8 (0.84%) AWS CloudFormation
16 8 (0.84%) Cloud Computing
17 6 (0.63%) Amazon SQS
18 4 (0.42%) Azure Active Directory
18 4 (0.42%) Azure Functions
Communications & Networking
1 22 (2.30%) HTTP
2 11 (1.15%) Broadband
3 8 (0.84%) Internet
4 2 (0.21%) Firewall
4 2 (0.21%) HTTPS
4 2 (0.21%) Kerberos
4 2 (0.21%) SSL
4 2 (0.21%) VPN
5 1 (0.10%) TCP/IP
Database & Business Intelligence
1 452 (47.33%) Big Data
2 366 (38.32%) Hadoop
3 252 (26.39%) Data Warehouse
4 185 (19.37%) Apache Hive
5 182 (19.06%) NoSQL
6 123 (12.88%) Data Lake
7 120 (12.57%) Apache Cassandra
8 102 (10.68%) MongoDB
9 91 (9.53%) Amazon Redshift
9 91 (9.53%) Power BI
10 90 (9.42%) SQL Server
11 82 (8.59%) Relational Database
12 68 (7.12%) HBase
13 63 (6.60%) Sqoop
14 58 (6.07%) PostgreSQL
15 56 (5.86%) Pentaho
16 49 (5.13%) QlikView
17 44 (4.61%) Impala
18 42 (4.40%) SQL Server Integration Services
19 40 (4.19%) Oracle Database
Development Applications
1 112 (11.73%) Jenkins
2 102 (10.68%) Git (software)
3 62 (6.49%) Maven
4 54 (5.65%) Artifactory
4 54 (5.65%) Bitbucket
5 40 (4.19%) JIRA
6 26 (2.72%) Team Foundation Server
7 19 (1.99%) Selenium
8 14 (1.47%) ERwin
9 13 (1.36%) Travis CI
10 11 (1.15%) Cucumber
10 11 (1.15%) RStudio
11 10 (1.05%) Atlassian Bamboo
11 10 (1.05%) GitLab
12 8 (0.84%) HP UFT
13 6 (0.63%) Subversion
14 4 (0.42%) GoCD
15 2 (0.21%) CircleCI
15 2 (0.21%) Vagrant
15 2 (0.21%) Visual Studio
General
1 130 (13.61%) Finance
2 79 (8.27%) Retail
3 50 (5.24%) Marketing
4 42 (4.40%) Banking
4 42 (4.40%) Law
5 16 (1.68%) Public Sector
6 15 (1.57%) Advertising
7 11 (1.15%) Legal
8 10 (1.05%) Telecoms
9 7 (0.73%) Games
10 6 (0.63%) Front Office
11 5 (0.52%) Aerospace
11 5 (0.52%) Financial Institution
11 5 (0.52%) Publishing
12 4 (0.42%) Pharmaceutical
13 2 (0.21%) Investment Banking
13 2 (0.21%) Retail Banking
14 1 (0.10%) Corporate Banking
14 1 (0.10%) Manufacturing
14 1 (0.10%) Music Industry
Job Titles
1 443 (46.39%) Data Engineer
2 131 (13.72%) Developer
3 127 (13.30%) Senior Data Engineer
4 108 (11.31%) Big Data Engineer
5 77 (8.06%) Python Engineer
6 76 (7.96%) Data Scientist
7 62 (6.49%) Architect
8 41 (4.29%) Senior Developer
9 37 (3.87%) Data Architect
9 37 (3.87%) Hadoop Engineer
10 29 (3.04%) Software Engineer
11 28 (2.93%) Cloud Engineer
12 26 (2.72%) Data Manager
13 25 (2.62%) Senior Data Scientist
14 24 (2.51%) DevOps Engineer
14 24 (2.51%) ETL Developer
15 22 (2.30%) AWS Engineer
16 21 (2.20%) Python Developer
17 20 (2.09%) Analyst
17 20 (2.09%) Scala Developer
Libraries, Frameworks & Software Standards
1 185 (19.37%) Kafka
2 52 (5.45%) Apache NiFi
3 38 (3.98%) Pandas
4 34 (3.56%) Spark Streaming
5 32 (3.35%) REST
6 31 (3.25%) scikit-learn
7 29 (3.04%) Web Services
8 28 (2.93%) PySpark
9 27 (2.83%) Natural Language Processing
10 23 (2.41%) NumPy
11 21 (2.20%) Apache Flex
12 20 (2.09%) Akka
12 20 (2.09%) Apache Beam
13 19 (1.99%) JSON
13 19 (1.99%) MLlib
13 19 (1.99%) Scalaz
14 18 (1.88%) .NET
15 14 (1.47%) Flask
16 13 (1.36%) Matplotlib
16 13 (1.36%) Node.js
Miscellaneous
1 170 (17.80%) Analytical Skills
2 72 (7.54%) Algorithms
3 30 (3.14%) Greenfield Project
4 29 (3.04%) Management Information System
5 27 (2.83%) Robotics
6 24 (2.51%) Data Structures
7 20 (2.09%) Data Protection Act
8 16 (1.68%) Self-Motivation
9 12 (1.26%) Data Centre
9 12 (1.26%) Virtual Team
10 9 (0.94%) Semantic Web
10 9 (0.94%) User Experience
11 7 (0.73%) Hybrid Cloud
11 7 (0.73%) Linux Command Line
12 6 (0.63%) Social Media
13 5 (0.52%) Lidar
14 4 (0.42%) Clustering
14 4 (0.42%) Hedge funds
14 4 (0.42%) Internet of Things
15 3 (0.31%) Fixed Income
Operating Systems
1 114 (11.94%) Linux
2 84 (8.80%) Unix
3 12 (1.26%) Windows
4 6 (0.63%) Red Hat Enterprise Linux
5 5 (0.52%) Windows Server
6 3 (0.31%) Apple iOS
7 2 (0.21%) Mac OS X
8 1 (0.10%) VMS
Processes & Methodologies
1 441 (46.18%) Agile Software Development
2 378 (39.58%) Data Science
3 377 (39.48%) Analytics
4 325 (34.03%) ETL
5 254 (26.60%) Machine Learning
6 218 (22.83%) DevOps
7 215 (22.51%) Business Intelligence
8 203 (21.26%) Data Modelling
9 146 (15.29%) Software Engineering
10 132 (13.82%) Visualisation
11 124 (12.98%) Scrum
12 117 (12.25%) Mentoring
13 112 (11.73%) Artificial Intelligence
13 112 (11.73%) Data Analytics
14 99 (10.37%) Computer Science
15 83 (8.69%) Problem-Solving
16 78 (8.17%) Test Automation
17 72 (7.54%) Data Management
18 71 (7.43%) Data Architecture
18 71 (7.43%) Microservices
Programming Languages
1 575 (60.21%) Python
2 503 (52.67%) SQL
3 313 (32.77%) Scala
4 306 (32.04%) Java
5 115 (12.04%) R
6 78 (8.17%) PL/SQL
7 46 (4.82%) C#
8 22 (2.30%) JavaScript
9 21 (2.20%) T-SQL
10 14 (1.47%) MATLAB
10 14 (1.47%) Shell Script
11 13 (1.36%) Go
12 11 (1.15%) Bash Shell
12 11 (1.15%) Java 8
12 11 (1.15%) PowerShell
13 10 (1.05%) C++
14 9 (0.94%) Ruby
15 8 (0.84%) Kotlin
15 8 (0.84%) PHP
16 7 (0.73%) C
Qualifications
1 99 (10.37%) Degree
2 38 (3.98%) PhD
3 29 (3.04%) Computer Science Degree
4 21 (2.20%) Master's Degree
5 19 (1.99%) CISSP
6 18 (1.88%) CEH
7 17 (1.78%) AWS Certification
8 16 (1.68%) CISM
9 15 (1.57%) CISA
9 15 (1.57%) CSSLP
9 15 (1.57%) GIAC
9 15 (1.57%) PMI Certification
9 15 (1.57%) PMP
9 15 (1.57%) SANS
9 15 (1.57%) SSCP
10 11 (1.15%) Numerate Degree
11 10 (1.05%) SC Cleared
11 10 (1.05%) Security Cleared
12 8 (0.84%) CompTIA Security+
13 6 (0.63%) ISTQB
Quality Assurance & Compliance
1 56 (5.86%) Data Quality
2 41 (4.29%) QA
3 6 (0.63%) GDPR
4 4 (0.42%) ISO/IEC 27001
5 1 (0.10%) PMO
5 1 (0.10%) SLA
System Software
1 111 (11.62%) Docker
2 55 (5.76%) HDFS
3 37 (3.87%) Apache ZooKeeper
4 23 (2.41%) Apache Flume
5 5 (0.52%) Active Directory
6 2 (0.21%) VirtualBox
7 1 (0.10%) Firmware
7 1 (0.10%) Virtual Servers
Systems Management
1 88 (9.21%) Kubernetes
2 84 (8.80%) Terraform
3 63 (6.60%) Ansible
4 56 (5.86%) Oozie
5 26 (2.72%) Hadoop YARN
6 21 (2.20%) Ambari
7 8 (0.84%) Opscode Chef
8 6 (0.63%) Kibana
9 5 (0.52%) Chukwa
9 5 (0.52%) Puppet
9 5 (0.52%) Rancher
10 4 (0.42%) Mesos
11 3 (0.31%) logstash
11 3 (0.31%) Prometheus
12 2 (0.21%) Docker Swarm
12 2 (0.21%) Single Sign-On
13 1 (0.10%) Consul
13 1 (0.10%) Istio
13 1 (0.10%) linkerd
13 1 (0.10%) Packer
Vendors
1 146 (15.29%) Cloudera
1 146 (15.29%) Google
2 117 (12.25%) Oracle
3 103 (10.79%) Tableau
4 87 (9.11%) Informatica
5 70 (7.33%) Microsoft
6 37 (3.87%) Databricks
7 35 (3.66%) Hortonworks
8 29 (3.04%) Teradata
9 25 (2.62%) Splunk
10 23 (2.41%) Ab Initio
11 22 (2.30%) Talend
12 15 (1.57%) SAP
13 12 (1.26%) Cognizant
13 12 (1.26%) MapR
14 10 (1.05%) Atlassian
14 10 (1.05%) Netezza
15 7 (0.73%) Aprimo
15 7 (0.73%) Pega
16 6 (0.63%) Red Hat