Period
to 15 December 2018

The following table provides summary statistics for permanent job vacancies with a requirement for Data Engineering skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Engineering over the 6 months to 15 December 2018 with a comparison to the same period in the previous 2 years.

Data Engineering
UK
6 months to
15 Dec 2018
Same period 2017 Same period 2016
Rank 506 875 1021
Rank change year-on-year +369 +146 +133
Permanent jobs citing Data Engineering 863 353 211
As % of all permanent IT jobs advertised in the UK 0.55% 0.20% 0.11%
As % of the Processes & Methodologies category 0.60% 0.21% 0.12%
Number of salaries quoted 580 287 174
UK median annual salary £67,500 £67,500 £70,000
Median salary % change year-on-year - -3.57% +3.70%
10th Percentile £45,000 £42,500 £42,500
90th Percentile £100,000 £110,000 £95,000
UK excluding London median annual salary £62,138 £50,000 £50,000
% change year-on-year +24.28% - +4.38%

Data Engineering is in the Processes and Methodologies category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for process or methodology skills.

All Process and Methodology Skills
UK
Permanent vacancies with a requirement for process or methodology skills 144,803 164,913 169,137
As % of all permanent IT jobs advertised in the UK 91.59% 91.90% 91.29%
Number of salaries quoted 114,496 131,060 140,287
UK median annual salary £52,500 £50,000 £48,500
Median salary % change year-on-year +5.00% +3.09% +2.11%
10th Percentile £29,750 £28,418 £27,625
90th Percentile £85,000 £80,000 £78,750
UK excluding London median annual salary £45,000 £43,500 £42,500
% change year-on-year +3.45% +2.35% -

Data Engineering
Job Vacancy Trend

Job postings citing Data Engineering as a percentage of all IT jobs advertised.

Job vacancy trend for Data Engineering in the UK

Data Engineering
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Data Engineering.

Salary trend for Data Engineering in the UK

Data Engineering
Salary Histogram

The salary distribution of IT jobs citing Data Engineering over the 6 months to 15 December 2018.

Salary histogram for Data Engineering in the UK

Data Engineering
Top 15 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Engineering within the UK over the 6 months to 15 December 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +348 819 £67,500 -3.57% 104
UK excluding London +337 447 £62,138 +24.28% 50
London +176 412 £70,000 -6.67% 60
South East +175 181 £70,000 +40.00% 15
South West +60 89 £60,000 +20.00% 9
North of England +91 80 £70,000 +47.37% 6
North West +83 65 £70,000 +47.37% 1
Midlands +71 39 £53,000 +6.00% 3
Scotland +40 36 £50,000 - 5
East Midlands +31 22 £51,250 +2.50% 2
East of England +41 17 £70,000 +16.67% 11
West Midlands - 17 £64,733 - 1
Yorkshire +36 8 £65,707 +54.60% 5
North East - 7 £49,337 -
Wales +3 5 £47,500 +34.75%

For the 6 months to 15 December 2018, IT jobs citing Data Engineering also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Data Engineering.

1 482 (55.85%) Python
2 415 (48.09%) SQL
3 410 (47.51%) Analytics
4 408 (47.28%) Data Science
5 387 (44.84%) Big Data
6 355 (41.14%) Hadoop
7 350 (40.56%) Agile Software Development
8 319 (36.96%) ETL
9 313 (36.27%) Apache Spark
10 306 (35.46%) Data Modelling
11 266 (30.82%) Business Intelligence
12 256 (29.66%) Degree
13 245 (28.39%) Microsoft Azure
13 245 (28.39%) Data Warehouse
14 238 (27.58%) Amazon AWS
15 218 (25.26%) Machine Learning
16 216 (25.03%) Scala
17 215 (24.91%) Java
18 206 (23.87%) Finance
19 196 (22.71%) Kafka
20 192 (22.25%) Visualisation
21 187 (21.67%) Artificial Intelligence
22 174 (20.16%) Tableau
23 173 (20.05%) NoSQL
24 167 (19.35%) Apache Hive
25 163 (18.89%) Power BI
26 162 (18.77%) Retail
27 144 (16.69%) Data Visualisation
28 142 (16.45%) Apache
29 138 (15.99%) R

Data Engineering
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 313 (36.27%) Apache Spark
2 142 (16.45%) Apache
3 64 (7.42%) SAS
4 23 (2.67%) Apache Pig
5 17 (1.97%) Elasticsearch
6 14 (1.62%) Apache Airflow
6 14 (1.62%) CMS
6 14 (1.62%) Hyperledger
7 11 (1.27%) Apache Solr
8 7 (0.81%) Luigi
9 5 (0.58%) WebSphere
10 2 (0.23%) Jupyter
10 2 (0.23%) Tornado
11 1 (0.12%) Confluence
11 1 (0.12%) SharePoint
Applications
1 23 (2.67%) MS Visio
2 9 (1.04%) SPSS
3 5 (0.58%) Microsoft Excel
4 4 (0.46%) Microsoft Office
5 1 (0.12%) Microsoft Project
Business Applications
1 5 (0.58%) Magento
2 2 (0.23%) NetSuite
3 1 (0.12%) Odoo
Cloud Services
1 245 (28.39%) Microsoft Azure
2 238 (27.58%) Amazon AWS
3 79 (9.15%) Google Cloud Platform
4 69 (8.00%) Cloud Computing
5 55 (6.37%) Amazon EC2
6 51 (5.91%) Amazon EMR
7 41 (4.75%) Serverless
8 34 (3.94%) Azure Data Factory
9 21 (2.43%) AWS Lambda
10 19 (2.20%) Amazon S3
11 17 (1.97%) Amazon Kinesis
12 13 (1.51%) Azure Functions
13 11 (1.27%) GitHub
14 8 (0.93%) Data Lake Analytics
14 8 (0.93%) PaaS
15 7 (0.81%) SaaS
16 6 (0.70%) Google Analytics
17 5 (0.58%) Azure Machine Learning
18 4 (0.46%) AWS CloudFormation
18 4 (0.46%) Cortana Intelligence Suite
Communications & Networking
1 19 (2.20%) Internet
2 9 (1.04%) HTTP
3 2 (0.23%) Kerberos
4 1 (0.12%) SMS
Database & Business Intelligence
1 387 (44.84%) Big Data
2 355 (41.14%) Hadoop
3 245 (28.39%) Data Warehouse
4 173 (20.05%) NoSQL
5 167 (19.35%) Apache Hive
6 163 (18.89%) Power BI
7 105 (12.17%) Relational Database
8 104 (12.05%) MongoDB
9 93 (10.78%) Data Lake
10 80 (9.27%) SQL Server
11 78 (9.04%) Amazon Redshift
12 69 (8.00%) HBase
13 59 (6.84%) Apache Cassandra
14 50 (5.79%) SQL Server Integration Services
15 49 (5.68%) SAP BW
16 48 (5.56%) PostgreSQL
17 44 (5.10%) Flink
17 44 (5.10%) SAP HANA
17 44 (5.10%) Text Mining
18 40 (4.63%) Impala
Development Applications
1 48 (5.56%) JIRA
2 37 (4.29%) Git (software)
2 37 (4.29%) Jenkins
3 19 (2.20%) TeamCity
4 13 (1.51%) Octopus Deploy
5 12 (1.39%) tSQLt
6 11 (1.27%) Visual Studio Team System
7 8 (0.93%) JUnit
7 8 (0.93%) SpecFlow
8 6 (0.70%) SonarQube
9 5 (0.58%) GitLab
10 4 (0.46%) Maven
10 4 (0.46%) Team Foundation Server
10 4 (0.46%) Visual Studio
11 3 (0.35%) Eclipse
11 3 (0.35%) IntelliJ
12 2 (0.23%) Bitbucket
12 2 (0.23%) FindBugs
12 2 (0.23%) Robot Framework
12 2 (0.23%) Selenium
General
1 206 (23.87%) Finance
2 162 (18.77%) Retail
3 91 (10.54%) Telecoms
4 38 (4.40%) Banking
5 35 (4.06%) Marketing
6 27 (3.13%) Law
7 25 (2.90%) Military
8 17 (1.97%) Advertising
9 13 (1.51%) Corporate Banking
9 13 (1.51%) Investment Banking
10 12 (1.39%) Games
11 11 (1.27%) Front Office
12 5 (0.58%) Manufacturing
13 4 (0.46%) Publishing
14 3 (0.35%) Automotive
14 3 (0.35%) French Language
14 3 (0.35%) Music Industry
14 3 (0.35%) Pharmaceutical
15 2 (0.23%) Electronics
15 2 (0.23%) Legal
Job Titles
1 393 (45.54%) Data Engineer
2 99 (11.47%) Big Data Engineer
3 95 (11.01%) Data Scientist
4 89 (10.31%) Senior Data Engineer
5 85 (9.85%) Developer
6 67 (7.76%) Architect
7 57 (6.60%) Python Engineer
8 52 (6.03%) Data Architect
9 48 (5.56%) Senior Data Scientist
10 39 (4.52%) Analyst
11 38 (4.40%) Hadoop Engineer
12 33 (3.82%) Consultant
13 25 (2.90%) Senior Architect
14 21 (2.43%) Azure Engineer
14 21 (2.43%) Data Analyst
14 21 (2.43%) Data Manager
14 21 (2.43%) Senior Data Architect
15 17 (1.97%) Machine Learning Scientist
16 15 (1.74%) Junior
17 14 (1.62%) Database Engineer
Libraries, Frameworks & Software Standards
1 196 (22.71%) Kafka
2 63 (7.30%) Web Services
3 55 (6.37%) Spark Streaming
4 47 (5.45%) D3.js
5 41 (4.75%) ggplot
6 37 (4.29%) Pandas
7 36 (4.17%) NumPy
8 32 (3.71%) TensorFlow
9 30 (3.48%) .NET
10 29 (3.36%) Natural Language Processing
11 24 (2.78%) AngularJS
12 22 (2.55%) Node.js
12 22 (2.55%) REST
13 21 (2.43%) React
14 20 (2.32%) scikit-learn
15 19 (2.20%) JSON
15 19 (2.20%) Middleware
15 19 (2.20%) SciPy
16 16 (1.85%) Express
17 15 (1.74%) Akka
Miscellaneous
1 112 (12.98%) Analytical Skills
2 79 (9.15%) Algorithms
3 49 (5.68%) FMCG
4 43 (4.98%) Data Structures
5 41 (4.75%) Social Network
6 30 (3.48%) Fintech
7 24 (2.78%) Greenfield Project
8 21 (2.43%) Self-Motivation
9 20 (2.32%) User Experience
10 18 (2.09%) Internet of Things
11 17 (1.97%) Enterprise Software
12 15 (1.74%) Management Information System
13 13 (1.51%) Data Protection Act
13 13 (1.51%) Derivative
13 13 (1.51%) Linux Command Line
14 12 (1.39%) Semantic Web
15 10 (1.16%) Equities
15 10 (1.16%) Linked Data
15 10 (1.16%) Reference Data
15 10 (1.16%) Renewable Energy
Operating Systems
1 120 (13.90%) Linux
2 48 (5.56%) Unix
3 33 (3.82%) Windows
4 3 (0.35%) Apple iOS
5 1 (0.12%) Android
5 1 (0.12%) Red Hat Enterprise Linux
5 1 (0.12%) Windows Server
Processes & Methodologies
1 410 (47.51%) Analytics
2 408 (47.28%) Data Science
3 350 (40.56%) Agile Software Development
4 319 (36.96%) ETL
5 306 (35.46%) Data Modelling
6 266 (30.82%) Business Intelligence
7 218 (25.26%) Machine Learning
8 192 (22.25%) Visualisation
9 187 (21.67%) Artificial Intelligence
10 144 (16.69%) Data Visualisation
11 125 (14.48%) Mathematics
12 123 (14.25%) Computer Science
13 121 (14.02%) Decision-Making
14 115 (13.33%) Data Integration
14 115 (13.33%) Data Management
14 115 (13.33%) DevOps
15 102 (11.82%) Open Source
16 100 (11.59%) Stakeholder Management
17 98 (11.36%) Statistics
18 95 (11.01%) Data Security
Programming Languages
1 482 (55.85%) Python
2 415 (48.09%) SQL
3 216 (25.03%) Scala
4 215 (24.91%) Java
5 138 (15.99%) R
6 53 (6.14%) C#
7 41 (4.75%) T-SQL
8 39 (4.52%) JavaScript
9 36 (4.17%) Go
10 28 (3.24%) Bash Shell
11 20 (2.32%) PowerShell
12 19 (2.20%) Shell Script
13 16 (1.85%) Ruby
14 15 (1.74%) C
14 15 (1.74%) U-SQL
15 14 (1.62%) C++
16 12 (1.39%) MATLAB
16 12 (1.39%) PL/SQL
17 6 (0.70%) Java 8
17 6 (0.70%) Perl
Qualifications
1 256 (29.66%) Degree
2 74 (8.57%) Computer Science Degree
3 42 (4.87%) Numerate Degree
4 27 (3.13%) Microsoft Certification
5 23 (2.67%) MCSA
6 22 (2.55%) PhD
7 20 (2.32%) Master's Degree
8 9 (1.04%) SC Cleared
8 9 (1.04%) Security Cleared
9 4 (0.46%) MCAD
9 4 (0.46%) MCDBA
9 4 (0.46%) MCSD
10 3 (0.35%) AWS Certification
11 2 (0.23%) CISA
11 2 (0.23%) CISM
11 2 (0.23%) CISSP
11 2 (0.23%) GIAC
11 2 (0.23%) ITIL Certification
11 2 (0.23%) SANS
11 2 (0.23%) SSCP
Quality Assurance & Compliance
1 62 (7.18%) Data Quality
2 18 (2.09%) QA
3 4 (0.46%) GDPR
4 1 (0.12%) PMO
System Software
1 44 (5.10%) Docker
2 43 (4.98%) HDFS
3 35 (4.06%) Apache Flume
4 16 (1.85%) Apache ZooKeeper
5 2 (0.23%) Active Directory
6 1 (0.12%) VirtualBox
Systems Management
1 60 (6.95%) Hadoop YARN
2 43 (4.98%) Oozie
3 27 (3.13%) Kubernetes
4 15 (1.74%) Mesos
5 9 (1.04%) Ambari
6 7 (0.81%) Ansible
6 7 (0.81%) Opscode Chef
7 5 (0.58%) Kibana
8 4 (0.46%) Terraform
9 2 (0.23%) logstash
9 2 (0.23%) Puppet
10 1 (0.12%) Docker Swarm
10 1 (0.12%) Grafana
10 1 (0.12%) Graylog
10 1 (0.12%) Packer
10 1 (0.12%) Prometheus
Vendors
1 174 (20.16%) Tableau
2 134 (15.53%) Microsoft
3 115 (13.33%) SAP
4 86 (9.97%) Oracle
5 69 (8.00%) Google
6 55 (6.37%) Cloudera
7 45 (5.21%) Informatica
8 43 (4.98%) Hortonworks
9 42 (4.87%) CA Agile Central
10 38 (4.40%) Teradata
11 32 (3.71%) Netezza
12 30 (3.48%) MapR
13 24 (2.78%) WorldPay
14 22 (2.55%) Ab Initio
15 13 (1.51%) Cognos
16 12 (1.39%) Salesforce.com
16 12 (1.39%) Talend
17 10 (1.16%) RapidMiner
18 9 (1.04%) Sybase
19 8 (0.93%) MicroStrategy