Period
to 21 October 2018

The following table provides summary statistics for permanent job vacancies with a requirement for Data Engineering skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Engineering over the 6 months to 21 October 2018 with a comparison to the same period in the previous 2 years.

Data Engineering
UK
6 months to
21 Oct 2018
Same period 2017 Same period 2016
Rank 601 917 1065
Rank change year-on-year +316 +148 +129
Permanent jobs citing Data Engineering 662 284 167
As % of all permanent IT jobs advertised in the UK 0.40% 0.16% 0.087%
As % of the Processes & Methodologies category 0.44% 0.18% 0.096%
Number of salaries quoted 475 240 126
UK median annual salary £67,500 £68,750 £70,000
Median salary % change year-on-year -1.82% -1.79% +5.66%
10th Percentile £46,250 £43,625 £38,750
90th Percentile £100,000 £110,000 £95,625
UK excluding London median annual salary £62,138 £50,000 £45,000
% change year-on-year +24.28% +11.11% +38.46%

Data Engineering is in the Processes and Methodologies category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for process or methodology skills.

All Process and Methodology Skills
UK
Permanent vacancies with a requirement for process or methodology skills 148,908 159,518 173,686
As % of all permanent IT jobs advertised in the UK 91.04% 91.56% 90.90%
Number of salaries quoted 118,438 125,808 144,551
UK median annual salary £50,000 £50,000 £48,500
Median salary % change year-on-year - +3.09% +2.11%
10th Percentile £29,000 £28,250 £27,500
90th Percentile £83,750 £80,000 £78,750
UK excluding London median annual salary £45,000 £42,500 £42,500
% change year-on-year +5.88% - -

Data Engineering
Job Vacancy Trend

Job postings citing Data Engineering as a percentage of all IT jobs advertised.

Job vacancy trend for Data Engineering in the UK

Data Engineering
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Data Engineering.

Salary trend for Data Engineering in the UK

Data Engineering
Salary Histogram

The salary distribution of IT jobs citing Data Engineering over the 6 months to 21 October 2018.

Salary histogram for Data Engineering in the UK

Data Engineering
Top 14 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Engineering within the UK over the 6 months to 21 October 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +303 621 £67,500 -3.57% 135
London +127 328 £75,000 - 73
UK excluding London +261 326 £62,138 +24.28% 67
South East +121 126 £65,000 +30.00% 13
South West +39 58 £61,069 +22.14% 17
North of England +93 51 £65,707 +46.02% 17
North West +55 44 £65,000 +36.84% 9
Midlands +57 39 £67,367 +34.73% 7
Scotland +40 34 £50,000 - 4
East Midlands +12 24 £70,000 +40.00% 4
East of England +30 18 £70,000 +16.67% 8
West Midlands - 15 £64,733 - 3
Yorkshire +32 4 £70,354 +65.54% 5
North East - 3 £65,707 - 3

For the 6 months to 21 October 2018, IT jobs citing Data Engineering also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Data Engineering.

1 391 (59.06%) Python
2 342 (51.66%) Big Data
3 318 (48.04%) Data Science
4 316 (47.73%) Analytics
5 282 (42.60%) Hadoop
6 278 (41.99%) SQL
7 268 (40.48%) Agile Software Development
8 261 (39.43%) Apache Spark
9 223 (33.69%) ETL
10 213 (32.18%) Machine Learning
11 192 (29.00%) Java
12 190 (28.70%) Data Modelling
13 186 (28.10%) Data Warehouse
14 181 (27.34%) Scala
15 180 (27.19%) Amazon AWS
16 176 (26.59%) Degree
17 166 (25.08%) Business Intelligence
18 160 (24.17%) Microsoft Azure
19 159 (24.02%) Finance
20 143 (21.60%) Kafka
21 141 (21.30%) Apache Hive
22 140 (21.15%) Visualisation
22 140 (21.15%) Artificial Intelligence
23 134 (20.24%) NoSQL
24 122 (18.43%) R
25 111 (16.77%) Retail
26 109 (16.47%) Tableau
27 97 (14.65%) Data Visualisation
27 97 (14.65%) Software Engineering
28 94 (14.20%) Power BI

Data Engineering
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 261 (39.43%) Apache Spark
2 81 (12.24%) Apache
3 40 (6.04%) SAS
4 18 (2.72%) Apache Pig
5 17 (2.57%) Elasticsearch
6 14 (2.11%) CMS
6 14 (2.11%) Hyperledger
7 10 (1.51%) Apache Airflow
8 6 (0.91%) Apache Solr
8 6 (0.91%) NServiceBus
8 6 (0.91%) SharePoint
9 5 (0.76%) Documentum
9 5 (0.76%) FileNet
10 2 (0.30%) Luigi
10 2 (0.30%) Tornado
11 1 (0.15%) Confluence
11 1 (0.15%) Jupyter
Applications
1 10 (1.51%) MS Visio
2 7 (1.06%) Microsoft Excel
3 6 (0.91%) SPSS
4 1 (0.15%) Adobe Illustrator
4 1 (0.15%) Microsoft Office
4 1 (0.15%) OmniGraffle
4 1 (0.15%) Photoshop
Business Applications
1 2 (0.30%) NetSuite
2 1 (0.15%) Odoo
Cloud Services
1 180 (27.19%) Amazon AWS
2 160 (24.17%) Microsoft Azure
3 42 (6.34%) Google Cloud Platform
4 39 (5.89%) Amazon EC2
5 37 (5.59%) Amazon EMR
5 37 (5.59%) Cloud Computing
6 29 (4.38%) Serverless
7 25 (3.78%) Amazon Kinesis
8 19 (2.87%) AWS Lambda
9 16 (2.42%) Azure Data Factory
10 15 (2.27%) Amazon S3
11 8 (1.21%) Azure Functions
11 8 (1.21%) Google Analytics
12 6 (0.91%) Amazon SQS
12 6 (0.91%) Data Lake Analytics
12 6 (0.91%) IBM Cloud
13 4 (0.60%) Cortana Intelligence Suite
13 4 (0.60%) Google App Engine
13 4 (0.60%) PaaS
14 3 (0.45%) IaaS
Communications & Networking
1 12 (1.81%) Internet
2 2 (0.30%) Kerberos
2 2 (0.30%) MQTT
3 1 (0.15%) 5G
3 1 (0.15%) CHAP
3 1 (0.15%) DAB
3 1 (0.15%) HTTP
3 1 (0.15%) SMS
Database & Business Intelligence
1 342 (51.66%) Big Data
2 282 (42.60%) Hadoop
3 186 (28.10%) Data Warehouse
4 141 (21.30%) Apache Hive
5 134 (20.24%) NoSQL
6 94 (14.20%) Power BI
7 80 (12.08%) Data Lake
8 68 (10.27%) Relational Database
8 68 (10.27%) SQL Server
9 56 (8.46%) Impala
10 54 (8.16%) Amazon Redshift
10 54 (8.16%) HBase
10 54 (8.16%) PostgreSQL
11 53 (8.01%) MongoDB
12 47 (7.10%) Apache Cassandra
13 36 (5.44%) Sqoop
14 29 (4.38%) SQL Server Integration Services
15 28 (4.23%) Data Mining
16 27 (4.08%) SAP BW
17 26 (3.93%) Text Mining
Development Applications
1 40 (6.04%) Jenkins
2 34 (5.14%) Git (software)
3 23 (3.47%) TeamCity
4 19 (2.87%) JIRA
5 11 (1.66%) JUnit
6 8 (1.21%) GitLab
7 7 (1.06%) Octopus Deploy
8 6 (0.91%) SonarQube
8 6 (0.91%) SpecFlow
8 6 (0.91%) tSQLt
8 6 (0.91%) Visual Studio Team System
9 5 (0.76%) Maven
10 4 (0.60%) RStudio
11 2 (0.30%) CircleCI
11 2 (0.30%) Eclipse
11 2 (0.30%) FindBugs
11 2 (0.30%) IntelliJ
11 2 (0.30%) Robot Framework
11 2 (0.30%) Selenium
12 1 (0.15%) Subversion
General
1 159 (24.02%) Finance
2 111 (16.77%) Retail
3 54 (8.16%) Telecoms
4 46 (6.95%) Banking
5 34 (5.14%) Marketing
6 18 (2.72%) Advertising
7 14 (2.11%) Investment Banking
8 11 (1.66%) Corporate Banking
8 11 (1.66%) Front Office
9 10 (1.51%) Games
10 8 (1.21%) Law
11 7 (1.06%) Electronics
12 6 (0.91%) Manufacturing
13 4 (0.60%) Military
14 3 (0.45%) Automotive
14 3 (0.45%) Financial Institution
15 2 (0.30%) Music Industry
15 2 (0.30%) Pharmaceutical
15 2 (0.30%) Retail Banking
16 1 (0.15%) Legal
Job Titles
1 283 (42.75%) Data Engineer
2 94 (14.20%) Data Scientist
3 88 (13.29%) Big Data Engineer
4 69 (10.42%) Senior Data Engineer
5 67 (10.12%) Developer
6 55 (8.31%) Architect
7 41 (6.19%) Senior Data Scientist
8 39 (5.89%) Data Architect
9 32 (4.83%) Hadoop Engineer
10 28 (4.23%) Python Engineer
11 27 (4.08%) Consultant
12 19 (2.87%) Machine Learning Scientist
13 18 (2.72%) Senior Architect
14 17 (2.57%) Analyst
14 17 (2.57%) Senior Hadoop Engineer
15 15 (2.27%) Junior
15 15 (2.27%) Software Engineer
16 14 (2.11%) Senior Data Architect
17 13 (1.96%) Data Consultant
17 13 (1.96%) Software Developer
Libraries, Frameworks & Software Standards
1 143 (21.60%) Kafka
2 46 (6.95%) Web Services
3 43 (6.50%) TensorFlow
4 40 (6.04%) D3.js
5 37 (5.59%) Natural Language Processing
6 33 (4.98%) Pandas
7 32 (4.83%) AngularJS
8 29 (4.38%) Node.js
8 29 (4.38%) NumPy
9 28 (4.23%) .NET
10 26 (3.93%) REST
11 25 (3.78%) ggplot
12 24 (3.63%) React
13 22 (3.32%) Spark Streaming
14 21 (3.17%) Keras
15 19 (2.87%) MLlib
16 18 (2.72%) Middleware
17 17 (2.57%) JSON
18 16 (2.42%) SciPy
19 15 (2.27%) scikit-learn
Miscellaneous
1 85 (12.84%) Computer Science
2 80 (12.08%) Algorithms
3 77 (11.63%) Analytical Skills
4 38 (5.74%) Fintech
5 36 (5.44%) Data Structures
6 31 (4.68%) User Experience
7 27 (4.08%) FMCG
8 25 (3.78%) Greenfield Project
9 24 (3.63%) Self-Motivation
9 24 (3.63%) Social Network
10 17 (2.57%) Enterprise Software
11 16 (2.42%) Derivative
12 13 (1.96%) Data Protection Act
13 12 (1.81%) Internet of Things
13 12 (1.81%) Linux Command Line
14 11 (1.66%) Equities
15 10 (1.51%) Management Information System
16 9 (1.36%) Foreign Exchange (FX)
17 8 (1.21%) Renewable Energy
18 6 (0.91%) Reference Data
Operating Systems
1 92 (13.90%) Linux
2 27 (4.08%) Unix
3 12 (1.81%) Windows
4 1 (0.15%) Red Hat Enterprise Linux
4 1 (0.15%) Windows Server
Processes & Methodologies
1 318 (48.04%) Data Science
2 316 (47.73%) Analytics
3 268 (40.48%) Agile Software Development
4 223 (33.69%) ETL
5 213 (32.18%) Machine Learning
6 190 (28.70%) Data Modelling
7 166 (25.08%) Business Intelligence
8 140 (21.15%) Artificial Intelligence
8 140 (21.15%) Visualisation
9 97 (14.65%) Data Visualisation
9 97 (14.65%) Software Engineering
10 86 (12.99%) Mathematics
11 82 (12.39%) DevOps
11 82 (12.39%) Statistics
12 76 (11.48%) Data Management
13 69 (10.42%) Open Source
14 67 (10.12%) Data Architecture
15 62 (9.37%) Data Integration
16 58 (8.76%) Stakeholder Management
16 58 (8.76%) Use Case
Programming Languages
1 391 (59.06%) Python
2 278 (41.99%) SQL
3 192 (29.00%) Java
4 181 (27.34%) Scala
5 122 (18.43%) R
6 50 (7.55%) JavaScript
7 49 (7.40%) C#
8 41 (6.19%) Go
9 30 (4.53%) T-SQL
10 26 (3.93%) Bash Shell
11 25 (3.78%) Shell Script
12 19 (2.87%) C
13 18 (2.72%) C++
13 18 (2.72%) MATLAB
13 18 (2.72%) PowerShell
14 10 (1.51%) Java 8
14 10 (1.51%) U-SQL
15 7 (1.06%) Kotlin
15 7 (1.06%) Ruby
16 3 (0.45%) Groovy
Qualifications
1 176 (26.59%) Degree
2 49 (7.40%) Computer Science Degree
3 20 (3.02%) PhD
4 18 (2.72%) Numerate Degree
5 12 (1.81%) Master's Degree
6 11 (1.66%) SC Cleared
6 11 (1.66%) Security Cleared
7 10 (1.51%) MCSA
7 10 (1.51%) Microsoft Certification
8 5 (0.76%) MPhil
9 2 (0.30%) DV Cleared
9 2 (0.30%) ITIL Certification
10 1 (0.15%) AWS Certification
10 1 (0.15%) CEH
10 1 (0.15%) CISA
10 1 (0.15%) CISM
10 1 (0.15%) CISSP
10 1 (0.15%) CSSLP
10 1 (0.15%) GIAC
10 1 (0.15%) SSCP
Quality Assurance & Compliance
1 47 (7.10%) Data Quality
2 13 (1.96%) GDPR
2 13 (1.96%) QA
System Software
1 43 (6.50%) Apache Flume
1 43 (6.50%) HDFS
2 37 (5.59%) Docker
3 9 (1.36%) Apache ZooKeeper
4 3 (0.45%) Active Directory
5 1 (0.15%) VirtualBox
Systems Management
1 60 (9.06%) Hadoop YARN
2 29 (4.38%) Oozie
3 26 (3.93%) Kubernetes
4 18 (2.72%) Mesos
5 13 (1.96%) Ansible
6 9 (1.36%) Terraform
7 8 (1.21%) Kibana
7 8 (1.21%) Opscode Chef
8 6 (0.91%) Consul
9 5 (0.76%) Ambari
10 4 (0.60%) Puppet
11 1 (0.15%) Docker Swarm
11 1 (0.15%) Packer
11 1 (0.15%) Prometheus
Vendors
1 109 (16.47%) Tableau
2 76 (11.48%) Microsoft
3 59 (8.91%) SAP
4 55 (8.31%) Cloudera
5 53 (8.01%) Oracle
6 46 (6.95%) Google
7 39 (5.89%) Hortonworks
8 38 (5.74%) Informatica
9 34 (5.14%) Teradata
10 32 (4.83%) MapR
11 24 (3.63%) Netezza
12 22 (3.32%) Talend
13 17 (2.57%) Ab Initio
14 16 (2.42%) CA Agile Central
15 14 (2.11%) RapidMiner
15 14 (2.11%) WorldPay
16 12 (1.81%) Salesforce.com
17 10 (1.51%) Cognos
18 5 (0.76%) IBM
18 5 (0.76%) Sybase