Period
to 19 August 2018

The following table provides summary statistics for permanent job vacancies with a requirement for Data Engineering skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Engineering over the 6 months to 19 August 2018 with a comparison to the same period in the previous 2 years.

Data Engineering
UK
6 months to
19 Aug 2018
Same period 2017 Same period 2016
Rank 630 939 1106
Rank change year-on-year +309 +167 +129
Permanent jobs citing Data Engineering 615 263 175
As % of all permanent IT jobs advertised in the UK 0.36% 0.15% 0.089%
As % of the Processes & Methodologies category 0.40% 0.16% 0.098%
Number of salaries quoted 491 225 123
UK median annual salary £70,000 £70,000 £65,000
Median salary % change year-on-year - +7.69% -
10th Percentile £48,750 £45,500 £42,500
90th Percentile £105,000 £102,500 £105,000
UK excluding London median annual salary £60,000 £50,000 £52,500
% change year-on-year +20.00% -4.76% +40.00%

Data Engineering is in the Processes and Methodologies category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for process or methodology skills.

All Process and Methodology Skills
UK
Permanent vacancies with a requirement for process or methodology skills 153,971 162,157 179,115
As % of all permanent IT jobs advertised in the UK 91.11% 91.53% 90.82%
Number of salaries quoted 122,527 129,139 149,493
UK median annual salary £50,000 £50,000 £48,000
Median salary % change year-on-year - +4.17% +1.05%
10th Percentile £29,000 £28,000 £27,500
90th Percentile £82,500 £80,000 £77,500
UK excluding London median annual salary £45,000 £42,500 £42,500
% change year-on-year +5.88% - +1.19%

Data Engineering
Job Vacancy Trend

Job postings citing Data Engineering as a percentage of all IT jobs advertised.

Job vacancy trend for Data Engineering in the UK

Data Engineering
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Data Engineering.

Salary trend for Data Engineering in the UK

Data Engineering
Salary Histogram

The salary distribution of IT jobs citing Data Engineering over the 6 months to 19 August 2018.

Salary histogram for Data Engineering in the UK

Data Engineering
Top 14 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Engineering within the UK over the 6 months to 19 August 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +268 586 £70,000 - 95
London +186 392 £80,000 +13.88% 69
UK excluding London +181 209 £60,000 +20.00% 32
South East +70 72 £68,750 +30.95% 9
Midlands +56 39 £65,000 +30.00% 6
North of England +45 35 £60,000 +20.00% 3
North West +43 29 £55,000 +10.00% 2
East Midlands +27 29 £65,000 +30.00% 5
East of England +36 27 £55,000 -8.33% 2
South West +18 20 £60,000 +4.35% 6
Scotland +1 16 £61,250 - 4
West Midlands +34 9 - - 1
Yorkshire +39 5 £75,000 +50.00% 1
North East -22 1 £47,500 -26.92%

For the 6 months to 19 August 2018, IT jobs citing Data Engineering also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Data Engineering.

1 404 (65.69%) Python
2 370 (60.16%) Big Data
3 265 (43.09%) Hadoop
3 265 (43.09%) Data Science
4 259 (42.11%) SQL
5 254 (41.30%) Machine Learning
6 250 (40.65%) Apache Spark
7 235 (38.21%) Analytics
8 224 (36.42%) Agile Software Development
9 223 (36.26%) Java
10 203 (33.01%) Scala
11 195 (31.71%) Amazon AWS
12 175 (28.46%) ETL
13 159 (25.85%) NoSQL
14 156 (25.37%) Data Warehouse
15 135 (21.95%) Kafka
16 125 (20.33%) Apache Hive
17 102 (16.59%) Linux
18 101 (16.42%) Artificial Intelligence
19 100 (16.26%) Degree
20 97 (15.77%) Finance
20 97 (15.77%) Microsoft Azure
21 96 (15.61%) Data Modelling
22 95 (15.45%) R
23 91 (14.80%) Algorithms
24 89 (14.47%) Business Intelligence
25 87 (14.15%) Software Engineering
26 81 (13.17%) Data Lake
27 80 (13.01%) DevOps
27 80 (13.01%) Visualisation

Data Engineering
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 250 (40.65%) Apache Spark
2 37 (6.02%) Apache
3 22 (3.58%) SAS
4 19 (3.09%) Apache Pig
5 18 (2.93%) Elasticsearch
6 14 (2.28%) CMS
6 14 (2.28%) Hyperledger
7 11 (1.79%) SharePoint
8 10 (1.63%) Documentum
8 10 (1.63%) FileNet
9 8 (1.30%) Apache Airflow
9 8 (1.30%) Apache Solr
10 7 (1.14%) NServiceBus
11 3 (0.49%) IIS
12 1 (0.16%) Confluence
12 1 (0.16%) Jupyter
12 1 (0.16%) Luigi
12 1 (0.16%) Tornado
Applications
1 10 (1.63%) Microsoft Excel
2 4 (0.65%) SPSS
3 2 (0.33%) MS Visio
4 1 (0.16%) Adobe Illustrator
4 1 (0.16%) Chatbot
4 1 (0.16%) OmniGraffle
4 1 (0.16%) Photoshop
Cloud Services
1 195 (31.71%) Amazon AWS
2 97 (15.77%) Microsoft Azure
3 46 (7.48%) Amazon EMR
4 40 (6.50%) Serverless
5 39 (6.34%) AWS Lambda
6 37 (6.02%) Google Cloud Platform
7 33 (5.37%) Amazon Kinesis
8 32 (5.20%) Amazon S3
9 29 (4.72%) Amazon EC2
10 20 (3.25%) Cloud Computing
11 10 (1.63%) Azure Data Factory
12 9 (1.46%) IBM Cloud
13 8 (1.30%) Google Analytics
14 7 (1.14%) Amazon SQS
15 4 (0.65%) Google App Engine
16 3 (0.49%) Cortana Intelligence Suite
16 3 (0.49%) PaaS
17 2 (0.33%) Google Tag Manager
17 2 (0.33%) IaaS
18 1 (0.16%) Data Lake Analytics
Communications & Networking
1 5 (0.81%) MQTT
2 4 (0.65%) Firewall
3 3 (0.49%) Internet
4 1 (0.16%) 5G
4 1 (0.16%) CHAP
4 1 (0.16%) DAB
4 1 (0.16%) Kerberos
4 1 (0.16%) SMS
Database & Business Intelligence
1 370 (60.16%) Big Data
2 265 (43.09%) Hadoop
3 159 (25.85%) NoSQL
4 156 (25.37%) Data Warehouse
5 125 (20.33%) Apache Hive
6 81 (13.17%) Data Lake
7 74 (12.03%) PostgreSQL
8 67 (10.89%) Impala
9 64 (10.41%) Apache Cassandra
10 62 (10.08%) HBase
11 53 (8.62%) Amazon Redshift
12 52 (8.46%) Relational Database
13 51 (8.29%) SQL Server
13 51 (8.29%) Sqoop
14 40 (6.50%) MongoDB
15 36 (5.85%) Data Mining
16 33 (5.37%) Power BI
16 33 (5.37%) RDBMS
17 32 (5.20%) SQL Server Integration Services
18 27 (4.39%) Cosmos DB
Development Applications
1 42 (6.83%) Git (software)
1 42 (6.83%) Jenkins
2 26 (4.23%) TeamCity
3 10 (1.63%) GitLab
3 10 (1.63%) JUnit
4 6 (0.98%) RStudio
5 5 (0.81%) JIRA
5 5 (0.81%) Maven
5 5 (0.81%) SonarQube
6 3 (0.49%) Gradle
7 2 (0.33%) ERwin
7 2 (0.33%) Mercurial
7 2 (0.33%) Octopus Deploy
7 2 (0.33%) Robot Framework
8 1 (0.16%) Bitbucket
8 1 (0.16%) Gerrit
8 1 (0.16%) Selenium
8 1 (0.16%) SpecFlow
8 1 (0.16%) Subversion
8 1 (0.16%) Visual Studio Team System
General
1 97 (15.77%) Finance
2 48 (7.80%) Retail
3 44 (7.15%) Banking
4 33 (5.37%) Advertising
5 20 (3.25%) Marketing
6 13 (2.11%) Investment Banking
7 11 (1.79%) Legal
8 10 (1.63%) Telecoms
9 9 (1.46%) Games
10 7 (1.14%) Corporate Banking
11 6 (0.98%) Front Office
12 5 (0.81%) Electronics
12 5 (0.81%) Financial Institution
12 5 (0.81%) Law
12 5 (0.81%) Manufacturing
12 5 (0.81%) Retail Banking
13 4 (0.65%) Pharmaceutical
14 2 (0.33%) Automotive
14 2 (0.33%) Music Industry
15 1 (0.16%) French Language
Job Titles
1 230 (37.40%) Data Engineer
2 87 (14.15%) Data Scientist
3 81 (13.17%) Big Data Engineer
3 81 (13.17%) Developer
4 56 (9.11%) Senior Data Engineer
5 48 (7.80%) Architect
6 39 (6.34%) Senior Data Scientist
7 37 (6.02%) Hadoop Engineer
8 32 (5.20%) Python Engineer
9 31 (5.04%) Data Architect
10 24 (3.90%) Senior Hadoop Engineer
11 23 (3.74%) Consultant
12 22 (3.58%) Analyst
12 22 (3.58%) Data Consultant
13 21 (3.41%) AWS Engineer
13 21 (3.41%) Software Engineer
14 16 (2.60%) C++ Developer
14 16 (2.60%) Machine Learning Scientist
14 16 (2.60%) Software Developer
15 14 (2.28%) Python Developer
Libraries, Frameworks & Software Standards
1 135 (21.95%) Kafka
2 60 (9.76%) Natural Language Processing
3 59 (9.59%) TensorFlow
4 48 (7.80%) Pandas
5 44 (7.15%) NumPy
6 35 (5.69%) AngularJS
7 31 (5.04%) Keras
8 26 (4.23%) Web Services
9 25 (4.07%) .NET
9 25 (4.07%) MLlib
9 25 (4.07%) Node.js
10 24 (3.90%) SciPy
11 23 (3.74%) D3.js
11 23 (3.74%) JSON
11 23 (3.74%) React
11 23 (3.74%) REST
12 19 (3.09%) scikit-learn
13 16 (2.60%) Middleware
14 15 (2.44%) Express
14 15 (2.44%) JMS
Miscellaneous
1 91 (14.80%) Algorithms
2 69 (11.22%) Fintech
3 57 (9.27%) Computer Science
4 47 (7.64%) Analytical Skills
5 37 (6.02%) Greenfield Project
6 36 (5.85%) User Experience
7 31 (5.04%) Data Structures
8 16 (2.60%) Derivative
8 16 (2.60%) Self-Motivation
9 15 (2.44%) Enterprise Software
10 12 (1.95%) Tandem
11 10 (1.63%) Linux Command Line
12 8 (1.30%) Data Protection Act
12 8 (1.30%) Equities
12 8 (1.30%) Reference Data
13 6 (0.98%) Foreign Exchange (FX)
13 6 (0.98%) Management Information System
14 5 (0.81%) Cyberthreat
14 5 (0.81%) Semantic Web
14 5 (0.81%) Smart City
Operating Systems
1 102 (16.59%) Linux
2 19 (3.09%) Unix
3 6 (0.98%) Windows
4 4 (0.65%) Windows Server
Processes & Methodologies
1 265 (43.09%) Data Science
2 254 (41.30%) Machine Learning
3 235 (38.21%) Analytics
4 224 (36.42%) Agile Software Development
5 175 (28.46%) ETL
6 101 (16.42%) Artificial Intelligence
7 96 (15.61%) Data Modelling
8 89 (14.47%) Business Intelligence
9 87 (14.15%) Software Engineering
10 80 (13.01%) DevOps
10 80 (13.01%) Visualisation
11 76 (12.36%) MapReduce
12 67 (10.89%) Statistics
13 66 (10.73%) Problem-Solving
14 62 (10.08%) Deep Learning
15 60 (9.76%) Data Analytics
15 60 (9.76%) Mathematics
16 49 (7.97%) Open Source
16 49 (7.97%) Test Automation
17 47 (7.64%) Data Architecture
Programming Languages
1 404 (65.69%) Python
2 259 (42.11%) SQL
3 223 (36.26%) Java
4 203 (33.01%) Scala
5 95 (15.45%) R
6 49 (7.97%) Go
6 49 (7.97%) JavaScript
7 37 (6.02%) C#
8 34 (5.53%) C++
9 32 (5.20%) Shell Script
10 23 (3.74%) C
11 22 (3.58%) MATLAB
12 15 (2.44%) Bash Shell
13 14 (2.28%) Java 8
14 12 (1.95%) PowerShell
15 10 (1.63%) T-SQL
16 7 (1.14%) Ruby
17 6 (0.98%) Kotlin
18 4 (0.65%) Clojure
18 4 (0.65%) Groovy
Qualifications
1 100 (16.26%) Degree
2 27 (4.39%) PhD
3 24 (3.90%) Computer Science Degree
4 11 (1.79%) Master's Degree
4 11 (1.79%) Numerate Degree
4 11 (1.79%) SANS
5 9 (1.46%) SC Cleared
5 9 (1.46%) Security Cleared
6 8 (1.30%) CEH
6 8 (1.30%) CISA
6 8 (1.30%) CISSP
7 6 (0.98%) CISM
8 5 (0.81%) CompTIA Security+
8 5 (0.81%) CSSLP
8 5 (0.81%) GIAC
8 5 (0.81%) SSCP
9 3 (0.49%) EC-Council LPT
9 3 (0.49%) ECSA
10 2 (0.33%) AWS Certification
10 2 (0.33%) ITIL Certification
Quality Assurance & Compliance
1 29 (4.72%) Data Quality
2 22 (3.58%) GDPR
3 12 (1.95%) QA
4 7 (1.14%) GCP
5 2 (0.33%) ISO/IEC 27001
5 2 (0.33%) PCI DSS
6 1 (0.16%) SLA
System Software
1 65 (10.57%) HDFS
2 57 (9.27%) Apache Flume
3 52 (8.46%) Docker
4 9 (1.46%) Apache ZooKeeper
5 6 (0.98%) Active Directory
Systems Management
1 50 (8.13%) Hadoop YARN
2 37 (6.02%) Kubernetes
3 32 (5.20%) Oozie
4 23 (3.74%) Mesos
5 15 (2.44%) Ansible
5 15 (2.44%) Kibana
6 11 (1.79%) Terraform
7 10 (1.63%) Opscode Chef
8 7 (1.14%) Consul
9 4 (0.65%) Ambari
9 4 (0.65%) Docker Swarm
9 4 (0.65%) Puppet
10 3 (0.49%) CASB
11 1 (0.16%) Packer
Vendors
1 57 (9.27%) Cloudera
2 49 (7.97%) Hortonworks
2 49 (7.97%) Oracle
3 45 (7.32%) Google
4 44 (7.15%) Tableau
5 42 (6.83%) WorldPay
6 33 (5.37%) Informatica
6 33 (5.37%) Talend
7 32 (5.20%) Microsoft
8 23 (3.74%) MapR
9 20 (3.25%) RapidMiner
10 19 (3.09%) Netezza
11 17 (2.76%) Sybase
12 13 (2.11%) Teradata
13 10 (1.63%) IBM
14 9 (1.46%) Salesforce.com
15 3 (0.49%) Databricks
15 3 (0.49%) MicroStrategy
16 2 (0.33%) CA Agile Central
16 2 (0.33%) SnapLogic