Period
to 17 February 2019

The following table provides summary statistics for permanent job vacancies with a requirement for Data Engineering skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Engineering over the 6 months to 17 February 2019 with a comparison to the same period in the previous 2 years.

Data Engineering
UK
6 months to
17 Feb 2019
Same period 2018 Same period 2017
Rank 461 765 1000
Rank change year-on-year +304 +235 +156
Permanent jobs citing Data Engineering 952 446 200
As % of all permanent IT jobs advertised in the UK 0.64% 0.25% 0.11%
As % of the Processes & Methodologies category 0.69% 0.28% 0.12%
Number of salaries quoted 661 384 173
UK median annual salary £65,000 £67,500 £70,000
Median salary % change year-on-year -3.70% -3.57% +5.66%
10th Percentile £45,000 £42,500 £46,250
90th Percentile £92,500 £100,000 £92,500
UK excluding London median annual salary £62,138 £50,000 £55,000
% change year-on-year +24.28% -9.09% +10.00%

Data Engineering is in the Processes and Methodologies category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for process or methodology skills.

All Process and Methodology Skills
UK
Permanent vacancies with a requirement for process or methodology skills 137,171 160,827 160,381
As % of all permanent IT jobs advertised in the UK 91.74% 91.83% 91.31%
Number of salaries quoted 108,142 130,549 132,251
UK median annual salary £52,500 £50,000 £47,500
Median salary % change year-on-year +5.00% +5.26% -
10th Percentile £30,000 £28,750 £27,750
90th Percentile £85,000 £80,000 £78,750
UK excluding London median annual salary £45,000 £45,000 £42,500
% change year-on-year - +5.88% -

Data Engineering
Job Vacancy Trend

Job postings citing Data Engineering as a percentage of all IT jobs advertised.

Job vacancy trend for Data Engineering in the UK

Data Engineering
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Data Engineering.

Salary trend for Data Engineering in the UK

Data Engineering
Salary Histogram

The salary distribution of IT jobs citing Data Engineering over the 6 months to 17 February 2019.

Salary histogram for Data Engineering in the UK

Data Engineering
Top 15 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Engineering within the UK over the 6 months to 17 February 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +312 905 £65,000 -3.70% 129
UK excluding London +321 475 £62,138 +24.28% 59
London +165 475 £67,500 -9.40% 77
South East +180 171 £67,500 +31.71% 12
South West +61 97 £60,000 +14.29% 11
North of England +135 96 £65,707 +38.33% 9
North West +94 59 £70,000 +27.27% 5
Scotland +39 41 £48,500 +6.01% 7
Midlands +76 39 £53,500 -19.25% 7
East of England +40 26 £58,750 +6.82% 13
Yorkshire +76 25 £63,354 +49.07% 3
East Midlands +25 20 £51,750 -17.20% 3
West Midlands +62 19 £64,733 -21.54% 4
North East +26 12 £49,337 +51.81% 1
Wales +15 5 £47,500 +23.38%

For the 6 months to 17 February 2019, IT jobs citing Data Engineering also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Data Engineering.

1 545 (57.25%) Python
2 515 (54.10%) SQL
3 452 (47.48%) Analytics
4 433 (45.48%) Big Data
5 411 (43.17%) Data Science
6 408 (42.86%) Agile Software Development
7 403 (42.33%) Hadoop
8 380 (39.92%) ETL
9 372 (39.08%) Data Modelling
10 362 (38.03%) Apache Spark
11 311 (32.67%) Microsoft Azure
12 302 (31.72%) Business Intelligence
13 285 (29.94%) Amazon AWS
14 266 (27.94%) Java
15 260 (27.31%) Data Warehouse
16 254 (26.68%) Degree
17 252 (26.47%) Scala
18 240 (25.21%) Kafka
19 234 (24.58%) Machine Learning
20 218 (22.90%) NoSQL
21 213 (22.37%) Finance
22 196 (20.59%) Apache Hive
23 191 (20.06%) Visualisation
24 180 (18.91%) Artificial Intelligence
25 179 (18.80%) Power BI
26 167 (17.54%) Tableau
27 166 (17.44%) DevOps
28 151 (15.86%) Analytical Skills
29 149 (15.65%) Data Integration
30 144 (15.13%) Retail

Data Engineering
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 362 (38.03%) Apache Spark
2 142 (14.92%) Apache
3 85 (8.93%) SAS
4 34 (3.57%) Apache Pig
5 19 (2.00%) Apache Airflow
6 16 (1.68%) Elasticsearch
7 12 (1.26%) Apache Solr
8 9 (0.95%) Luigi
9 8 (0.84%) Jupyter
10 5 (0.53%) WebSphere
11 2 (0.21%) Drupal
12 1 (0.11%) Confluence
12 1 (0.11%) Tornado
Applications
1 29 (3.05%) MS Visio
2 7 (0.74%) SPSS
3 6 (0.63%) Microsoft Office
4 4 (0.42%) Microsoft Excel
5 1 (0.11%) Microsoft Project
Business Applications
1 5 (0.53%) Magento
2 2 (0.21%) NetSuite
3 1 (0.11%) Odoo
Cloud Services
1 311 (32.67%) Microsoft Azure
2 285 (29.94%) Amazon AWS
3 120 (12.61%) Google Cloud Platform
4 70 (7.35%) Cloud Computing
5 52 (5.46%) Amazon EMR
6 50 (5.25%) Amazon EC2
7 48 (5.04%) Serverless
8 40 (4.20%) Azure Data Factory
9 35 (3.68%) GitHub
10 23 (2.42%) AWS Lambda
11 20 (2.10%) PaaS
12 18 (1.89%) Amazon S3
13 16 (1.68%) SaaS
14 15 (1.58%) Azure Functions
15 11 (1.16%) Amazon Kinesis
15 11 (1.16%) Data Lake Analytics
16 6 (0.63%) AWS CloudFormation
16 6 (0.63%) AWS Data Pipeline
16 6 (0.63%) Azure Machine Learning
17 4 (0.42%) IaaS
Communications & Networking
1 20 (2.10%) Internet
2 10 (1.05%) HTTP
3 2 (0.21%) Kerberos
3 2 (0.21%) TCP/IP
4 1 (0.11%) DNS
Database & Business Intelligence
1 433 (45.48%) Big Data
2 403 (42.33%) Hadoop
3 260 (27.31%) Data Warehouse
4 218 (22.90%) NoSQL
5 196 (20.59%) Apache Hive
6 179 (18.80%) Power BI
7 139 (14.60%) MongoDB
8 115 (12.08%) Relational Database
9 109 (11.45%) Data Lake
10 108 (11.34%) HBase
11 90 (9.45%) Amazon Redshift
12 88 (9.24%) Apache Cassandra
13 85 (8.93%) SQL Server
14 73 (7.67%) Sqoop
15 68 (7.14%) SQL Server Integration Services
16 49 (5.15%) Flink
16 49 (5.15%) SAP BW
17 47 (4.94%) PostgreSQL
18 46 (4.83%) Text Mining
19 43 (4.52%) SAP HANA
Development Applications
1 56 (5.88%) JIRA
2 46 (4.83%) Jenkins
3 45 (4.73%) Git (software)
4 15 (1.58%) Artifactory
4 15 (1.58%) Bitbucket
4 15 (1.58%) Maven
4 15 (1.58%) Octopus Deploy
4 15 (1.58%) TeamCity
5 14 (1.47%) tSQLt
6 11 (1.16%) Visual Studio Team System
7 10 (1.05%) SpecFlow
8 6 (0.63%) Eclipse
8 6 (0.63%) IntelliJ
8 6 (0.63%) Visual Studio
9 5 (0.53%) JUnit
9 5 (0.53%) RStudio
9 5 (0.53%) Selenium
10 4 (0.42%) Team Foundation Server
11 2 (0.21%) FindBugs
11 2 (0.21%) PHPUnit
General
1 213 (22.37%) Finance
2 144 (15.13%) Retail
3 90 (9.45%) Telecoms
4 57 (5.99%) Law
5 51 (5.36%) Military
6 39 (4.10%) Marketing
7 35 (3.68%) Banking
8 13 (1.37%) Advertising
9 10 (1.05%) Games
9 10 (1.05%) Investment Banking
10 9 (0.95%) Corporate Banking
11 8 (0.84%) Legal
12 6 (0.63%) Publishing
13 5 (0.53%) Automotive
13 5 (0.53%) Front Office
14 2 (0.21%) Digital Economy
14 2 (0.21%) Electronics
14 2 (0.21%) French Language
14 2 (0.21%) Pharmaceutical
15 1 (0.11%) Prime Brokerage
Job Titles
1 460 (48.32%) Data Engineer
2 126 (13.24%) Big Data Engineer
3 97 (10.19%) Python Engineer
4 93 (9.77%) Senior Data Engineer
5 91 (9.56%) Developer
6 90 (9.45%) Data Scientist
7 66 (6.93%) Architect
8 54 (5.67%) Data Architect
9 49 (5.15%) Analyst
10 46 (4.83%) Hadoop Engineer
11 40 (4.20%) Senior Data Scientist
12 34 (3.57%) Data Analyst
13 33 (3.47%) Consultant
14 30 (3.15%) Senior Architect
15 29 (3.05%) Data Manager
16 28 (2.94%) Azure Engineer
17 24 (2.52%) Senior Data Architect
18 17 (1.79%) Business Intelligence Developer
18 17 (1.79%) Software Engineer
19 16 (1.68%) Cloud Engineer
Libraries, Frameworks & Software Standards
1 240 (25.21%) Kafka
2 91 (9.56%) Spark Streaming
3 69 (7.25%) Web Services
4 52 (5.46%) Apache NiFi
5 44 (4.62%) D3.js
6 41 (4.31%) ggplot
7 33 (3.47%) Natural Language Processing
8 30 (3.15%) Pandas
9 28 (2.94%) .NET
10 27 (2.84%) NumPy
11 26 (2.73%) scikit-learn
12 21 (2.21%) TensorFlow
13 20 (2.10%) REST
14 18 (1.89%) JSON
15 16 (1.68%) Apache Beam
15 16 (1.68%) SciPy
16 15 (1.58%) XML
17 14 (1.47%) Matplotlib
18 13 (1.37%) AngularJS
18 13 (1.37%) Node.js
Miscellaneous
1 151 (15.86%) Analytical Skills
2 77 (8.09%) Algorithms
3 60 (6.30%) Data Structures
4 49 (5.15%) FMCG
5 41 (4.31%) Social Network
6 38 (3.99%) Fintech
7 25 (2.63%) Greenfield Project
8 21 (2.21%) Internet of Things
9 20 (2.10%) Management Information System
10 19 (2.00%) Self-Motivation
11 18 (1.89%) Semantic Web
12 16 (1.68%) Cloud Native
13 15 (1.58%) Linked Data
14 14 (1.47%) Renewable Energy
14 14 (1.47%) User Experience
15 9 (0.95%) Reference Data
16 7 (0.74%) Linux Command Line
16 7 (0.74%) Virtual Team
17 6 (0.63%) Marketing Technology
17 6 (0.63%) NHS
Operating Systems
1 120 (12.61%) Linux
2 86 (9.03%) Unix
3 41 (4.31%) Windows
4 5 (0.53%) Apple iOS
5 2 (0.21%) Android
5 2 (0.21%) Mac OS X
6 1 (0.11%) Red Hat Enterprise Linux
Processes & Methodologies
1 452 (47.48%) Analytics
2 411 (43.17%) Data Science
3 408 (42.86%) Agile Software Development
4 380 (39.92%) ETL
5 372 (39.08%) Data Modelling
6 302 (31.72%) Business Intelligence
7 234 (24.58%) Machine Learning
8 191 (20.06%) Visualisation
9 180 (18.91%) Artificial Intelligence
10 166 (17.44%) DevOps
11 149 (15.65%) Data Integration
12 135 (14.18%) Data Visualisation
13 134 (14.08%) Data Analysis
13 134 (14.08%) Data Management
14 128 (13.45%) Decision-Making
15 121 (12.71%) Mathematics
16 119 (12.50%) Stakeholder Management
17 113 (11.87%) Computer Science
18 101 (10.61%) Data Security
18 101 (10.61%) Use Case
Programming Languages
1 545 (57.25%) Python
2 515 (54.10%) SQL
3 266 (27.94%) Java
4 252 (26.47%) Scala
5 129 (13.55%) R
6 53 (5.57%) C#
7 43 (4.52%) T-SQL
8 26 (2.73%) PL/SQL
9 20 (2.10%) Bash Shell
9 20 (2.10%) Ruby
10 17 (1.79%) Go
11 16 (1.68%) U-SQL
12 15 (1.58%) C
12 15 (1.58%) JavaScript
12 15 (1.58%) PowerShell
13 14 (1.47%) Shell Script
14 11 (1.16%) MATLAB
15 8 (0.84%) C++
15 8 (0.84%) Java 8
16 7 (0.74%) ES6
Qualifications
1 254 (26.68%) Degree
2 68 (7.14%) Computer Science Degree
3 43 (4.52%) Numerate Degree
4 33 (3.47%) Microsoft Certification
5 29 (3.05%) MCSA
6 24 (2.52%) PhD
7 19 (2.00%) Master's Degree
8 8 (0.84%) CISM
8 8 (0.84%) CISSP
9 7 (0.74%) CISA
10 6 (0.63%) CEH
10 6 (0.63%) CSSLP
10 6 (0.63%) GIAC
10 6 (0.63%) SANS
10 6 (0.63%) SSCP
11 4 (0.42%) MCAD
11 4 (0.42%) MCDBA
11 4 (0.42%) MCSD
12 3 (0.32%) AWS Certification
12 3 (0.32%) Security Cleared
Quality Assurance & Compliance
1 76 (7.98%) Data Quality
2 30 (3.15%) QA
3 5 (0.53%) GDPR
4 3 (0.32%) ISO/IEC 27001
5 1 (0.11%) PMO
5 1 (0.11%) SLA
System Software
1 64 (6.72%) HDFS
2 60 (6.30%) Docker
3 50 (5.25%) Apache Flume
4 28 (2.94%) Apache ZooKeeper
5 1 (0.11%) VirtualBox
Systems Management
1 77 (8.09%) Oozie
2 56 (5.88%) Hadoop YARN
3 39 (4.10%) Kubernetes
4 23 (2.42%) Terraform
5 21 (2.21%) Ansible
6 12 (1.26%) Ambari
7 8 (0.84%) Mesos
8 4 (0.42%) Kibana
9 3 (0.32%) Opscode Chef
9 3 (0.32%) Puppet
10 2 (0.21%) Docker Swarm
10 2 (0.21%) logstash
11 1 (0.11%) Grafana
11 1 (0.11%) Graylog
11 1 (0.11%) Prometheus
11 1 (0.11%) Salt
Vendors
1 167 (17.54%) Tableau
2 142 (14.92%) Microsoft
3 132 (13.87%) SAP
4 121 (12.71%) Oracle
5 99 (10.40%) Cloudera
6 90 (9.45%) Google
7 63 (6.62%) Hortonworks
8 50 (5.25%) Informatica
9 46 (4.83%) CA Agile Central
10 36 (3.78%) Teradata
11 32 (3.36%) Netezza
12 22 (2.31%) WorldPay
13 21 (2.21%) Ab Initio
13 21 (2.21%) MapR
14 14 (1.47%) Cognos
15 12 (1.26%) Alteryx
16 11 (1.16%) MicroStrategy
17 10 (1.05%) Talend
18 8 (0.84%) Databricks
18 8 (0.84%) Sybase