Period
to 22 March 2019

The following table provides summary statistics for permanent job vacancies advertised in London with a requirement for Data Engineering skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Engineering over the 6 months to 22 March 2019 with a comparison to the same period in the previous 2 years.

Data Engineering
England > London
6 months to
22 Mar 2019
Same period 2018 Same period 2017
Rank 339 505 664
Rank change year-on-year +166 +159 +115
Permanent jobs citing Data Engineering 551 346 170
As % of all permanent IT jobs advertised in London 0.89% 0.50% 0.25%
As % of the Processes & Methodologies category 0.97% 0.54% 0.27%
Number of salaries quoted 455 287 156
London median annual salary £70,000 £72,500 £70,000
Median salary % change year-on-year -3.45% +3.57% -
10th Percentile £48,750 £47,500 £49,375
90th Percentile £107,000 £102,500 £95,000
England median annual salary £67,500 £67,500 £70,000
% change year-on-year - -3.57% +5.66%

Data Engineering is in the Processes and Methodologies category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in London with a requirement for process or methodology skills.

All Process and Methodology Skills
London
Permanent vacancies with a requirement for process or methodology skills 56,885 64,341 63,169
As % of all permanent IT jobs advertised in London 92.19% 92.76% 92.11%
Number of salaries quoted 45,986 53,167 53,167
London median annual salary £62,500 £60,000 £60,000
Median salary % change year-on-year +4.17% - +9.09%
10th Percentile £36,250 £35,000 £33,750
90th Percentile £95,000 £92,500 £90,000
England median annual salary £52,500 £50,000 £49,155
% change year-on-year +5.00% +1.72% +3.48%

Data Engineering
Job Vacancy Trend in London

Job postings citing Data Engineering as a percentage of all IT jobs advertised in London.

Job vacancy trend for Data Engineering in London

Data Engineering
Salary Trend in London

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Data Engineering in London.

Salary trend for Data Engineering in London

Data Engineering
Salary Histogram in London

The salary distribution of IT jobs citing Data Engineering in London over the 6 months to 22 March 2019.

Salary histogram for Data Engineering in London

Data Engineering
Job Locations in London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Engineering within the London region over the 6 months to 22 March 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +60 93 £65,000 -14.75% 25
West London +28 17 £80,000 -15.79% 13
North London +25 8 £86,250 - 3
East London +28 4 £62,500 - 9
South London - 2 £65,000 - 2
Data Engineering
England

For the 6 months to 22 March 2019, IT jobs citing Data Engineering also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the London region with a requirement for Data Engineering.

1 395 (71.69%) Python
2 326 (59.17%) SQL
3 269 (48.82%) Big Data
4 246 (44.65%) Hadoop
5 229 (41.56%) Apache Spark
6 226 (41.02%) Amazon AWS
7 225 (40.83%) Analytics
8 204 (37.02%) Agile Software Development
9 196 (35.57%) Java
10 188 (34.12%) Microsoft Azure
11 185 (33.58%) ETL
11 185 (33.58%) Data Science
12 179 (32.49%) Machine Learning
13 169 (30.67%) Data Modelling
14 164 (29.76%) Scala
15 151 (27.40%) Business Intelligence
16 143 (25.95%) NoSQL
17 140 (25.41%) Data Warehouse
18 139 (25.23%) Kafka
19 123 (22.32%) Google Cloud Platform
20 121 (21.96%) Apache Hive
21 119 (21.60%) Analytical Skills
22 110 (19.96%) DevOps
23 99 (17.97%) Software Engineering
24 98 (17.79%) Linux
25 94 (17.06%) Google
26 89 (16.15%) Relational Database
26 89 (16.15%) HBase
27 85 (15.43%) MongoDB
28 78 (14.16%) Visualisation

Data Engineering
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 229 (41.56%) Apache Spark
2 55 (9.98%) Apache
3 49 (8.89%) Apache Pig
4 21 (3.81%) SAS
5 20 (3.63%) Apache Airflow
6 13 (2.36%) Apache Solr
6 13 (2.36%) Elasticsearch
7 10 (1.81%) Luigi
8 5 (0.91%) WebSphere
9 3 (0.54%) Jupyter
10 1 (0.18%) Confluence
Applications
1 22 (3.99%) MS Visio
2 5 (0.91%) SPSS
Business Applications
1 5 (0.91%) Magento
2 2 (0.36%) PolarLake
3 1 (0.18%) Odoo
Cloud Services
1 226 (41.02%) Amazon AWS
2 188 (34.12%) Microsoft Azure
3 123 (22.32%) Google Cloud Platform
4 40 (7.26%) Serverless
5 31 (5.63%) Azure Data Factory
6 23 (4.17%) PaaS
7 20 (3.63%) AWS Lambda
8 17 (3.09%) Cloud Computing
9 16 (2.90%) SaaS
10 15 (2.72%) Azure Functions
11 14 (2.54%) Amazon S3
12 12 (2.18%) Amazon Kinesis
12 12 (2.18%) GitHub
13 10 (1.81%) Google Analytics
14 9 (1.63%) Data Lake Analytics
15 8 (1.45%) Amazon EC2
15 8 (1.45%) Amazon EMR
15 8 (1.45%) IaaS
16 6 (1.09%) AWS CloudFormation
17 5 (0.91%) Azure Machine Learning
Communications & Networking
1 12 (2.18%) HTTP
2 3 (0.54%) TCP/IP
3 1 (0.18%) DNS
3 1 (0.18%) Internet
3 1 (0.18%) Wireless
Database & Business Intelligence
1 269 (48.82%) Big Data
2 246 (44.65%) Hadoop
3 143 (25.95%) NoSQL
4 140 (25.41%) Data Warehouse
5 121 (21.96%) Apache Hive
6 89 (16.15%) HBase
6 89 (16.15%) Relational Database
7 85 (15.43%) MongoDB
8 76 (13.79%) Power BI
9 72 (13.07%) Apache Cassandra
10 56 (10.16%) Data Lake
11 55 (9.98%) Sqoop
12 44 (7.99%) SQL Server
13 43 (7.80%) SQL Server Integration Services
14 35 (6.35%) Amazon Redshift
15 33 (5.99%) PostgreSQL
16 29 (5.26%) QlikView
17 24 (4.36%) BigQuery
17 24 (4.36%) MySQL
18 20 (3.63%) Cosmos DB
Development Applications
1 42 (7.62%) Jenkins
2 36 (6.53%) Git (software)
3 13 (2.36%) Artifactory
3 13 (2.36%) TeamCity
4 12 (2.18%) tSQLt
5 11 (2.00%) Maven
5 11 (2.00%) Octopus Deploy
6 9 (1.63%) Bitbucket
6 9 (1.63%) Visual Studio Team System
7 8 (1.45%) SpecFlow
8 6 (1.09%) Visual Studio
9 4 (0.73%) Atlassian Bamboo
9 4 (0.73%) JIRA
9 4 (0.73%) Team Foundation Server
10 3 (0.54%) RStudio
10 3 (0.54%) Selenium
10 3 (0.54%) Travis CI
11 2 (0.36%) Eclipse
11 2 (0.36%) IntelliJ
11 2 (0.36%) JUnit
General
1 76 (13.79%) Finance
2 72 (13.07%) Law
3 68 (12.34%) Military
4 41 (7.44%) Retail
5 31 (5.63%) Banking
6 13 (2.36%) Marketing
7 11 (2.00%) Advertising
8 8 (1.45%) Telecoms
9 6 (1.09%) Games
10 5 (0.91%) Corporate Banking
10 5 (0.91%) Investment Banking
11 4 (0.73%) Automotive
12 3 (0.54%) Publishing
13 2 (0.36%) Digital Economy
13 2 (0.36%) French Language
13 2 (0.36%) Music Industry
14 1 (0.18%) Financial Institution
14 1 (0.18%) Front Office
14 1 (0.18%) Legal
Job Titles
1 278 (50.45%) Data Engineer
2 78 (14.16%) Python Engineer
3 66 (11.98%) Big Data Engineer
4 52 (9.44%) Hadoop Engineer
5 47 (8.53%) Developer
6 44 (7.99%) Architect
7 43 (7.80%) Senior Data Engineer
8 37 (6.72%) Data Architect
9 35 (6.35%) Analyst
10 29 (5.26%) Data Scientist
11 24 (4.36%) Data Analyst
12 18 (3.27%) Cloud Engineer
12 18 (3.27%) Consultant
13 17 (3.09%) Azure Engineer
14 16 (2.90%) Data Manager
14 16 (2.90%) Head of Engineering
15 14 (2.54%) Python Developer
15 14 (2.54%) Senior Data Scientist
15 14 (2.54%) Senior Hadoop Engineer
16 13 (2.36%) Solutions Architect
Libraries, Frameworks & Software Standards
1 139 (25.23%) Kafka
2 61 (11.07%) Apache NiFi
3 35 (6.35%) Spark Streaming
4 25 (4.54%) Natural Language Processing
5 24 (4.36%) Web Services
6 23 (4.17%) Pandas
7 18 (3.27%) Apache Beam
8 17 (3.09%) NumPy
8 17 (3.09%) scikit-learn
9 16 (2.90%) TensorFlow
10 14 (2.54%) REST
11 13 (2.36%) PySpark
12 12 (2.18%) SciPy
13 11 (2.00%) Akka
13 11 (2.00%) JSON
14 10 (1.81%) .NET
14 10 (1.81%) AngularJS
14 10 (1.81%) MLlib
15 9 (1.63%) HTML
16 8 (1.45%) XML
Miscellaneous
1 119 (21.60%) Analytical Skills
2 55 (9.98%) Data Structures
3 41 (7.44%) Fintech
4 23 (4.17%) Algorithms
5 18 (3.27%) User Experience
6 15 (2.72%) Cloud Native
7 14 (2.54%) Semantic Web
8 13 (2.36%) Self-Motivation
9 12 (2.18%) Linked Data
10 11 (2.00%) Greenfield Project
10 11 (2.00%) Virtual Team
11 8 (1.45%) Linux Command Line
11 8 (1.45%) Reference Data
12 7 (1.27%) Hybrid Cloud
13 6 (1.09%) Management Information System
13 6 (1.09%) Robotics
14 5 (0.91%) Credit Risk
14 5 (0.91%) Renewable Energy
15 3 (0.54%) Mobile App
15 3 (0.54%) Public Cloud
Operating Systems
1 98 (17.79%) Linux
2 51 (9.26%) Unix
3 29 (5.26%) Windows
4 4 (0.73%) Mac OS X
5 2 (0.36%) Apple iOS
6 1 (0.18%) Android
Processes & Methodologies
1 225 (40.83%) Analytics
2 204 (37.02%) Agile Software Development
3 185 (33.58%) Data Science
3 185 (33.58%) ETL
4 179 (32.49%) Machine Learning
5 169 (30.67%) Data Modelling
6 151 (27.40%) Business Intelligence
7 110 (19.96%) DevOps
8 99 (17.97%) Software Engineering
9 78 (14.16%) Visualisation
10 66 (11.98%) Data Analysis
11 64 (11.62%) Artificial Intelligence
12 62 (11.25%) Problem-Solving
13 59 (10.71%) Use Case
14 54 (9.80%) Data Integration
15 52 (9.44%) Microservices
16 51 (9.26%) Data Management
17 50 (9.07%) Computer Science
18 49 (8.89%) Data Architecture
19 47 (8.53%) Data Analytics
Programming Languages
1 395 (71.69%) Python
2 326 (59.17%) SQL
3 196 (35.57%) Java
4 164 (29.76%) Scala
5 52 (9.44%) R
6 34 (6.17%) C#
7 23 (4.17%) PL/SQL
7 23 (4.17%) T-SQL
8 19 (3.45%) Go
9 13 (2.36%) C
9 13 (2.36%) JavaScript
10 12 (2.18%) Ruby
10 12 (2.18%) U-SQL
11 11 (2.00%) Bash Shell
11 11 (2.00%) PowerShell
12 10 (1.81%) Kotlin
13 7 (1.27%) Clojure
13 7 (1.27%) ES6
13 7 (1.27%) Jython
13 7 (1.27%) TypeScript
Qualifications
1 56 (10.16%) Degree
2 27 (4.90%) PhD
3 26 (4.72%) Microsoft Certification
4 22 (3.99%) MCSA
5 10 (1.81%) Master's Degree
6 8 (1.45%) Numerate Degree
7 6 (1.09%) Computer Science Degree
8 4 (0.73%) AWS Certification
8 4 (0.73%) CISM
8 4 (0.73%) CISSP
8 4 (0.73%) MCAD
8 4 (0.73%) MCDBA
8 4 (0.73%) MCSD
9 2 (0.36%) CISA
10 1 (0.18%) CEH
10 1 (0.18%) CompTIA Security+
10 1 (0.18%) CSSLP
10 1 (0.18%) GIAC
10 1 (0.18%) SANS
10 1 (0.18%) Security Cleared
Quality Assurance & Compliance
1 48 (8.71%) Data Quality
2 24 (4.36%) QA
3 2 (0.36%) SLA
4 1 (0.18%) GDPR
4 1 (0.18%) ISO/IEC 27001
System Software
1 70 (12.70%) Docker
2 38 (6.90%) Apache ZooKeeper
3 31 (5.63%) HDFS
4 22 (3.99%) Apache Flume
5 1 (0.18%) VirtualBox
Systems Management
1 64 (11.62%) Oozie
2 49 (8.89%) Kubernetes
3 23 (4.17%) Terraform
4 21 (3.81%) Ambari
4 21 (3.81%) Ansible
5 11 (2.00%) Hadoop YARN
6 8 (1.45%) Opscode Chef
7 7 (1.27%) Mesos
8 4 (0.73%) Kibana
8 4 (0.73%) Puppet
9 2 (0.36%) Docker Swarm
9 2 (0.36%) logstash
10 1 (0.18%) Grafana
10 1 (0.18%) Graylog
10 1 (0.18%) Salt
Vendors
1 94 (17.06%) Google
2 78 (14.16%) Cloudera
3 75 (13.61%) Tableau
4 73 (13.25%) Microsoft
5 48 (8.71%) Oracle
6 36 (6.53%) Hortonworks
7 22 (3.99%) WorldPay
8 16 (2.90%) Informatica
8 16 (2.90%) SAP
9 13 (2.36%) Databricks
10 12 (2.18%) Talend
11 10 (1.81%) Netezza
12 9 (1.63%) Splunk
12 9 (1.63%) Sybase
13 7 (1.27%) MapR
14 6 (1.09%) Algorithmics
14 6 (1.09%) Alteryx
14 6 (1.09%) Facebook
15 5 (0.91%) hybris
15 5 (0.91%) Intershop