Period
to 14 December 2018

The following table provides summary statistics for contract job vacancies advertised in the City of London with a requirement for Data Engineering skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Data Engineering over the 6 months to 14 December 2018 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Data Engineering
Central London > City of London
6 months to
14 Dec 2018
Same period 2017 Same period 2016
Rank 213 220 229
Rank change year-on-year +7 +9 -
Contract jobs citing Data Engineering 40 38 9
As % of all contract IT jobs advertised in the City of London 0.56% 0.53% 0.14%
As % of the Processes & Methodologies category 0.64% 0.59% 0.16%
Number of daily rates quoted 29 38 7
City of London median daily rate £560 £550 £593
Median daily rate % change year-on-year +1.82% -7.17% -
10th Percentile £411 £501 £440
90th Percentile £660 £675 £731
Central London median daily rate £555 £550 £575
% change year-on-year +0.91% -4.35% -
Number of hourly rates quoted 0 0 0
City of London median hourly rate - - -
Central London median hourly rate - - -

Data Engineering is in the Processes and Methodologies category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in the City of London with a requirement for process or methodology skills.

All Process and Methodology Skills
City of London
Contract vacancies with a requirement for process or methodology skills 6,218 6,438 5,799
As % of all contract IT jobs advertised in the City of London 87.33% 89.54% 88.60%
Number of daily rates quoted 4,898 5,137 4,593
City of London median daily rate £513 £505 £475
Median daily rate % change year-on-year +1.49% +6.32% -
10th Percentile £325 £325 £319
90th Percentile £675 £688 £675
Central London median daily rate £505 £500 £475
% change year-on-year +1.00% +5.26% +2.70%
Number of hourly rates quoted 84 60 53
City of London median hourly rate £18.85 £18.70 £20.00
Median hourly rate % change year-on-year +0.80% -6.50% -
10th Percentile £13.31 £13.15 £13.75
90th Percentile £34.40 £31.05 £55.00
Central London median hourly rate £19.00 £20.25 £19.00
% change year-on-year -6.17% +6.58% -5.00%

Data Engineering
Job Vacancy Trend in the City of London

Job postings citing Data Engineering as a percentage of all IT jobs advertised in the City of London.

Job vacancy trend for Data Engineering in the City of London

Data Engineering
Contractor Daily Rate Trend in the City of London

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Data Engineering in the City of London.

Contractor daily rate trend for Data Engineering in the City of London

Data Engineering
Contractor Daily Rate Histogram in the City of London

The daily rate distribution of IT jobs citing Data Engineering in the City of London over the 6 months to 14 December 2018.

Contractor daily rate histogram for Data Engineering in the City of London

For the 6 months to 14 December 2018, IT contractor jobs citing Data Engineering also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the City of London region with a requirement for Data Engineering.

1 18 (45.00%) Analytics
2 17 (42.50%) Python
3 15 (37.50%) Finance
4 14 (35.00%) Apache Spark
5 13 (32.50%) Data Science
6 12 (30.00%) Hadoop
7 11 (27.50%) Machine Learning
7 11 (27.50%) Big Data
7 11 (27.50%) Java
8 10 (25.00%) SQL
8 10 (25.00%) Amazon AWS
9 9 (22.50%) Microsoft Azure
9 9 (22.50%) Jenkins
10 7 (17.50%) Scala
10 7 (17.50%) KDB
10 7 (17.50%) Linux
10 7 (17.50%) Artificial Intelligence
11 6 (15.00%) Agile Software Development
11 6 (15.00%) Git (software)
11 6 (15.00%) Kafka
11 6 (15.00%) Data Structures
11 6 (15.00%) Algorithms
12 5 (12.50%) Kubernetes
12 5 (12.50%) scikit-learn
12 5 (12.50%) HDFS
12 5 (12.50%) Problem-Solving
12 5 (12.50%) Task Automation
12 5 (12.50%) NumPy
12 5 (12.50%) MapReduce
12 5 (12.50%) Tableau

Data Engineering
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 14 (35.00%) Apache Spark
2 3 (7.50%) Apache
3 1 (2.50%) Apache Airflow
3 1 (2.50%) SAS
Applications
1 2 (5.00%) MS Visio
2 1 (2.50%) Chatbot
Business Applications
1 1 (2.50%) Unica
Cloud Services
1 10 (25.00%) Amazon AWS
2 9 (22.50%) Microsoft Azure
3 3 (7.50%) Amazon EC2
3 3 (7.50%) Amazon S3
3 3 (7.50%) GitHub
3 3 (7.50%) Google Cloud Platform
3 3 (7.50%) IBM Cloud
4 2 (5.00%) Amazon Kinesis
4 2 (5.00%) AWS CloudFormation
4 2 (5.00%) AWS Lambda
4 2 (5.00%) Serverless
Communications & Networking
1 1 (2.50%) Firewall
1 1 (2.50%) LAN
Database & Business Intelligence
1 12 (30.00%) Hadoop
2 11 (27.50%) Big Data
3 7 (17.50%) KDB
4 5 (12.50%) Data Warehouse
5 4 (10.00%) Apache Hive
5 4 (10.00%) PostgreSQL
5 4 (10.00%) Relational Database
5 4 (10.00%) Tick Data
6 3 (7.50%) Data Lake
6 3 (7.50%) DB2
6 3 (7.50%) Kdb+
6 3 (7.50%) NoSQL
6 3 (7.50%) Sqoop
7 2 (5.00%) Amazon Athena
7 2 (5.00%) Amazon QuickSight
7 2 (5.00%) Amazon Redshift
7 2 (5.00%) Data Mining
7 2 (5.00%) Power BI
7 2 (5.00%) SQL Server
8 1 (2.50%) Oracle Database
Development Applications
1 9 (22.50%) Jenkins
2 6 (15.00%) Git (software)
3 3 (7.50%) Maven
3 3 (7.50%) Robot Framework
4 1 (2.50%) ERwin
4 1 (2.50%) JIRA
4 1 (2.50%) SonarQube
4 1 (2.50%) Team Foundation Server
4 1 (2.50%) Toad
General
1 15 (37.50%) Finance
2 4 (10.00%) Banking
3 1 (2.50%) Financial Institution
3 1 (2.50%) Games
3 1 (2.50%) Marketing
3 1 (2.50%) Retail
Job Titles
1 15 (37.50%) Data Engineer
2 12 (30.00%) Developer
3 7 (17.50%) KDB Developer
4 4 (10.00%) Administrator
4 4 (10.00%) Hadoop Administrator
5 3 (7.50%) Big Data Engineer
5 3 (7.50%) Scala Developer
5 3 (7.50%) Senior Developer
6 2 (5.00%) Apache Spark Developer
6 2 (5.00%) AWS Developer
6 2 (5.00%) AWS Engineer
6 2 (5.00%) Data Scientist
6 2 (5.00%) Machine Learning Developer
6 2 (5.00%) Python Engineer
6 2 (5.00%) Scrum Master
6 2 (5.00%) Senior Data Engineer
7 1 (2.50%) Campaign Analyst
7 1 (2.50%) JavaScript Developer
7 1 (2.50%) Senior Architect
7 1 (2.50%) Senior Data Architect
Libraries, Frameworks & Software Standards
1 6 (15.00%) Kafka
2 5 (12.50%) NumPy
2 5 (12.50%) scikit-learn
3 4 (10.00%) TensorFlow
4 3 (7.50%) FIX Protocol
4 3 (7.50%) JMS
4 3 (7.50%) Middleware
4 3 (7.50%) Natural Language Processing
4 3 (7.50%) PySpark
4 3 (7.50%) Rendezvous
4 3 (7.50%) SciPy
4 3 (7.50%) Theano
4 3 (7.50%) Ultra Messaging
4 3 (7.50%) Velocity
4 3 (7.50%) ZeroMQ
5 2 (5.00%) Pandas
5 2 (5.00%) React
6 1 (2.50%) Flask
6 1 (2.50%) Keras
6 1 (2.50%) Protocol Buffers
Miscellaneous
1 6 (15.00%) Algorithms
1 6 (15.00%) Data Structures
2 4 (10.00%) Analytical Skills
3 3 (7.50%) Derivative
3 3 (7.50%) Equities
3 3 (7.50%) Self-Motivation
4 2 (5.00%) Private Cloud
4 2 (5.00%) User Experience
5 1 (2.50%) Fixed Income
5 1 (2.50%) Market Risk
5 1 (2.50%) Marketing Technology
5 1 (2.50%) Reference Data
Operating Systems
1 7 (17.50%) Linux
2 2 (5.00%) Red Hat Enterprise Linux
2 2 (5.00%) Unix
2 2 (5.00%) Windows
Processes & Methodologies
1 18 (45.00%) Analytics
2 13 (32.50%) Data Science
3 11 (27.50%) Machine Learning
4 7 (17.50%) Artificial Intelligence
5 6 (15.00%) Agile Software Development
6 5 (12.50%) Data Modelling
6 5 (12.50%) MapReduce
6 5 (12.50%) Problem-Solving
6 5 (12.50%) Task Automation
7 4 (10.00%) Business Intelligence
7 4 (10.00%) Computer Science
7 4 (10.00%) Data Management
7 4 (10.00%) Information Security
7 4 (10.00%) Stakeholder Management
8 3 (7.50%) Applied Mathematics
8 3 (7.50%) ETL
8 3 (7.50%) Mathematics
8 3 (7.50%) OO
8 3 (7.50%) OOD
8 3 (7.50%) Software Engineering
Programming Languages
1 17 (42.50%) Python
2 11 (27.50%) Java
3 10 (25.00%) SQL
4 7 (17.50%) Scala
5 4 (10.00%) C
6 2 (5.00%) Bash Shell
6 2 (5.00%) JavaScript
6 2 (5.00%) Shell Script
7 1 (2.50%) Perl
7 1 (2.50%) R
7 1 (2.50%) Ruby
Qualifications
1 2 (5.00%) MCSA
1 2 (5.00%) Microsoft Certification
2 1 (2.50%) Degree
2 1 (2.50%) Security Cleared
Quality Assurance & Compliance
1 4 (10.00%) Data Quality
2 1 (2.50%) QA
System Software
1 5 (12.50%) HDFS
2 3 (7.50%) Apache Flume
3 2 (5.00%) Docker
Systems Management
1 5 (12.50%) Kubernetes
2 3 (7.50%) Ambari
2 3 (7.50%) Hadoop YARN
2 3 (7.50%) Oozie
3 1 (2.50%) Opscode Chef
Vendors
1 5 (12.50%) Tableau
2 4 (10.00%) Google
2 4 (10.00%) IBM
2 4 (10.00%) Oracle
3 3 (7.50%) Cloudera
3 3 (7.50%) Informatica
3 3 (7.50%) Microsoft
3 3 (7.50%) Sybase
3 3 (7.50%) TIBCO
4 2 (5.00%) Red Hat
5 1 (2.50%) Adobe
5 1 (2.50%) Alteryx
5 1 (2.50%) Splunk
5 1 (2.50%) Teradata
5 1 (2.50%) Vertica