Period
to 16 October 2018

The following table provides summary statistics for contract job vacancies advertised in the City of London with a requirement for Data Engineering skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Data Engineering over the 6 months to 16 October 2018 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Data Engineering
Central London > City of London
6 months to
16 Oct 2018
Same period 2017 Same period 2016
Rank 242 216 246
Rank change year-on-year -26 +30 +35
Contract jobs citing Data Engineering 29 29 3
As % of all contract IT jobs advertised in the City of London 0.41% 0.42% 0.045%
As % of the Processes & Methodologies category 0.48% 0.47% 0.051%
Number of daily rates quoted 22 29 3
City of London median daily rate £536 £550 £400
Median daily rate % change year-on-year -2.50% +37.50% -
10th Percentile £416 £425 £348
90th Percentile £653 £675 £574
Central London median daily rate £549 £550 £550
% change year-on-year -0.23% - -
Number of hourly rates quoted 0 0 0
City of London median hourly rate - - -
Central London median hourly rate - - -

Data Engineering is in the Processes and Methodologies category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in the City of London with a requirement for process or methodology skills.

All Process and Methodology Skills
City of London
Contract vacancies with a requirement for process or methodology skills 6,063 6,109 5,834
As % of all contract IT jobs advertised in the City of London 86.49% 89.16% 87.93%
Number of daily rates quoted 4,824 4,802 4,685
City of London median daily rate £505 £500 £475
Median daily rate % change year-on-year +1.00% +5.26% +2.70%
10th Percentile £325 £325 £313
90th Percentile £675 £675 £663
Central London median daily rate £500 £500 £475
% change year-on-year - +5.26% +5.56%
Number of hourly rates quoted 100 57 51
City of London median hourly rate £18.70 £18.70 £19.00
Median hourly rate % change year-on-year - -1.58% -5.00%
10th Percentile £13.00 £13.31 £11.25
90th Percentile £33.08 £31.05 £55.00
Central London median hourly rate £18.70 £18.70 £19.00
% change year-on-year - -1.58% -5.00%

Data Engineering
Job Vacancy Trend in the City of London

Job postings citing Data Engineering as a percentage of all IT jobs advertised in the City of London.

Job vacancy trend for Data Engineering in the City of London

Data Engineering
Contractor Daily Rate Trend in the City of London

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Data Engineering in the City of London.

Contractor daily rate trend for Data Engineering in the City of London

Data Engineering
Contractor Daily Rate Histogram in the City of London

The daily rate distribution of IT jobs citing Data Engineering in the City of London over the 6 months to 16 October 2018.

Contractor daily rate histogram for Data Engineering in the City of London

Data Engineering
Contract Job Locations in the City of London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Data Engineering within the City of London region over the 6 months to 16 October 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London Bridge - 2 £575 -
Data Engineering
Central London

For the 6 months to 16 October 2018, IT contractor jobs citing Data Engineering also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the City of London region with a requirement for Data Engineering.

1 15 (51.72%) SQL
2 13 (44.83%) Hadoop
2 13 (44.83%) Apache Spark
3 12 (41.38%) Python
4 11 (37.93%) Java
4 11 (37.93%) Big Data
5 9 (31.03%) Amazon AWS
5 9 (31.03%) SQL Server
6 8 (27.59%) Analytics
6 8 (27.59%) HDFS
6 8 (27.59%) Data Science
7 7 (24.14%) Performance Tuning
7 7 (24.14%) Machine Learning
7 7 (24.14%) ETL
7 7 (24.14%) Finance
8 6 (20.69%) Agile Software Development
8 6 (20.69%) Artificial Intelligence
8 6 (20.69%) Kafka
8 6 (20.69%) Sqoop
8 6 (20.69%) Jenkins
8 6 (20.69%) Microsoft Azure
8 6 (20.69%) Apache Hive
8 6 (20.69%) Oozie
9 5 (17.24%) Relational Database
9 5 (17.24%) MapReduce
9 5 (17.24%) Scala
9 5 (17.24%) Git (software)
9 5 (17.24%) Stakeholder Management
9 5 (17.24%) Informatica
10 4 (13.79%) OLTP

Data Engineering
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 13 (44.83%) Apache Spark
2 1 (3.45%) Apache
2 1 (3.45%) Apache Airflow
2 1 (3.45%) SAS
Applications
1 1 (3.45%) Chatbot
1 1 (3.45%) Microsoft Excel
1 1 (3.45%) Spreadsheet
Business Applications
1 1 (3.45%) Unica
Cloud Services
1 9 (31.03%) Amazon AWS
2 6 (20.69%) Microsoft Azure
3 4 (13.79%) GitHub
4 3 (10.34%) Amazon S3
4 3 (10.34%) IBM Cloud
5 2 (6.90%) Amazon EMR
5 2 (6.90%) Amazon Kinesis
5 2 (6.90%) AWS Lambda
5 2 (6.90%) Cloud Computing
5 2 (6.90%) Serverless
6 1 (3.45%) Amazon EC2
6 1 (3.45%) AWS CloudFormation
6 1 (3.45%) Google Cloud Platform
Communications & Networking
1 1 (3.45%) Firewall
1 1 (3.45%) LAN
Database & Business Intelligence
1 13 (44.83%) Hadoop
2 11 (37.93%) Big Data
3 9 (31.03%) SQL Server
4 6 (20.69%) Apache Hive
4 6 (20.69%) Sqoop
5 5 (17.24%) Relational Database
6 4 (13.79%) Data Warehouse
6 4 (13.79%) OLTP
6 4 (13.79%) PostgreSQL
6 4 (13.79%) SQL Server 2016
6 4 (13.79%) SQL Server Integration Services
7 3 (10.34%) Amazon Redshift
7 3 (10.34%) KDB
8 2 (6.90%) Amazon Athena
8 2 (6.90%) Data Mining
8 2 (6.90%) Kdb+
8 2 (6.90%) NoSQL
8 2 (6.90%) SQL Server 2008
8 2 (6.90%) SQL Server 2012
8 2 (6.90%) SQL Server 2014
Development Applications
1 6 (20.69%) Jenkins
2 5 (17.24%) Git (software)
3 2 (6.90%) Maven
3 2 (6.90%) Robot Framework
4 1 (3.45%) JIRA
4 1 (3.45%) Selenium
4 1 (3.45%) Team Foundation Server
General
1 7 (24.14%) Finance
2 3 (10.34%) Banking
2 3 (10.34%) Marketing
3 1 (3.45%) Games
3 1 (3.45%) Retail
Job Titles
1 10 (34.48%) Data Engineer
2 7 (24.14%) Developer
3 6 (20.69%) Big Data Engineer
4 3 (10.34%) Consultant
4 3 (10.34%) Data Consultant
4 3 (10.34%) Java Developer
4 3 (10.34%) KDB Developer
4 3 (10.34%) Senior Data Engineer
5 2 (6.90%) Administrator
5 2 (6.90%) Big Data Developer
5 2 (6.90%) Database Engineer
5 2 (6.90%) Hadoop Administrator
5 2 (6.90%) Scrum Master
5 2 (6.90%) Senior Database Engineer
6 1 (3.45%) Java Scala Developer
6 1 (3.45%) Machine Learning Developer
6 1 (3.45%) Machine Learning Engineer
6 1 (3.45%) Migration Analyst
6 1 (3.45%) Scala Developer
6 1 (3.45%) SQL Analyst
Libraries, Frameworks & Software Standards
1 6 (20.69%) Kafka
2 4 (13.79%) JMS
3 3 (10.34%) Natural Language Processing
3 3 (10.34%) NumPy
3 3 (10.34%) PySpark
3 3 (10.34%) scikit-learn
3 3 (10.34%) TensorFlow
4 2 (6.90%) FIX Protocol
4 2 (6.90%) Middleware
4 2 (6.90%) Rendezvous
4 2 (6.90%) SciPy
4 2 (6.90%) Theano
4 2 (6.90%) Ultra Messaging
4 2 (6.90%) Velocity
4 2 (6.90%) ZeroMQ
5 1 (3.45%) .NET
5 1 (3.45%) CSS
5 1 (3.45%) HTML
5 1 (3.45%) JSON
5 1 (3.45%) Ruby on Rails
Miscellaneous
1 3 (10.34%) Analytical Skills
2 2 (6.90%) Algorithms
2 2 (6.90%) Derivative
2 2 (6.90%) Equities
2 2 (6.90%) Private Cloud
2 2 (6.90%) Self-Motivation
3 1 (3.45%) Computer Science
3 1 (3.45%) Fixed Income
3 1 (3.45%) Marketing Technology
3 1 (3.45%) User Experience
Operating Systems
1 3 (10.34%) Linux
2 2 (6.90%) Red Hat Enterprise Linux
3 1 (3.45%) Unix
Processes & Methodologies
1 8 (27.59%) Analytics
1 8 (27.59%) Data Science
2 7 (24.14%) ETL
2 7 (24.14%) Machine Learning
2 7 (24.14%) Performance Tuning
3 6 (20.69%) Agile Software Development
3 6 (20.69%) Artificial Intelligence
4 5 (17.24%) MapReduce
4 5 (17.24%) Stakeholder Management
5 4 (13.79%) Data Modelling
5 4 (13.79%) Database Tuning
5 4 (13.79%) Mathematics
6 3 (10.34%) Applied Mathematics
6 3 (10.34%) DevOps
6 3 (10.34%) ETL Development
6 3 (10.34%) Mentoring
6 3 (10.34%) Open Source
6 3 (10.34%) Scrum
6 3 (10.34%) SDLC
6 3 (10.34%) TDD
Programming Languages
1 15 (51.72%) SQL
2 12 (41.38%) Python
3 11 (37.93%) Java
4 5 (17.24%) Scala
5 4 (13.79%) T-SQL
6 2 (6.90%) Bash Shell
6 2 (6.90%) C
7 1 (3.45%) C#
7 1 (3.45%) Go
7 1 (3.45%) JavaScript
7 1 (3.45%) Perl
7 1 (3.45%) R
7 1 (3.45%) Ruby
7 1 (3.45%) VB
Qualifications
1 2 (6.90%) Degree
2 1 (3.45%) Computer Science Degree
2 1 (3.45%) Numerate Degree
Quality Assurance & Compliance
1 2 (6.90%) Data Quality
1 2 (6.90%) QA
System Software
1 8 (27.59%) HDFS
2 3 (10.34%) Apache Flume
3 1 (3.45%) Docker
Systems Management
1 6 (20.69%) Oozie
2 3 (10.34%) Ambari
2 3 (10.34%) Hadoop YARN
2 3 (10.34%) Kubernetes
Vendors
1 5 (17.24%) Informatica
2 4 (13.79%) IBM
3 3 (10.34%) Tableau
4 2 (6.90%) Google
4 2 (6.90%) Microsoft
4 2 (6.90%) Oracle
4 2 (6.90%) Red Hat
4 2 (6.90%) Sybase
4 2 (6.90%) TIBCO
5 1 (3.45%) Adobe
5 1 (3.45%) Cloudera
5 1 (3.45%) Teradata
5 1 (3.45%) Vertica