Period
to 18 November 2018

The median contractor rate for a Big Data Engineer was £575 per day in advertised job vacancies in the City of London during the 6 months to 18 November 2018.

The first table below provides contractor rate benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Big Data Engineer
Central London > City of London
6 months to
18 Nov 2018
Same period 2017 Same period 2016
Rank 204 236 227
Rank change year-on-year +32 -9 +40
Contract jobs requiring a Big Data Engineer 48 19 25
As % of all contract IT jobs advertised in the City of London 0.69% 0.27% 0.37%
As % of the Job Titles category 0.73% 0.28% 0.39%
Number of daily rates quoted 42 15 16
City of London median daily rate £575 £533 £575
Median daily rate % change year-on-year +7.98% -7.39% -11.54%
10th Percentile £513 £444 £438
90th Percentile £675 £635 £650
Central London median daily rate £575 £533 £575
% change year-on-year +7.98% -7.39% -4.17%
Number of hourly rates quoted 0 0 0
City of London median hourly rate - - -
Central London median hourly rate - - -

The following table is for comparison with the above and includes summary statistics for all contract IT job vacancies advertised in the City of London. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of contract jobs in our overall sample.

All Contract IT Job Vacancies
City of London
Contract vacancies in the City of London with a recognized job title 6,614 6,709 6,421
% of contract IT jobs with a recognized job title 95.30% 94.72% 95.69%
Number of daily rates quoted 5,248 5,305 5,111
City of London median daily rate £500 £500 £475
Median daily rate % change year-on-year - +5.26% +5.56%
10th Percentile £305 £313 £313
90th Percentile £675 £675 £663
Central London median daily rate £500 £500 £475
% change year-on-year - +5.26% +5.56%
Number of hourly rates quoted 112 94 79
City of London median hourly rate £18.70 £17.00 £17.25
Median hourly rate % change year-on-year +10.00% -1.45% -4.17%
10th Percentile £13.00 £11.25 £11.45
90th Percentile £27.30 £29.29 £47.83
Central London median hourly rate £18.70 £18.05 £17.08
% change year-on-year +3.60% +5.66% -5.10%

Big Data Engineer
Job Vacancy Trend in the City of London

Job postings that featured Big Data Engineer in the job title as a percentage of all IT jobs advertised in the City of London.

Job vacancy trend for Big Data Engineer in the City of London

Big Data Engineer
Contractor Daily Rate Trend in the City of London

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Big Data Engineer in the City of London.

Contractor daily rate trend for Big Data Engineer in the City of London

Big Data Engineer
Contractor Daily Rate Histogram in the City of London

The daily rate distribution of IT jobs citing Big Data Engineer in the City of London over the 6 months to 18 November 2018.

Contractor daily rate histogram for Big Data Engineer in the City of London

Big Data Engineer
Contract Job Locations in the City of London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Big Data Engineer within the City of London region over the 6 months to 18 November 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London Bridge - 5 £525 -
Big Data Engineer
Central London

Big Data Engineer Skill Set
Top 30 Co-occurring IT Skills in the City of London

For the 6 months to 18 November 2018, Big Data Engineer contract jobs required the following IT skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the City of London region featuring Big Data Engineer in the job title.

1 48 (100.00%) Big Data
2 43 (89.58%) Hadoop
3 36 (75.00%) Kafka
4 30 (62.50%) Apache Hive
5 26 (54.17%) Java
6 24 (50.00%) Finance
6 24 (50.00%) Apache Spark
7 17 (35.42%) Python
8 16 (33.33%) Maven
8 16 (33.33%) Elasticsearch
8 16 (33.33%) Java 8
9 14 (29.17%) AngularJS
10 11 (22.92%) Agile Software Development
10 11 (22.92%) SQL
11 10 (20.83%) Jenkins
11 10 (20.83%) Data Science
12 9 (18.75%) Oozie
12 9 (18.75%) Amazon AWS
12 9 (18.75%) Machine Learning
13 8 (16.67%) MapReduce
13 8 (16.67%) Scala
13 8 (16.67%) ETL
13 8 (16.67%) Git (software)
14 7 (14.58%) Data Modelling
14 7 (14.58%) Data Warehouse
14 7 (14.58%) Analytics
14 7 (14.58%) HBase
14 7 (14.58%) DevOps
15 6 (12.50%) HDFS
15 6 (12.50%) Sqoop

Big Data Engineer Skill Set
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 24 (50.00%) Apache Spark
2 16 (33.33%) Elasticsearch
3 4 (8.33%) Apache
4 2 (4.17%) Apache Pig
4 2 (4.17%) Cloud Foundry
4 2 (4.17%) Confluence
5 1 (2.08%) OpenStack
Cloud Services
1 9 (18.75%) Amazon AWS
2 5 (10.42%) Google Cloud Platform
2 5 (10.42%) IBM Cloud
3 4 (8.33%) GitHub
4 3 (6.25%) Amazon EMR
4 3 (6.25%) Amazon S3
4 3 (6.25%) Microsoft Azure
5 2 (4.17%) Amazon EC2
6 1 (2.08%) Amazon Kinesis
6 1 (2.08%) AWS Lambda
6 1 (2.08%) Serverless
Database & Business Intelligence
1 48 (100.00%) Big Data
2 43 (89.58%) Hadoop
3 30 (62.50%) Apache Hive
4 7 (14.58%) Data Warehouse
4 7 (14.58%) HBase
5 6 (12.50%) Sqoop
6 5 (10.42%) Apache Cassandra
6 5 (10.42%) NoSQL
7 4 (8.33%) Enterprise Data Warehouse
7 4 (8.33%) MongoDB
7 4 (8.33%) OLAP
7 4 (8.33%) OLAP Cube
7 4 (8.33%) PostgreSQL
7 4 (8.33%) SQL Server
7 4 (8.33%) Star Schema
8 3 (6.25%) Data Lake
8 3 (6.25%) Relational Database
9 2 (4.17%) Impala
9 2 (4.17%) Presto
10 1 (2.08%) Spark SQL
Development Applications
1 16 (33.33%) Maven
2 10 (20.83%) Jenkins
3 8 (16.67%) Git (software)
4 4 (8.33%) JIRA
5 3 (6.25%) Visual Studio Team System
6 2 (4.17%) Sonatype Nexus
6 2 (4.17%) Subversion
6 2 (4.17%) Visual Studio
7 1 (2.08%) Cucumber
7 1 (2.08%) Eclipse
7 1 (2.08%) Gerrit
7 1 (2.08%) JBehave
7 1 (2.08%) Selenium
7 1 (2.08%) Vagrant
General
1 24 (50.00%) Finance
2 4 (8.33%) Banking
3 2 (4.17%) Front Office
4 1 (2.08%) Advertising
4 1 (2.08%) Corporate Banking
4 1 (2.08%) Marketing
4 1 (2.08%) Publishing
4 1 (2.08%) Retail
Libraries, Frameworks & Software Standards
1 36 (75.00%) Kafka
2 14 (29.17%) AngularJS
3 3 (6.25%) PySpark
4 2 (4.17%) RESTful
4 2 (4.17%) Spark Streaming
4 2 (4.17%) Spring
5 1 (2.08%) Apache Commons
5 1 (2.08%) CSS
5 1 (2.08%) CSS3
5 1 (2.08%) Dask
5 1 (2.08%) Elastic Stack
5 1 (2.08%) Flask
5 1 (2.08%) Gherkin
5 1 (2.08%) Guava
5 1 (2.08%) HTML
5 1 (2.08%) HTML5
5 1 (2.08%) NumPy
5 1 (2.08%) OpenCV
5 1 (2.08%) RabbitMQ
5 1 (2.08%) TensorFlow
Miscellaneous
1 5 (10.42%) Algorithms
2 4 (8.33%) Greenfield Project
3 3 (6.25%) Fintech
4 2 (4.17%) iPad
5 1 (2.08%) Analytical Skills
5 1 (2.08%) Computer Science
5 1 (2.08%) NHS
Operating Systems
1 3 (6.25%) Linux
1 3 (6.25%) Unix
2 1 (2.08%) CentOS
2 1 (2.08%) Red Hat Enterprise Linux
Processes & Methodologies
1 11 (22.92%) Agile Software Development
2 10 (20.83%) Data Science
3 9 (18.75%) Machine Learning
4 8 (16.67%) ETL
4 8 (16.67%) MapReduce
5 7 (14.58%) Analytics
5 7 (14.58%) Data Modelling
5 7 (14.58%) DevOps
6 6 (12.50%) Open Source
6 6 (12.50%) Scrum
7 4 (8.33%) Data Engineering
7 4 (8.33%) SQL Optimisation
8 3 (6.25%) Kanban
9 2 (4.17%) Artificial Intelligence
9 2 (4.17%) Configuration Management
9 2 (4.17%) Continuous Integration
9 2 (4.17%) ETL Development
9 2 (4.17%) SDLC
9 2 (4.17%) TDD
9 2 (4.17%) Visualisation
Programming Languages
1 26 (54.17%) Java
2 17 (35.42%) Python
3 16 (33.33%) Java 8
4 11 (22.92%) SQL
5 8 (16.67%) Scala
6 4 (8.33%) JavaScript
7 3 (6.25%) Groovy
8 2 (4.17%) C#
8 2 (4.17%) R
9 1 (2.08%) AWK
9 1 (2.08%) Bash Shell
9 1 (2.08%) Ruby
9 1 (2.08%) sed
9 1 (2.08%) T-SQL
Qualifications
1 2 (4.17%) Degree
2 1 (2.08%) Master's Degree
Quality Assurance & Compliance
1 5 (10.42%) Data Quality
System Software
1 6 (12.50%) HDFS
2 3 (6.25%) Apache Flume
2 3 (6.25%) Docker
Systems Management
1 9 (18.75%) Oozie
2 5 (10.42%) Hadoop YARN
3 4 (8.33%) Ansible
4 3 (6.25%) Ambari
5 2 (4.17%) Kubernetes
5 2 (4.17%) Terraform
6 1 (2.08%) Kibana
6 1 (2.08%) logstash
Vendors
1 5 (10.42%) Google
1 5 (10.42%) Hortonworks
1 5 (10.42%) IBM
2 4 (8.33%) Tableau
3 3 (6.25%) Oracle
4 2 (4.17%) Microsoft
4 2 (4.17%) Talend
5 1 (2.08%) Informatica