Period
to 20 February 2019

The median contractor rate for a Big Data Engineer was £525 per day in advertised job vacancies in the City of London during the 6 months to 20 February 2019.

The first table below provides contractor rate benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Big Data Engineer
Central London > City of London
6 months to
20 Feb 2019
Same period 2018 Same period 2017
Rank 194 222 199
Rank change year-on-year +28 -23 +20
Contract jobs requiring a Big Data Engineer 53 35 36
As % of all contract IT jobs advertised in the City of London 0.78% 0.49% 0.59%
As % of the Job Titles category 0.82% 0.52% 0.61%
Number of daily rates quoted 44 34 29
City of London median daily rate £525 £550 £575
Median daily rate % change year-on-year -4.55% -4.35% +4.55%
10th Percentile £513 £465 £425
90th Percentile £671 £638 £650
Central London median daily rate £525 £563 £575
% change year-on-year -6.67% -2.17% +4.55%
Number of hourly rates quoted 0 0 0
City of London median hourly rate - - -
Central London median hourly rate - - -

The following table is for comparison with the above and includes summary statistics for all contract IT job vacancies advertised in the City of London. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of contract jobs in our overall sample.

All Contract IT Job Vacancies
City of London
Contract vacancies in the City of London with a recognized job title 6,449 6,707 5,855
% of contract IT jobs with a recognized job title 94.59% 94.48% 95.62%
Number of daily rates quoted 5,047 5,329 4,641
City of London median daily rate £500 £500 £475
Median daily rate % change year-on-year - +5.26% -
10th Percentile £305 £313 £313
90th Percentile £675 £688 £663
Central London median daily rate £500 £500 £475
% change year-on-year - +5.26% +5.56%
Number of hourly rates quoted 70 98 72
City of London median hourly rate £18.70 £16.25 £18.35
Median hourly rate % change year-on-year +15.08% -11.44% -8.22%
10th Percentile £12.25 £11.25 £12.53
90th Percentile £35.00 £31.68 £54.51
Central London median hourly rate £18.70 £18.00 £17.50
% change year-on-year +3.89% +2.86% -12.46%

Big Data Engineer
Job Vacancy Trend in the City of London

Job postings that featured Big Data Engineer in the job title as a percentage of all IT jobs advertised in the City of London.

Job vacancy trend for Big Data Engineer in the City of London

Big Data Engineer
Contractor Daily Rate Trend in the City of London

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Big Data Engineer in the City of London.

Contractor daily rate trend for Big Data Engineer in the City of London

Big Data Engineer
Contractor Daily Rate Histogram in the City of London

The daily rate distribution of IT jobs citing Big Data Engineer in the City of London over the 6 months to 20 February 2019.

Contractor daily rate histogram for Big Data Engineer in the City of London

Big Data Engineer
Contract Job Locations in the City of London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Big Data Engineer within the City of London region over the 6 months to 20 February 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London Bridge - 22 £525 -
Big Data Engineer
Central London

Big Data Engineer Skill Set
Top 30 Co-occurring IT Skills in the City of London

For the 6 months to 20 February 2019, Big Data Engineer contract jobs required the following IT skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the City of London region featuring Big Data Engineer in the job title.

1 53 (100.00%) Big Data
2 39 (73.58%) Apache Spark
3 35 (66.04%) Finance
4 31 (58.49%) Python
5 29 (54.72%) Hadoop
5 29 (54.72%) Amazon AWS
5 29 (54.72%) Kafka
6 28 (52.83%) Data Science
7 24 (45.28%) Google Cloud Platform
7 24 (45.28%) Microsoft Azure
8 23 (43.40%) Google
9 22 (41.51%) Apache
10 19 (35.85%) PySpark
10 19 (35.85%) Data Engineering
11 18 (33.96%) Apache Airflow
11 18 (33.96%) Data Cleansing
11 18 (33.96%) Mentoring
11 18 (33.96%) Coaching
12 15 (28.30%) Apache Hive
13 14 (26.42%) Agile Software Development
14 13 (24.53%) Java
15 12 (22.64%) Jenkins
16 11 (20.75%) Git (software)
17 10 (18.87%) SQL
18 9 (16.98%) Docker
18 9 (16.98%) Machine Learning
19 8 (15.09%) MapReduce
19 8 (15.09%) DevOps
20 7 (13.21%) Elasticsearch
20 7 (13.21%) Apache Cassandra

Big Data Engineer Skill Set
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 39 (73.58%) Apache Spark
2 22 (41.51%) Apache
3 18 (33.96%) Apache Airflow
4 7 (13.21%) Elasticsearch
5 3 (5.66%) Cloud Foundry
5 3 (5.66%) Confluence
6 2 (3.77%) Apache Pig
Applications
1 2 (3.77%) Microsoft Excel
Business Applications
1 1 (1.89%) Distributed Ledger
Cloud Services
1 29 (54.72%) Amazon AWS
2 24 (45.28%) Google Cloud Platform
2 24 (45.28%) Microsoft Azure
3 4 (7.55%) IBM Cloud
4 2 (3.77%) GitHub
5 1 (1.89%) Amazon EC2
5 1 (1.89%) Amazon EMR
5 1 (1.89%) Amazon Kinesis
5 1 (1.89%) Amazon S3
5 1 (1.89%) AWS Lambda
5 1 (1.89%) Serverless
Database & Business Intelligence
1 53 (100.00%) Big Data
2 29 (54.72%) Hadoop
3 15 (28.30%) Apache Hive
4 7 (13.21%) Apache Cassandra
4 7 (13.21%) HBase
5 6 (11.32%) MongoDB
6 5 (9.43%) Data Warehouse
6 5 (9.43%) NoSQL
7 4 (7.55%) Enterprise Data Warehouse
7 4 (7.55%) OLAP
7 4 (7.55%) OLAP Cube
7 4 (7.55%) SQL Server
7 4 (7.55%) Star Schema
8 3 (5.66%) Data Lake
8 3 (5.66%) Sqoop
9 2 (3.77%) Power BI
9 2 (3.77%) Presto
9 2 (3.77%) Spotfire
10 1 (1.89%) PostgreSQL
10 1 (1.89%) Spark SQL
Development Applications
1 12 (22.64%) Jenkins
2 11 (20.75%) Git (software)
3 6 (11.32%) Maven
4 5 (9.43%) JIRA
5 4 (7.55%) Visual Studio
5 4 (7.55%) Visual Studio Team System
6 3 (5.66%) Sonatype Nexus
7 2 (3.77%) Subversion
8 1 (1.89%) Cucumber
8 1 (1.89%) Eclipse
8 1 (1.89%) Gerrit
8 1 (1.89%) JBehave
8 1 (1.89%) Selenium
General
1 35 (66.04%) Finance
2 7 (13.21%) Banking
3 4 (7.55%) Front Office
4 1 (1.89%) Corporate Banking
4 1 (1.89%) Publishing
4 1 (1.89%) Retail
Libraries, Frameworks & Software Standards
1 29 (54.72%) Kafka
2 19 (35.85%) PySpark
3 5 (9.43%) AngularJS
4 2 (3.77%) RESTful
4 2 (3.77%) Spring
5 1 (1.89%) Apache Commons
5 1 (1.89%) Elastic Stack
5 1 (1.89%) Flask
5 1 (1.89%) Gherkin
5 1 (1.89%) Guava
5 1 (1.89%) NumPy
5 1 (1.89%) Spark Streaming
Miscellaneous
1 4 (7.55%) Algorithms
2 3 (5.66%) Analytical Skills
2 3 (5.66%) iPad
3 2 (3.77%) Fintech
3 2 (3.77%) Greenfield Project
4 1 (1.89%) NHS
Operating Systems
1 4 (7.55%) Unix
2 2 (3.77%) Linux
Processes & Methodologies
1 28 (52.83%) Data Science
2 19 (35.85%) Data Engineering
3 18 (33.96%) Coaching
3 18 (33.96%) Data Cleansing
3 18 (33.96%) Mentoring
4 14 (26.42%) Agile Software Development
5 9 (16.98%) Machine Learning
6 8 (15.09%) DevOps
6 8 (15.09%) MapReduce
7 7 (13.21%) Scrum
8 6 (11.32%) Analytics
9 5 (9.43%) Data Modelling
10 4 (7.55%) Open Source
10 4 (7.55%) Software Deployment
10 4 (7.55%) SQL Optimisation
11 3 (5.66%) Data Transformation
11 3 (5.66%) ETL
11 3 (5.66%) Kanban
11 3 (5.66%) SDLC
11 3 (5.66%) Visualisation
Programming Languages
1 31 (58.49%) Python
2 13 (24.53%) Java
3 10 (18.87%) SQL
4 6 (11.32%) Java 8
5 5 (9.43%) Scala
6 4 (7.55%) Groovy
7 3 (5.66%) JavaScript
8 2 (3.77%) Bash Shell
8 2 (3.77%) C#
8 2 (3.77%) R
8 2 (3.77%) VBA
9 1 (1.89%) AWK
9 1 (1.89%) Pig Latin
9 1 (1.89%) Ruby
9 1 (1.89%) sed
Qualifications
1 2 (3.77%) Degree
2 1 (1.89%) Master's Degree
Quality Assurance & Compliance
1 5 (9.43%) Data Quality
System Software
1 9 (16.98%) Docker
2 3 (5.66%) HDFS
3 1 (1.89%) Apache Flume
Systems Management
1 6 (11.32%) Oozie
2 5 (9.43%) Ansible
2 5 (9.43%) Kubernetes
3 4 (7.55%) Terraform
4 3 (5.66%) Hadoop YARN
5 1 (1.89%) Ambari
5 1 (1.89%) Kibana
5 1 (1.89%) logstash
Vendors
1 23 (43.40%) Google
2 5 (9.43%) Hortonworks
2 5 (9.43%) Oracle
3 4 (7.55%) IBM
3 4 (7.55%) Microsoft
3 4 (7.55%) Talend
4 2 (3.77%) Tableau