Period
to 16 July 2018

The median contractor rate for a Big Data Engineer was £563 per day in advertised job vacancies in London during the 6 months to 16 July 2018.

The first table below provides contractor rate benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Big Data Engineer
England > London
6 months to
16 Jul 2018
Same period 2017 Same period 2016
Rank 355 383 523
Rank change year-on-year +28 +140 +48
Contract jobs requiring a Big Data Engineer 299 214 96
As % of all contract IT jobs advertised in London 0.65% 0.51% 0.21%
As % of the Job Titles category 0.68% 0.54% 0.22%
Number of daily rates quoted 222 164 83
London median daily rate £563 £550 £600
Median daily rate % change year-on-year +2.27% -8.33% +9.09%
10th Percentile £450 £416 £478
90th Percentile £686 £675 £845
England median daily rate £550 £550 £600
% change year-on-year - -8.33% +9.09%
Number of hourly rates quoted 1 0 0
London median hourly rate £15.50 - -
England median hourly rate £15.50 - -

The following table is for comparison with the above and includes summary statistics for all contract IT job vacancies advertised in London. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of contract jobs in our overall sample.

All Contract IT Job Vacancies
London
Contract vacancies in London with a recognized job title 43,896 39,747 43,277
% of contract IT jobs with a recognized job title 94.74% 95.11% 95.17%
Number of daily rates quoted 31,768 28,280 31,143
London median daily rate £500 £475 £450
Median daily rate % change year-on-year +5.26% +5.56% +2.86%
10th Percentile £313 £304 £285
90th Percentile £675 £650 £638
England median daily rate £450 £438 £425
% change year-on-year +2.86% +2.94% +6.25%
Number of hourly rates quoted 650 514 509
London median hourly rate £21.50 £21.25 £19.00
Median hourly rate % change year-on-year +1.18% +11.84% -4.26%
10th Percentile £13.00 £12.50 £10.60
90th Percentile £50.57 £50.00 £46.75
England median hourly rate £18.70 £18.50 £19.00
% change year-on-year +1.08% -2.63% +8.57%

Big Data Engineer
Job Vacancy Trend in London

Job postings that featured Big Data Engineer in the job title as a percentage of all IT jobs advertised in London.

Job vacancy trend for Big Data Engineer in London

Big Data Engineer
Contractor Daily Rate Trend in London

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Big Data Engineer in London.

Contractor daily rate trend for Big Data Engineer in London

Big Data Engineer
Contractor Daily Rate Histogram in London

The daily rate distribution of IT jobs citing Big Data Engineer in London over the 6 months to 16 July 2018.

Contractor daily rate histogram for Big Data Engineer in London

Big Data Engineer
Contract Job Locations in London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Big Data Engineer within the London region over the 6 months to 16 July 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +5 70 £575 +4.55% 14
East London - 23 £550 - 2
South London -49 15 £550 +0.92% 2
West London -27 3 £565 +32.94% 2
Big Data Engineer
England

Big Data Engineer Skill Set
Top 30 Co-occurring IT Skills in London

For the 6 months to 16 July 2018, Big Data Engineer contract jobs required the following IT skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the London region featuring Big Data Engineer in the job title.

1 299 (100.00%) Big Data
2 224 (74.92%) Hadoop
3 192 (64.21%) Apache Spark
4 153 (51.17%) Python
5 139 (46.49%) Java
6 134 (44.82%) Apache Hive
7 128 (42.81%) SQL
7 128 (42.81%) Kafka
8 121 (40.47%) Scala
9 113 (37.79%) Finance
10 104 (34.78%) Analytics
11 100 (33.44%) Agile Software Development
12 89 (29.77%) ETL
13 87 (29.10%) Amazon AWS
14 81 (27.09%) Data Science
15 74 (24.75%) Linux
16 73 (24.41%) NoSQL
17 58 (19.40%) DevOps
18 54 (18.06%) SQL Server
19 53 (17.73%) HBase
20 51 (17.06%) Scrum
21 50 (16.72%) Data Warehouse
21 50 (16.72%) HDFS
22 48 (16.05%) Relational Database
22 48 (16.05%) Machine Learning
23 47 (15.72%) Microsoft Azure
24 45 (15.05%) Data Modelling
24 45 (15.05%) Software Engineering
25 44 (14.72%) Data Engineering
26 42 (14.05%) Open Source

Big Data Engineer Skill Set
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 192 (64.21%) Apache Spark
2 41 (13.71%) Apache
3 40 (13.38%) Elasticsearch
4 16 (5.35%) Apache Pig
5 12 (4.01%) nginx
5 12 (4.01%) Tomcat
6 7 (2.34%) Apache Solr
7 6 (2.01%) SAS
8 2 (0.67%) Apache Airflow
8 2 (0.67%) Cloud Foundry
8 2 (0.67%) Confluence
8 2 (0.67%) Jupyter
9 1 (0.33%) Luigi
9 1 (0.33%) OpenStack
9 1 (0.33%) SharePoint
Business Applications
1 7 (2.34%) SAP Oil and Gas
2 1 (0.33%) Markit EDM
Cloud Services
1 87 (29.10%) Amazon AWS
2 47 (15.72%) Microsoft Azure
3 30 (10.03%) Google Cloud Platform
4 9 (3.01%) Amazon EMR
4 9 (3.01%) Amazon S3
5 7 (2.34%) Amazon Kinesis
5 7 (2.34%) PaaS
6 5 (1.67%) GitHub
6 5 (1.67%) Serverless
7 4 (1.34%) Amazon EC2
7 4 (1.34%) AWS Lambda
8 3 (1.00%) IBM Cloud
9 2 (0.67%) Azure Data Factory
10 1 (0.33%) AWS CloudFormation
10 1 (0.33%) Cloud Computing
10 1 (0.33%) Data Lake Analytics
10 1 (0.33%) IaaS
10 1 (0.33%) OpenShift
10 1 (0.33%) SaaS
Communications & Networking
1 20 (6.69%) Kerberos
2 6 (2.01%) DNS
2 6 (2.01%) ICMP
2 6 (2.01%) TCP/IP
2 6 (2.01%) VPN
3 1 (0.33%) Intranet
3 1 (0.33%) Reverse Proxy
Database & Business Intelligence
1 299 (100.00%) Big Data
2 224 (74.92%) Hadoop
3 134 (44.82%) Apache Hive
4 73 (24.41%) NoSQL
5 54 (18.06%) SQL Server
6 53 (17.73%) HBase
7 50 (16.72%) Data Warehouse
8 48 (16.05%) Relational Database
9 32 (10.70%) MongoDB
10 30 (10.03%) Apache Cassandra
11 28 (9.36%) PostgreSQL
11 28 (9.36%) RDBMS
12 25 (8.36%) Impala
13 24 (8.03%) MySQL
14 22 (7.36%) Data Lake
14 22 (7.36%) Star Schema
15 20 (6.69%) Enterprise Data Warehouse
15 20 (6.69%) OLAP
16 19 (6.35%) OLAP Cube
17 18 (6.02%) Amazon Redshift
Development Applications
1 42 (14.05%) Git (software)
2 38 (12.71%) Jenkins
3 32 (10.70%) Maven
4 27 (9.03%) Visual Studio Team System
5 17 (5.69%) Artifactory
6 10 (3.34%) Bitbucket
6 10 (3.34%) Subversion
7 6 (2.01%) Cucumber
7 6 (2.01%) Eclipse
7 6 (2.01%) IntelliJ
7 6 (2.01%) JIRA
7 6 (2.01%) SonarQube
8 5 (1.67%) Sonatype Nexus
8 5 (1.67%) Visual Studio
9 3 (1.00%) GitLab
10 2 (0.67%) ERwin
10 2 (0.67%) JUnit
10 2 (0.67%) TeamCity
11 1 (0.33%) SpecFlow
11 1 (0.33%) Team Foundation Server
General
1 113 (37.79%) Finance
2 15 (5.02%) Banking
3 9 (3.01%) Marketing
4 7 (2.34%) Advertising
4 7 (2.34%) Games
4 7 (2.34%) Retail
5 5 (1.67%) Telecoms
6 4 (1.34%) Legal
7 1 (0.33%) Investment Banking
7 1 (0.33%) Retail Banking
Libraries, Frameworks & Software Standards
1 128 (42.81%) Kafka
2 33 (11.04%) AngularJS
3 21 (7.02%) OpenCV
4 20 (6.69%) Unity (Game Engine)
5 17 (5.69%) LDAP
5 17 (5.69%) Spring
6 14 (4.68%) Spring Boot
7 12 (4.01%) Elastic Stack
8 11 (3.68%) LEMP Stack
8 11 (3.68%) Spark Streaming
9 9 (3.01%) RESTful
10 8 (2.68%) JSON
10 8 (2.68%) MLlib
11 7 (2.34%) Samza
11 7 (2.34%) TensorFlow
12 6 (2.01%) RabbitMQ
12 6 (2.01%) REST
12 6 (2.01%) Sensu
12 6 (2.01%) Web Services
12 6 (2.01%) XML
Miscellaneous
1 36 (12.04%) Algorithms
2 25 (8.36%) Analytical Skills
3 20 (6.69%) HoloLens
4 16 (5.35%) Computer Science
5 8 (2.68%) Data Structures
6 7 (2.34%) Public Cloud
7 6 (2.01%) Greenfield Project
8 5 (1.67%) Fintech
8 5 (1.67%) Grid Computing
9 3 (1.00%) Linux Command Line
9 3 (1.00%) Mobile App
10 2 (0.67%) Derivative
10 2 (0.67%) Enterprise Software
10 2 (0.67%) Equity Derivative
10 2 (0.67%) Equity Risk
10 2 (0.67%) iPad
10 2 (0.67%) User Experience
10 2 (0.67%) Virtual Team
11 1 (0.33%) Clustering
11 1 (0.33%) Mobile Games
Operating Systems
1 74 (24.75%) Linux
2 33 (11.04%) Unix
3 13 (4.35%) Red Hat Enterprise Linux
4 9 (3.01%) Ubuntu
5 6 (2.01%) Windows
6 5 (1.67%) Windows Server
6 5 (1.67%) Windows Server 2008
7 4 (1.34%) CentOS
8 2 (0.67%) Android
8 2 (0.67%) Apple iOS
9 1 (0.33%) CoreOS
9 1 (0.33%) DC/OS
9 1 (0.33%) Mesosphere
Processes & Methodologies
1 104 (34.78%) Analytics
2 100 (33.44%) Agile Software Development
3 89 (29.77%) ETL
4 81 (27.09%) Data Science
5 58 (19.40%) DevOps
6 51 (17.06%) Scrum
7 48 (16.05%) Machine Learning
8 45 (15.05%) Data Modelling
8 45 (15.05%) Software Engineering
9 44 (14.72%) Data Engineering
10 42 (14.05%) Open Source
11 27 (9.03%) Problem-Solving
12 25 (8.36%) Deep Learning
13 21 (7.02%) Convolutional Neural Network
14 20 (6.69%) E-Commerce
15 19 (6.35%) Data Analytics
15 19 (6.35%) SQL Optimisation
15 19 (6.35%) Test Automation
15 19 (6.35%) Visualisation
16 18 (6.02%) Continuous Integration
Programming Languages
1 153 (51.17%) Python
2 139 (46.49%) Java
3 128 (42.81%) SQL
4 121 (40.47%) Scala
5 29 (9.70%) Java 8
6 16 (5.35%) JavaScript
7 12 (4.01%) Bash Shell
7 12 (4.01%) Shell Script
8 10 (3.34%) R
9 9 (3.01%) Perl
10 8 (2.68%) MATLAB
10 8 (2.68%) Ruby
11 7 (2.34%) Go
11 7 (2.34%) T-SQL
12 6 (2.01%) C
12 6 (2.01%) C#
12 6 (2.01%) Groovy
13 4 (1.34%) C++
14 2 (0.67%) PL/SQL
14 2 (0.67%) PowerShell
Qualifications
1 22 (7.36%) Security Cleared
2 21 (7.02%) SC Cleared
3 17 (5.69%) Degree
4 10 (3.34%) Computer Science Degree
5 7 (2.34%) DV Cleared
6 5 (1.67%) Numerate Degree
7 4 (1.34%) Master's Degree
8 3 (1.00%) PhD
Quality Assurance & Compliance
1 27 (9.03%) Data Quality
2 9 (3.01%) GDPR
3 5 (1.67%) GCP
3 5 (1.67%) QA
System Software
1 50 (16.72%) HDFS
2 41 (13.71%) Docker
3 21 (7.02%) Apache Flume
4 5 (1.67%) Apache ZooKeeper
5 2 (0.67%) Active Directory
Systems Management
1 39 (13.04%) Oozie
2 37 (12.37%) Ansible
3 25 (8.36%) Hadoop YARN
4 24 (8.03%) Kubernetes
5 16 (5.35%) Terraform
6 13 (4.35%) Ambari
7 11 (3.68%) Puppet
8 8 (2.68%) Prometheus
9 7 (2.34%) Host Intrusion Detection System
9 7 (2.34%) Kibana
9 7 (2.34%) Network Intrusion Detection System
9 7 (2.34%) Sysdig
10 6 (2.01%) Consul
10 6 (2.01%) HAProxy
10 6 (2.01%) Opscode Chef
11 5 (1.67%) Salt
12 4 (1.34%) logstash
13 2 (0.67%) Mesos
13 2 (0.67%) Packer
14 1 (0.33%) Nagios
Vendors
1 35 (11.71%) Cloudera
2 32 (10.70%) Google
3 24 (8.03%) Hortonworks
4 21 (7.02%) Microsoft
5 17 (5.69%) Alteryx
6 13 (4.35%) Tableau
6 13 (4.35%) Talend
7 12 (4.01%) Informatica
8 11 (3.68%) MapR
8 11 (3.68%) Splunk
9 10 (3.34%) Centrify
10 9 (3.01%) Oracle
10 9 (3.01%) Red Hat
11 7 (2.34%) SAP
12 6 (2.01%) Databricks
12 6 (2.01%) SaltStack
13 4 (1.34%) IBM
13 4 (1.34%) Salesforce.com
14 3 (1.00%) Ab Initio
15 2 (0.67%) Sybase