Period
to 17 October 2018

The median contractor rate for a Big Data Engineer was £575 per day in advertised job vacancies in London during the 6 months to 17 October 2018.

The first table below provides contractor rate benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Big Data Engineer
England > London
6 months to
17 Oct 2018
Same period 2017 Same period 2016
Rank 379 388 478
Rank change year-on-year +9 +90 +96
Contract jobs requiring a Big Data Engineer 265 250 137
As % of all contract IT jobs advertised in London 0.57% 0.56% 0.31%
As % of the Job Titles category 0.61% 0.59% 0.32%
Number of daily rates quoted 202 195 100
London median daily rate £575 £500 £590
Median daily rate % change year-on-year +15.00% -15.25% +7.27%
10th Percentile £450 £400 £463
90th Percentile £750 £650 £825
England median daily rate £550 £500 £555
% change year-on-year +10.00% -9.91% +3.74%
Number of hourly rates quoted 1 0 0
London median hourly rate £15.50 - -
England median hourly rate £15.50 - -

The following table is for comparison with the above and includes summary statistics for all contract IT job vacancies advertised in London. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of contract jobs in our overall sample.

All Contract IT Job Vacancies
London
Contract vacancies in London with a recognized job title 43,743 42,706 42,620
% of contract IT jobs with a recognized job title 94.71% 94.94% 95.15%
Number of daily rates quoted 31,599 30,786 30,444
London median daily rate £500 £488 £450
Median daily rate % change year-on-year +2.56% +8.33% -
10th Percentile £313 £300 £285
90th Percentile £675 £656 £638
England median daily rate £450 £450 £425
% change year-on-year - +5.88% +6.25%
Number of hourly rates quoted 598 519 543
London median hourly rate £19.00 £20.50 £19.03
Median hourly rate % change year-on-year -7.32% +7.72% +0.16%
10th Percentile £12.75 £12.50 £11.25
90th Percentile £53.75 £47.83 £52.20
England median hourly rate £18.30 £18.70 £18.50
% change year-on-year -2.14% +1.08% +4.11%

Big Data Engineer
Job Vacancy Trend in London

Job postings that featured Big Data Engineer in the job title as a percentage of all IT jobs advertised in London.

Job vacancy trend for Big Data Engineer in London

Big Data Engineer
Contractor Daily Rate Trend in London

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Big Data Engineer in London.

Contractor daily rate trend for Big Data Engineer in London

Big Data Engineer
Contractor Daily Rate Histogram in London

The daily rate distribution of IT jobs citing Big Data Engineer in London over the 6 months to 17 October 2018.

Contractor daily rate histogram for Big Data Engineer in London

Big Data Engineer
Contract Job Locations in London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Big Data Engineer within the London region over the 6 months to 17 October 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +14 64 £575 +7.98% 9
East London - 22 £563 - 1
South London -38 5 £625 +15.74%
West London -15 4 £595 +40.00% 3
Big Data Engineer
England

Big Data Engineer Skill Set
Top 30 Co-occurring IT Skills in London

For the 6 months to 17 October 2018, Big Data Engineer contract jobs required the following IT skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the London region featuring Big Data Engineer in the job title.

1 265 (100.00%) Big Data
2 202 (76.23%) Hadoop
3 169 (63.77%) Apache Spark
4 142 (53.58%) Kafka
4 142 (53.58%) Java
5 122 (46.04%) Python
6 113 (42.64%) Apache Hive
7 104 (39.25%) Agile Software Development
8 96 (36.23%) Scala
9 93 (35.09%) SQL
10 81 (30.57%) Finance
11 79 (29.81%) Analytics
12 69 (26.04%) Amazon AWS
13 68 (25.66%) Data Science
14 63 (23.77%) Linux
15 62 (23.40%) ETL
16 60 (22.64%) HBase
17 57 (21.51%) DevOps
18 54 (20.38%) Elasticsearch
18 54 (20.38%) NoSQL
19 52 (19.62%) Git (software)
19 52 (19.62%) Scrum
20 47 (17.74%) Jenkins
21 46 (17.36%) Ansible
21 46 (17.36%) SQL Server
22 43 (16.23%) Unix
23 41 (15.47%) Apache
23 41 (15.47%) Microsoft Azure
23 41 (15.47%) Google Cloud Platform
24 40 (15.09%) HDFS

Big Data Engineer Skill Set
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 169 (63.77%) Apache Spark
2 54 (20.38%) Elasticsearch
3 41 (15.47%) Apache
4 8 (3.02%) Apache Pig
5 6 (2.26%) Apache Solr
5 6 (2.26%) SAS
6 5 (1.89%) Tomcat
7 4 (1.51%) Confluence
8 2 (0.75%) Apache Airflow
8 2 (0.75%) Cloud Foundry
9 1 (0.38%) Luigi
9 1 (0.38%) nginx
9 1 (0.38%) OpenStack
Applications
1 3 (1.13%) Microsoft Excel
1 3 (1.13%) MS Visio
Business Applications
1 5 (1.89%) SAP Oil and Gas
2 1 (0.38%) Markit EDM
2 1 (0.38%) SAP MM
2 1 (0.38%) SAP R/3
Cloud Services
1 69 (26.04%) Amazon AWS
2 41 (15.47%) Google Cloud Platform
2 41 (15.47%) Microsoft Azure
3 17 (6.42%) GitHub
4 12 (4.53%) Amazon S3
5 10 (3.77%) Amazon EC2
5 10 (3.77%) Amazon EMR
6 5 (1.89%) IBM Cloud
7 4 (1.51%) Amazon Kinesis
8 3 (1.13%) AWS Lambda
8 3 (1.13%) Serverless
8 3 (1.13%) Virtual Private Cloud
9 2 (0.75%) Cloud Computing
10 1 (0.38%) AWS CloudFormation
10 1 (0.38%) AWS Elastic Beanstalk
10 1 (0.38%) Azure Data Factory
10 1 (0.38%) PaaS
10 1 (0.38%) SaaS
Communications & Networking
1 17 (6.42%) Kerberos
2 3 (1.13%) DNS
3 2 (0.75%) VPN
4 1 (0.38%) Firewall
4 1 (0.38%) HTTP
4 1 (0.38%) HTTPS
4 1 (0.38%) ICMP
4 1 (0.38%) Intranet
4 1 (0.38%) Reverse Proxy
4 1 (0.38%) SSL
4 1 (0.38%) TCP/IP
Database & Business Intelligence
1 265 (100.00%) Big Data
2 202 (76.23%) Hadoop
3 113 (42.64%) Apache Hive
4 60 (22.64%) HBase
5 54 (20.38%) NoSQL
6 46 (17.36%) SQL Server
7 37 (13.96%) Relational Database
8 36 (13.58%) Data Warehouse
9 32 (12.08%) Sqoop
10 28 (10.57%) Apache Cassandra
11 24 (9.06%) Data Lake
12 21 (7.92%) MongoDB
13 20 (7.55%) Presto
14 18 (6.79%) RDBMS
15 17 (6.42%) PostgreSQL
16 12 (4.53%) Enterprise Data Warehouse
16 12 (4.53%) OLAP
16 12 (4.53%) OLAP Cube
16 12 (4.53%) Power BI
16 12 (4.53%) Star Schema
Development Applications
1 52 (19.62%) Git (software)
2 47 (17.74%) Jenkins
3 33 (12.45%) Maven
4 21 (7.92%) JIRA
5 20 (7.55%) Visual Studio Team System
6 13 (4.91%) Artifactory
6 13 (4.91%) Bitbucket
7 8 (3.02%) Eclipse
7 8 (3.02%) IntelliJ
8 7 (2.64%) Cucumber
8 7 (2.64%) Sonatype Nexus
9 6 (2.26%) JUnit
10 5 (1.89%) Visual Studio
11 4 (1.51%) Ant
11 4 (1.51%) CruiseControl
11 4 (1.51%) IBM UrbanCode
12 3 (1.13%) Gerrit
12 3 (1.13%) JBehave
12 3 (1.13%) Selenium
12 3 (1.13%) Subversion
General
1 81 (30.57%) Finance
2 19 (7.17%) Banking
3 12 (4.53%) Legal
4 11 (4.15%) Publishing
5 8 (3.02%) Retail
6 5 (1.89%) Marketing
7 4 (1.51%) Advertising
8 3 (1.13%) Telecoms
9 2 (0.75%) Front Office
9 2 (0.75%) Retail Banking
10 1 (0.38%) Corporate Banking
10 1 (0.38%) Dutch Language
10 1 (0.38%) Financial Institution
10 1 (0.38%) Investment Banking
10 1 (0.38%) Manufacturing
Libraries, Frameworks & Software Standards
1 142 (53.58%) Kafka
2 29 (10.94%) AngularJS
3 23 (8.68%) Spring
4 19 (7.17%) RESTful
5 17 (6.42%) Spark Streaming
6 15 (5.66%) Elastic Stack
6 15 (5.66%) LDAP
6 15 (5.66%) OpenCV
7 13 (4.91%) Unity (Game Engine)
8 12 (4.53%) Spring Boot
9 11 (4.15%) REST
10 6 (2.26%) MLlib
11 5 (1.89%) CSS
11 5 (1.89%) HTML
11 5 (1.89%) HTML5
11 5 (1.89%) PySpark
12 4 (1.51%) GraphQL
12 4 (1.51%) Node.js
12 4 (1.51%) RabbitMQ
12 4 (1.51%) Web Services
Miscellaneous
1 29 (10.94%) Algorithms
2 19 (7.17%) Greenfield Project
3 13 (4.91%) Analytical Skills
3 13 (4.91%) HoloLens
4 11 (4.15%) Computer Science
5 6 (2.26%) Fintech
5 6 (2.26%) Virtual Team
6 3 (1.13%) Data Structures
6 3 (1.13%) Mobile App
6 3 (1.13%) Wiki
7 2 (0.75%) Cyber Defence
7 2 (0.75%) iPad
7 2 (0.75%) Management Information System
7 2 (0.75%) Self-Motivation
7 2 (0.75%) User Experience
8 1 (0.38%) NHS
8 1 (0.38%) Public Cloud
Operating Systems
1 63 (23.77%) Linux
2 43 (16.23%) Unix
3 12 (4.53%) Ubuntu
4 5 (1.89%) Red Hat Enterprise Linux
5 4 (1.51%) Windows
6 3 (1.13%) CentOS
7 2 (0.75%) Android
7 2 (0.75%) Apple iOS
8 1 (0.38%) DC/OS
8 1 (0.38%) Mesosphere
Processes & Methodologies
1 104 (39.25%) Agile Software Development
2 79 (29.81%) Analytics
3 68 (25.66%) Data Science
4 62 (23.40%) ETL
5 57 (21.51%) DevOps
6 52 (19.62%) Scrum
7 35 (13.21%) Data Engineering
8 33 (12.45%) Machine Learning
9 32 (12.08%) MapReduce
10 31 (11.70%) Data Modelling
10 31 (11.70%) TDD
11 30 (11.32%) Software Engineering
12 29 (10.94%) Open Source
13 20 (7.55%) Test Automation
14 18 (6.79%) Data Analytics
14 18 (6.79%) SDLC
15 17 (6.42%) Continuous Integration
16 16 (6.04%) BDD
16 16 (6.04%) Deep Learning
17 15 (5.66%) Convolutional Neural Network
Programming Languages
1 142 (53.58%) Java
2 122 (46.04%) Python
3 96 (36.23%) Scala
4 93 (35.09%) SQL
5 24 (9.06%) Java 8
6 15 (5.66%) JavaScript
7 13 (4.91%) Bash Shell
7 13 (4.91%) R
8 10 (3.77%) C#
8 10 (3.77%) MATLAB
9 7 (2.64%) Groovy
9 7 (2.64%) Ruby
10 6 (2.26%) C++
10 6 (2.26%) Shell Script
11 5 (1.89%) Perl
11 5 (1.89%) T-SQL
12 3 (1.13%) Java 9
12 3 (1.13%) Korn
12 3 (1.13%) PowerShell
13 2 (0.75%) Gremlin
Qualifications
1 12 (4.53%) SC Cleared
1 12 (4.53%) Security Cleared
2 11 (4.15%) Degree
3 7 (2.64%) Computer Science Degree
4 3 (1.13%) PhD
5 2 (0.75%) Master's Degree
6 1 (0.38%) CISMP
6 1 (0.38%) CISSP
Quality Assurance & Compliance
1 21 (7.92%) Data Quality
2 5 (1.89%) GDPR
3 3 (1.13%) QA
4 1 (0.38%) ISO/IEC 27001
System Software
1 40 (15.09%) HDFS
2 39 (14.72%) Docker
3 20 (7.55%) Apache Flume
4 5 (1.89%) Apache ZooKeeper
5 3 (1.13%) Active Directory
Systems Management
1 46 (17.36%) Ansible
2 38 (14.34%) Oozie
3 31 (11.70%) Hadoop YARN
4 19 (7.17%) Kubernetes
5 14 (5.28%) Kibana
5 14 (5.28%) Terraform
6 12 (4.53%) logstash
7 8 (3.02%) Ambari
7 8 (3.02%) Puppet
8 6 (2.26%) Opscode Chef
9 2 (0.75%) Mesos
9 2 (0.75%) Prometheus
10 1 (0.38%) Apache Brooklyn
10 1 (0.38%) Consul
10 1 (0.38%) Grafana
10 1 (0.38%) Host Intrusion Detection System
10 1 (0.38%) Marathon
10 1 (0.38%) Network Intrusion Detection System
10 1 (0.38%) Single Sign-On
10 1 (0.38%) Sysdig
Vendors
1 37 (13.96%) Google
2 22 (8.30%) Hortonworks
3 21 (7.92%) Cloudera
4 17 (6.42%) Informatica
5 15 (5.66%) Oracle
5 15 (5.66%) Talend
6 13 (4.91%) Microsoft
6 13 (4.91%) Red Hat
7 12 (4.53%) Centrify
8 11 (4.15%) Alteryx
8 11 (4.15%) Splunk
8 11 (4.15%) Tableau
9 7 (2.64%) Databricks
10 5 (1.89%) IBM
10 5 (1.89%) MapR
10 5 (1.89%) SAP
11 3 (1.13%) Ab Initio
12 1 (0.38%) Reuters
12 1 (0.38%) Teradata
12 1 (0.38%) VMware