Period
to 17 December 2017

The median contractor rate for a Big Data Engineer was £525 per day in advertised job vacancies in London during the 6 months to 17 December 2017.

The first table below provides contractor rate benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Big Data Engineer
England > London
6 months to
17 Dec 2017
Same period 2016 Same period 2015
Rank 403 400 536
Rank change year-on-year -3 +136 +14
Contract jobs requiring a Big Data Engineer 240 221 60
As % of all contract IT jobs advertised in London 0.51% 0.50% 0.13%
As % of the Job Titles category 0.54% 0.53% 0.14%
Number of daily rates quoted 192 155 48
London median daily rate £525 £550 £588
Median daily rate % change year-on-year -4.55% -6.38% +17.50%
10th Percentile £406 £438 £471
90th Percentile £650 £740 £675
England median daily rate £500 £550 £563
% change year-on-year -9.09% -2.22% +12.50%
Number of hourly rates quoted 0 1 1
London median hourly rate - £100.00 £37.50
Median hourly rate % change year-on-year - +166.67% -
England median hourly rate £27.50 £100.00 £37.50
% change year-on-year -72.50% +166.67% -

The following table is for comparison with the above and includes summary statistics for all contract IT job vacancies advertised in London. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of contract jobs in our overall sample.

All Contract IT Job Vacancies
London
Contract vacancies in London with a recognized job title 44783 41879 43850
% of contract IT jobs with a recognized job title 94.84% 94.88% 95.09%
Number of daily rates quoted 32527 29761 30910
London median daily rate £500 £463 £450
Median daily rate % change year-on-year +8.11% +2.78% +5.88%
10th Percentile £300 £299 £269
90th Percentile £669 £638 £638
England median daily rate £450 £425 £413
% change year-on-year +5.88% +3.03% +3.13%
Number of hourly rates quoted 551 559 668
London median hourly rate £20.00 £19.50 £19.02
Median hourly rate % change year-on-year +2.56% +2.55% +8.66%
10th Percentile £12.00 £11.38 £11.25
90th Percentile £47.83 £50.00 £50.00
England median hourly rate £18.70 £18.00 £17.89
% change year-on-year +3.89% +0.61% +8.42%

Big Data Engineer
Job Vacancy Trend in London

Job postings that featured Big Data Engineer in the job title as a percentage of all IT jobs advertised in London.

Job vacancy trend for Big Data Engineer in London

Big Data Engineer
Contractor Daily Rate Trend in London

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Big Data Engineer in London.

Contractor daily rate trend for Big Data Engineer in London

Big Data Engineer
Contractor Daily Rate Histogram in London

The daily rate distribution of IT jobs citing Big Data Engineer in London over the 6 months to 17 December 2017.

Contractor daily rate histogram for Big Data Engineer in London

Big Data Engineer
Contract Job Locations in London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Big Data Engineer within the London region over the 6 months to 17 December 2017. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London -13 25 £550 -4.35% 10
South London -18 20 £545 -
East London -7 2 £581 +1.09%
Big Data Engineer
England

Big Data Engineer Skill Set
Top 30 Co-occurring IT Skills in London

For the 6 months to 17 December 2017, Big Data Engineer contract jobs required the following IT skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all contract ads across the London region featuring Big Data Engineer in the job title.

1 240 (100.00%) Big Data
2 209 (87.08%) Apache Spark
3 183 (76.25%) Hadoop
4 141 (58.75%) Java
5 135 (56.25%) Python
6 111 (46.25%) Scala
7 98 (40.83%) Kafka
8 92 (38.33%) Apache Hive
9 85 (35.42%) Amazon AWS
10 68 (28.33%) HDFS
11 64 (26.67%) Agile Software Development
12 61 (25.42%) HBase
13 57 (23.75%) Analytics
14 56 (23.33%) SQL
15 55 (22.92%) DevOps
16 50 (20.83%) TDD
17 48 (20.00%) PostgreSQL
17 48 (20.00%) Apache
18 46 (19.17%) Data Engineering
19 45 (18.75%) Docker
19 45 (18.75%) ETL
19 45 (18.75%) NoSQL
20 44 (18.33%) MapReduce
20 44 (18.33%) Linux
21 42 (17.50%) Apache Flume
22 40 (16.67%) Open Source
23 38 (15.83%) Scrum
24 37 (15.42%) Software Engineering
24 37 (15.42%) Data Architecture
24 37 (15.42%) Machine Learning

Big Data Engineer Skill Set
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 209 (87.08%) Apache Spark
2 48 (20.00%) Apache
3 25 (10.42%) Apache Solr
4 22 (9.17%) Elasticsearch
5 19 (7.92%) Apache Pig
6 12 (5.00%) Confluence
7 5 (2.08%) Tomcat
8 4 (1.67%) nginx
9 2 (0.83%) GeoServer
10 1 (0.42%) Jupyter
10 1 (0.42%) OpenStack
Applications
1 1 (0.42%) Microsoft PowerPoint
Cloud Services
1 85 (35.42%) Amazon AWS
2 25 (10.42%) GitHub
3 15 (6.25%) Bluemix
3 15 (6.25%) Google Cloud Platform
3 15 (6.25%) IBM Cloud
3 15 (6.25%) Microsoft Azure
4 14 (5.83%) Amazon EMR
5 12 (5.00%) Amazon EC2
5 12 (5.00%) Amazon S3
6 8 (3.33%) Virtual Private Cloud
7 5 (2.08%) Amazon Glacier
7 5 (2.08%) Amazon Kinesis
8 4 (1.67%) AWS Data Pipeline
8 4 (1.67%) AWS Lambda
8 4 (1.67%) Serverless
9 3 (1.25%) Azure Data Factory
10 2 (0.83%) Amazon SQS
10 2 (0.83%) AWS CloudFormation
10 2 (0.83%) Cloud Computing
10 2 (0.83%) PaaS
Communications & Networking
1 30 (12.50%) Kerberos
2 5 (2.08%) LDAP
3 2 (0.83%) Network Security
3 2 (0.83%) TCP/IP
4 1 (0.42%) Internet
4 1 (0.42%) Multicast
4 1 (0.42%) SSL
Database & Business Intelligence
1 240 (100.00%) Big Data
2 183 (76.25%) Hadoop
3 92 (38.33%) Apache Hive
4 61 (25.42%) HBase
5 48 (20.00%) PostgreSQL
6 45 (18.75%) NoSQL
7 33 (13.75%) Apache Cassandra
8 27 (11.25%) Sqoop
9 23 (9.58%) Data Warehouse
10 20 (8.33%) MongoDB
11 19 (7.92%) Impala
12 17 (7.08%) Amazon Redshift
13 16 (6.67%) Flink
14 12 (5.00%) Presto
15 10 (4.17%) Relational Database
16 7 (2.92%) Data Lake
17 5 (2.08%) BigQuery
17 5 (2.08%) Mahout
17 5 (2.08%) MySQL
17 5 (2.08%) Oracle Database
Development Applications
1 23 (9.58%) Jenkins
2 18 (7.50%) Cucumber
3 17 (7.08%) JIRA
4 9 (3.75%) Git (software)
5 8 (3.33%) Talend Open Studio
6 4 (1.67%) Subversion
7 3 (1.25%) GitLab
8 2 (0.83%) ERwin
8 2 (0.83%) Vagrant
8 2 (0.83%) Visual Studio
8 2 (0.83%) Visual Studio Team System
9 1 (0.42%) Atlassian Bamboo
9 1 (0.42%) IntelliJ
9 1 (0.42%) JUnit
9 1 (0.42%) Maven
9 1 (0.42%) Mockito
General
1 27 (11.25%) Finance
2 14 (5.83%) Marketing
3 13 (5.42%) Publishing
4 11 (4.58%) Banking
5 9 (3.75%) Telecoms
6 5 (2.08%) Retail
7 4 (1.67%) Electronics
7 4 (1.67%) Games
7 4 (1.67%) Manufacturing
8 2 (0.83%) Investment Banking
9 1 (0.42%) Law
9 1 (0.42%) Legal
9 1 (0.42%) Music Industry
9 1 (0.42%) Retail Banking
Libraries, Frameworks & Software Standards
1 98 (40.83%) Kafka
2 31 (12.92%) REST
3 30 (12.50%) PySpark
3 30 (12.50%) Spring
4 24 (10.00%) Spring Boot
5 21 (8.75%) Web Services
6 19 (7.92%) Drools
7 11 (4.58%) Akka
7 11 (4.58%) JSON
8 10 (4.17%) Apache Camel
9 8 (3.33%) LAMP
9 8 (3.33%) Spark Streaming
10 5 (2.08%) Apache Drill
10 5 (2.08%) LAPP Stack
10 5 (2.08%) Node.js
11 4 (1.67%) gRPC
11 4 (1.67%) HTML
11 4 (1.67%) Sensu
12 3 (1.25%) Lucene
12 3 (1.25%) RabbitMQ
Miscellaneous
1 25 (10.42%) Algorithms
2 20 (8.33%) Computer Science
3 9 (3.75%) Analytical Skills
3 9 (3.75%) User Experience
4 7 (2.92%) Data Structures
4 7 (2.92%) Fintech
5 4 (1.67%) Greenfield Project
5 4 (1.67%) Online Games
5 4 (1.67%) Semantic Web
6 2 (0.83%) Linked Data
7 1 (0.42%) Enterprise Software
7 1 (0.42%) FMCG
7 1 (0.42%) Grid Computing
7 1 (0.42%) Linux Command Line
7 1 (0.42%) Management Information System
7 1 (0.42%) Replication
7 1 (0.42%) Video\Live Streaming
7 1 (0.42%) Virtual Team
Operating Systems
1 44 (18.33%) Linux
2 15 (6.25%) Unix
3 10 (4.17%) Red Hat Enterprise Linux
4 3 (1.25%) Ubuntu
5 2 (0.83%) CentOS
6 1 (0.42%) Windows
6 1 (0.42%) Windows Server
6 1 (0.42%) Windows Server 2008
Processes & Methodologies
1 64 (26.67%) Agile Software Development
2 57 (23.75%) Analytics
3 55 (22.92%) DevOps
4 50 (20.83%) TDD
5 46 (19.17%) Data Engineering
6 45 (18.75%) ETL
7 44 (18.33%) MapReduce
8 40 (16.67%) Open Source
9 38 (15.83%) Scrum
10 37 (15.42%) Data Architecture
10 37 (15.42%) Machine Learning
10 37 (15.42%) Software Engineering
11 28 (11.67%) BDD
12 27 (11.25%) Pair Programming
13 24 (10.00%) Microservices
14 23 (9.58%) Data Analytics
15 22 (9.17%) Mentoring
16 21 (8.75%) Data Modelling
17 17 (7.08%) Business Intelligence
18 16 (6.67%) Big Data Analytics
Programming Languages
1 141 (58.75%) Java
2 135 (56.25%) Python
3 111 (46.25%) Scala
4 56 (23.33%) SQL
5 13 (5.42%) Shell Script
6 12 (5.00%) R
7 9 (3.75%) JavaScript
8 8 (3.33%) PHP
9 7 (2.92%) Bash Shell
10 5 (2.08%) PL/SQL
11 4 (1.67%) Perl
11 4 (1.67%) SPARQL
12 3 (1.25%) Ruby
13 2 (0.83%) C
13 2 (0.83%) C#
13 2 (0.83%) C++
13 2 (0.83%) PowerShell
14 1 (0.42%) GraphQL
Qualifications
1 18 (7.50%) Security Cleared
2 14 (5.83%) SC Cleared
3 9 (3.75%) Degree
4 5 (2.08%) Computer Science Degree
5 3 (1.25%) Master's Degree
6 1 (0.42%) Numerate Degree
Quality Assurance & Compliance
1 10 (4.17%) QA
2 8 (3.33%) Actionable Recommendations
3 7 (2.92%) Data Quality
4 4 (1.67%) GCP
5 2 (0.83%) GDPR
System Software
1 68 (28.33%) HDFS
2 45 (18.75%) Docker
3 42 (17.50%) Apache Flume
4 19 (7.92%) Apache ZooKeeper
5 1 (0.42%) Active Directory
5 1 (0.42%) vCloud
5 1 (0.42%) VMware Infrastructure
Systems Management
1 34 (14.17%) Oozie
2 27 (11.25%) Ambari
3 26 (10.83%) Hadoop YARN
4 24 (10.00%) Terraform
5 23 (9.58%) Ansible
6 16 (6.67%) Kubernetes
7 13 (5.42%) Mesos
8 9 (3.75%) Puppet
9 7 (2.92%) Opscode Chef
10 4 (1.67%) Salt
11 3 (1.25%) Packer
12 1 (0.42%) Prometheus
Vendors
1 29 (12.08%) Cloudera
2 23 (9.58%) Google
2 23 (9.58%) Tableau
3 20 (8.33%) Hortonworks
4 15 (6.25%) IBM
5 9 (3.75%) MapR
5 9 (3.75%) Oracle
6 8 (3.33%) Informatica
6 8 (3.33%) Talend
7 6 (2.50%) Splunk
8 5 (2.08%) Teradata
9 4 (1.67%) Microsoft
9 4 (1.67%) SaltStack
9 4 (1.67%) SAP
10 2 (0.83%) Alteryx
11 1 (0.42%) Red Hat
11 1 (0.42%) VMware