Period
to 11 December 2018

The median contractor rate for a Big Data Engineer was £575 per day in advertised job vacancies in London during the 6 months to 11 December 2018.

The first table below provides contractor rate benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Big Data Engineer
England > London
6 months to
11 Dec 2018
Same period 2017 Same period 2016
Rank 386 409 404
Rank change year-on-year +23 -5 +150
Contract jobs requiring a Big Data Engineer 260 247 218
As % of all contract IT jobs advertised in London 0.55% 0.52% 0.49%
As % of the Job Titles category 0.58% 0.55% 0.52%
Number of daily rates quoted 195 199 153
London median daily rate £575 £525 £560
Median daily rate % change year-on-year +9.52% -6.25% -2.61%
10th Percentile £450 £405 £440
90th Percentile £775 £650 £750
England median daily rate £550 £500 £550
% change year-on-year +10.00% -9.09% -
Number of hourly rates quoted 0 0 1
London median hourly rate - - £100.00
England median hourly rate - £27.50 £100.00
% change year-on-year - -72.50% +166.67%

The following table is for comparison with the above and includes summary statistics for all contract IT job vacancies advertised in London. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of contract jobs in our overall sample.

All Contract IT Job Vacancies
London
Contract vacancies in London with a recognized job title 44,628 45,089 42,170
% of contract IT jobs with a recognized job title 94.76% 94.95% 94.97%
Number of daily rates quoted 32,255 32,755 29,957
London median daily rate £500 £500 £463
Median daily rate % change year-on-year - +8.11% +2.78%
10th Percentile £313 £300 £294
90th Percentile £675 £663 £638
England median daily rate £459 £450 £425
% change year-on-year +1.89% +5.88% +3.03%
Number of hourly rates quoted 610 553 571
London median hourly rate £20.00 £20.44 £19.23
Median hourly rate % change year-on-year -2.13% +6.27% +1.13%
10th Percentile £12.73 £12.00 £11.32
90th Percentile £53.75 £47.83 £50.00
England median hourly rate £18.12 £18.70 £18.38
% change year-on-year -3.13% +1.77% +2.08%

Big Data Engineer
Job Vacancy Trend in London

Job postings that featured Big Data Engineer in the job title as a percentage of all IT jobs advertised in London.

Job vacancy trend for Big Data Engineer in London

Big Data Engineer
Contractor Daily Rate Trend in London

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Big Data Engineer in London.

Contractor daily rate trend for Big Data Engineer in London

Big Data Engineer
Contractor Daily Rate Histogram in London

The daily rate distribution of IT jobs citing Big Data Engineer in London over the 6 months to 11 December 2018.

Contractor daily rate histogram for Big Data Engineer in London

Big Data Engineer
Contract Job Locations in London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Big Data Engineer within the London region over the 6 months to 11 December 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +15 57 £575 +4.55% 13
East London -8 25 £525 -9.68% 2
West London +1 7 £565 +32.94% 5
South London -36 6 £600 +10.09% 1
Big Data Engineer
England

Big Data Engineer Skill Set
Top 30 Co-occurring IT Skills in London

For the 6 months to 11 December 2018, Big Data Engineer contract jobs required the following IT skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the London region featuring Big Data Engineer in the job title.

1 260 (100.00%) Big Data
2 186 (71.54%) Hadoop
3 157 (60.38%) Apache Spark
4 147 (56.54%) Kafka
5 139 (53.46%) Java
6 137 (52.69%) Python
7 105 (40.38%) Apache Hive
8 104 (40.00%) Agile Software Development
9 89 (34.23%) Scala
10 88 (33.85%) Amazon AWS
11 82 (31.54%) SQL
12 72 (27.69%) HBase
12 72 (27.69%) Analytics
13 69 (26.54%) Finance
14 68 (26.15%) ETL
15 65 (25.00%) DevOps
16 64 (24.62%) Data Science
17 57 (21.92%) Microsoft Azure
18 54 (20.77%) Jenkins
19 53 (20.38%) Linux
20 50 (19.23%) Scrum
20 50 (19.23%) NoSQL
20 50 (19.23%) Elasticsearch
21 47 (18.08%) Ansible
21 47 (18.08%) Google Cloud Platform
22 45 (17.31%) Git (software)
23 41 (15.77%) Apache
23 41 (15.77%) Hadoop YARN
24 40 (15.38%) HDFS
24 40 (15.38%) Apache Cassandra

Big Data Engineer Skill Set
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 157 (60.38%) Apache Spark
2 50 (19.23%) Elasticsearch
3 41 (15.77%) Apache
4 14 (5.38%) SAS
5 11 (4.23%) Apache Pig
6 7 (2.69%) Apache Solr
7 5 (1.92%) Confluence
8 3 (1.15%) Apache Airflow
8 3 (1.15%) Cloud Foundry
8 3 (1.15%) nginx
9 2 (0.77%) Luigi
9 2 (0.77%) Tomcat
10 1 (0.38%) Jupyter
Applications
1 3 (1.15%) Microsoft Excel
1 3 (1.15%) MS Visio
2 1 (0.38%) Chatbot
Business Applications
1 13 (5.00%) SAP Oil and Gas
2 1 (0.38%) SAP MM
2 1 (0.38%) SAP R/3
Cloud Services
1 88 (33.85%) Amazon AWS
2 57 (21.92%) Microsoft Azure
3 47 (18.08%) Google Cloud Platform
4 16 (6.15%) Amazon EMR
4 16 (6.15%) GitHub
5 14 (5.38%) Amazon S3
6 10 (3.85%) Amazon EC2
7 8 (3.08%) Azure Data Factory
7 8 (3.08%) Data Lake Analytics
8 6 (2.31%) IBM Cloud
9 4 (1.54%) AWS Lambda
9 4 (1.54%) IaaS
9 4 (1.54%) Serverless
10 3 (1.15%) Amazon Kinesis
10 3 (1.15%) SaaS
10 3 (1.15%) Virtual Private Cloud
11 2 (0.77%) Cloud Computing
11 2 (0.77%) PaaS
12 1 (0.38%) AWS CloudFormation
12 1 (0.38%) Stackdriver
Communications & Networking
1 12 (4.62%) Kerberos
2 4 (1.54%) DNS
3 3 (1.15%) VPN
4 2 (0.77%) ICMP
4 2 (0.77%) TCP/IP
5 1 (0.38%) DSL
5 1 (0.38%) Firewall
5 1 (0.38%) HTTP
5 1 (0.38%) HTTPS
5 1 (0.38%) Intranet
5 1 (0.38%) LAN
5 1 (0.38%) Reverse Proxy
5 1 (0.38%) SSL
5 1 (0.38%) WAN
Database & Business Intelligence
1 260 (100.00%) Big Data
2 186 (71.54%) Hadoop
3 105 (40.38%) Apache Hive
4 72 (27.69%) HBase
5 50 (19.23%) NoSQL
6 40 (15.38%) Apache Cassandra
7 37 (14.23%) MongoDB
8 35 (13.46%) Sqoop
9 33 (12.69%) Data Lake
9 33 (12.69%) SQL Server
10 31 (11.92%) Data Warehouse
11 28 (10.77%) Relational Database
12 20 (7.69%) Presto
13 18 (6.92%) PostgreSQL
13 18 (6.92%) Redis
14 15 (5.77%) Power BI
15 14 (5.38%) Impala
16 13 (5.00%) Data Mining
16 13 (5.00%) SAP HANA
16 13 (5.00%) SQL Server Integration Services
Development Applications
1 54 (20.77%) Jenkins
2 45 (17.31%) Git (software)
3 24 (9.23%) Maven
4 23 (8.85%) JIRA
5 13 (5.00%) Visual Studio Team System
6 11 (4.23%) Artifactory
7 9 (3.46%) Cucumber
8 8 (3.08%) Bitbucket
8 8 (3.08%) Sonatype Nexus
9 6 (2.31%) Gerrit
9 6 (2.31%) JBehave
9 6 (2.31%) JUnit
9 6 (2.31%) Selenium
10 5 (1.92%) Eclipse
11 4 (1.54%) Ant
11 4 (1.54%) CruiseControl
11 4 (1.54%) IBM UrbanCode
11 4 (1.54%) IntelliJ
11 4 (1.54%) SonarQube
11 4 (1.54%) Visual Studio
General
1 69 (26.54%) Finance
2 21 (8.08%) Banking
3 17 (6.54%) Legal
3 17 (6.54%) Retail
4 12 (4.62%) Publishing
5 4 (1.54%) Marketing
5 4 (1.54%) Telecoms
6 3 (1.15%) Advertising
7 2 (0.77%) Front Office
7 2 (0.77%) Investment Banking
7 2 (0.77%) Retail Banking
8 1 (0.38%) Corporate Banking
8 1 (0.38%) Dutch Language
8 1 (0.38%) Financial Institution
8 1 (0.38%) Manufacturing
Libraries, Frameworks & Software Standards
1 147 (56.54%) Kafka
2 23 (8.85%) AngularJS
2 23 (8.85%) Spring
3 20 (7.69%) Elastic Stack
3 20 (7.69%) Spark Streaming
4 18 (6.92%) RESTful
5 12 (4.62%) PySpark
5 12 (4.62%) Spring Boot
6 10 (3.85%) LDAP
6 10 (3.85%) REST
7 8 (3.08%) OpenCV
8 6 (2.31%) Gherkin
8 6 (2.31%) HTML
8 6 (2.31%) Unity (Game Engine)
9 5 (1.92%) CSS
9 5 (1.92%) HTML5
10 4 (1.54%) GraphQL
10 4 (1.54%) RabbitMQ
10 4 (1.54%) React
10 4 (1.54%) Web Services
Miscellaneous
1 35 (13.46%) Algorithms
2 19 (7.31%) Greenfield Project
3 18 (6.92%) Analytical Skills
4 9 (3.46%) Data Structures
5 6 (2.31%) HoloLens
6 5 (1.92%) Fintech
6 5 (1.92%) Virtual Team
7 4 (1.54%) Self-Motivation
8 3 (1.15%) iPad
8 3 (1.15%) Replication
8 3 (1.15%) Wiki
9 2 (0.77%) Cyber Defence
9 2 (0.77%) Data Centre
9 2 (0.77%) Management Information System
9 2 (0.77%) Mobile App
9 2 (0.77%) Public Cloud
10 1 (0.38%) Analytical Mindset
10 1 (0.38%) NHS
Operating Systems
1 53 (20.38%) Linux
2 36 (13.85%) Unix
3 7 (2.69%) Ubuntu
4 4 (1.54%) Windows
5 2 (0.77%) Android
5 2 (0.77%) Apple iOS
6 1 (0.38%) CentOS
6 1 (0.38%) DC/OS
6 1 (0.38%) Mesosphere
6 1 (0.38%) Red Hat Enterprise Linux
Processes & Methodologies
1 104 (40.00%) Agile Software Development
2 72 (27.69%) Analytics
3 68 (26.15%) ETL
4 65 (25.00%) DevOps
5 64 (24.62%) Data Science
6 50 (19.23%) Scrum
7 38 (14.62%) MapReduce
8 34 (13.08%) Data Engineering
9 30 (11.54%) TDD
10 28 (10.77%) Data Modelling
10 28 (10.77%) Machine Learning
11 27 (10.38%) Open Source
12 26 (10.00%) Test Automation
13 25 (9.62%) Software Engineering
14 24 (9.23%) Computer Science
15 20 (7.69%) Business Intelligence
15 20 (7.69%) Statistical Modelling
15 20 (7.69%) Visualisation
16 19 (7.31%) BDD
17 18 (6.92%) Continuous Integration
Programming Languages
1 139 (53.46%) Java
2 137 (52.69%) Python
3 89 (34.23%) Scala
4 82 (31.54%) SQL
5 21 (8.08%) JavaScript
5 21 (8.08%) R
6 19 (7.31%) Java 8
7 17 (6.54%) Bash Shell
8 12 (4.62%) Groovy
8 12 (4.62%) Ruby
9 11 (4.23%) C#
9 11 (4.23%) MATLAB
10 6 (2.31%) C++
10 6 (2.31%) Shell Script
11 5 (1.92%) AWK
12 4 (1.54%) PowerShell
13 3 (1.15%) Go
13 3 (1.15%) Korn
13 3 (1.15%) Perl
13 3 (1.15%) sed
Qualifications
1 29 (11.15%) Degree
2 18 (6.92%) Computer Science Degree
3 14 (5.38%) Security Cleared
4 13 (5.00%) SC Cleared
5 11 (4.23%) Master's Degree
6 1 (0.38%) CISMP
6 1 (0.38%) CISSP
6 1 (0.38%) Numerate Degree
6 1 (0.38%) PhD
Quality Assurance & Compliance
1 13 (5.00%) Data Quality
2 5 (1.92%) GDPR
3 4 (1.54%) QA
4 2 (0.77%) SLA
5 1 (0.38%) ISO/IEC 27001
System Software
1 40 (15.38%) HDFS
2 37 (14.23%) Docker
3 26 (10.00%) Apache Flume
4 19 (7.31%) Apache ZooKeeper
5 3 (1.15%) Active Directory
6 1 (0.38%) Oracle RAC
6 1 (0.38%) Virtual Machines
Systems Management
1 47 (18.08%) Ansible
2 41 (15.77%) Hadoop YARN
3 32 (12.31%) Oozie
4 21 (8.08%) Kubernetes
5 17 (6.54%) Kibana
5 17 (6.54%) Terraform
6 12 (4.62%) logstash
7 11 (4.23%) Ambari
8 9 (3.46%) Puppet
9 7 (2.69%) Opscode Chef
10 4 (1.54%) Consul
11 3 (1.15%) Grafana
11 3 (1.15%) Mesos
11 3 (1.15%) Prometheus
12 2 (0.77%) HAProxy
12 2 (0.77%) Host Intrusion Detection System
12 2 (0.77%) Network Intrusion Detection System
12 2 (0.77%) Salt
12 2 (0.77%) Sysdig
13 1 (0.38%) Marathon
Vendors
1 40 (15.38%) Google
2 28 (10.77%) Cloudera
2 28 (10.77%) Hortonworks
3 26 (10.00%) Oracle
4 25 (9.62%) Talend
5 20 (7.69%) Alteryx
6 19 (7.31%) Informatica
7 15 (5.77%) Databricks
8 13 (5.00%) SAP
8 13 (5.00%) Splunk
8 13 (5.00%) Tableau
9 11 (4.23%) Microsoft
10 8 (3.08%) Red Hat
11 7 (2.69%) Centrify
12 6 (2.31%) IBM
13 2 (0.77%) MapR
13 2 (0.77%) New Relic
13 2 (0.77%) Teradata
14 1 (0.38%) Ab Initio
14 1 (0.38%) F5