Period
to 25 September 2018

The median contractor rate for a Big Data Engineer was £550 per day in advertised job vacancies in England during the 6 months to 25 September 2018.

The first table below provides contractor rate benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Big Data Engineer
UK > England
6 months to
25 Sep 2018
Same period 2017 Same period 2016
Rank 508 545 726
Rank change year-on-year +37 +181 +91
Contract jobs requiring a Big Data Engineer 382 298 155
As % of all contract IT jobs advertised in England 0.41% 0.33% 0.16%
As % of the Job Titles category 0.43% 0.35% 0.17%
Number of daily rates quoted 297 230 103
England median daily rate £550 £506 £575
Median daily rate % change year-on-year +8.64% -11.96% +4.55%
10th Percentile £450 £400 £421
90th Percentile £721 £638 £825
UK median daily rate £550 £500 £575
% change year-on-year +10.00% -13.04% +4.55%
Number of hourly rates quoted 1 0 0
England median hourly rate £15.50 - -
UK median hourly rate £15.50 - -

The following table is for comparison with the above and includes summary statistics for all contract IT job vacancies advertised in England. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of contract jobs in our overall sample.

All Contract IT Job Vacancies
England
Contract vacancies in England with a recognized job title 89,014 85,687 93,164
% of contract IT jobs with a recognized job title 94.72% 94.88% 94.96%
Number of daily rates quoted 59,186 56,557 61,513
England median daily rate £450 £450 £425
Median daily rate % change year-on-year - +5.88% +6.25%
10th Percentile £275 £263 £258
90th Percentile £638 £625 £600
UK median daily rate £450 £438 £425
% change year-on-year +2.86% +2.94% +6.25%
Number of hourly rates quoted 2,851 2,541 2,879
England median hourly rate £18.00 £18.70 £19.00
Median hourly rate % change year-on-year -3.74% -1.58% +7.04%
10th Percentile £11.11 £10.75 £10.50
90th Percentile £48.75 £48.75 £46.00
UK median hourly rate £17.50 £18.94 £18.94
% change year-on-year -7.60% +0.03% +6.02%

Big Data Engineer
Job Vacancy Trend in England

Job postings that featured Big Data Engineer in the job title as a percentage of all IT jobs advertised in England.

Job vacancy trend for Big Data Engineer in England

Big Data Engineer
Contractor Daily Rate Trend in England

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Big Data Engineer in England.

Contractor daily rate trend for Big Data Engineer in England

Big Data Engineer
Contractor Daily Rate Histogram in England

The daily rate distribution of IT jobs citing Big Data Engineer in England over the 6 months to 25 September 2018.

Contractor daily rate histogram for Big Data Engineer in England

Big Data Engineer
Contractor Hourly Rate Trend in England

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing Big Data Engineer in England.

Contractor hourly rate trend for Big Data Engineer in England

Big Data Engineer
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Big Data Engineer within the England region over the 6 months to 25 September 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London +28 289 £575 +9.52% 33
South East -33 31 £475 -9.52% 4
East of England +5 23 £475 -2.56%
South West -1 17 £525 -16.83% 6
North of England -8 14 £575 +21.05% 2
North West -17 8 £600 +29.73% 2
Midlands -15 6 £482 +7.17% 2
Yorkshire -21 6 £550 +15.79%
East Midlands - 4 £465 - 2
West Midlands -23 2 £495 +9.89%
Big Data Engineer
UK

Big Data Engineer Skill Set
Top 30 Co-occurring IT Skills in England

For the 6 months to 25 September 2018, Big Data Engineer contract jobs required the following IT skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region featuring Big Data Engineer in the job title.

1 382 (100.00%) Big Data
2 297 (77.75%) Hadoop
3 267 (69.90%) Apache Spark
4 193 (50.52%) Kafka
5 192 (50.26%) Java
6 179 (46.86%) Python
7 165 (43.19%) Apache Hive
8 149 (39.01%) SQL
9 147 (38.48%) Scala
10 135 (35.34%) Analytics
11 122 (31.94%) Agile Software Development
12 120 (31.41%) Data Science
13 118 (30.89%) Finance
14 115 (30.10%) Amazon AWS
15 90 (23.56%) ETL
16 84 (21.99%) Linux
17 82 (21.47%) NoSQL
18 79 (20.68%) HBase
19 71 (18.59%) DevOps
20 70 (18.32%) Data Warehouse
21 64 (16.75%) Scrum
22 63 (16.49%) Git (software)
23 61 (15.97%) Apache
23 61 (15.97%) Machine Learning
23 61 (15.97%) Oozie
24 58 (15.18%) Elasticsearch
25 57 (14.92%) SQL Server
26 55 (14.40%) Microsoft Azure
27 54 (14.14%) Jenkins
27 54 (14.14%) Data Modelling

Big Data Engineer Skill Set
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 267 (69.90%) Apache Spark
2 61 (15.97%) Apache
3 58 (15.18%) Elasticsearch
4 10 (2.62%) Apache Airflow
4 10 (2.62%) Apache Pig
4 10 (2.62%) Luigi
5 8 (2.09%) Oracle Workflow
6 7 (1.83%) Apache Solr
6 7 (1.83%) SAS
7 6 (1.57%) Tomcat
8 5 (1.31%) Cloud Foundry
9 4 (1.05%) Confluence
10 3 (0.79%) BizTalk Server
10 3 (0.79%) nginx
10 3 (0.79%) SharePoint
11 1 (0.26%) Jupyter
11 1 (0.26%) OpenStack
Applications
1 2 (0.52%) Microsoft Excel
1 2 (0.52%) MS Visio
Business Applications
1 8 (2.09%) SAP Oil and Gas
2 3 (0.79%) Dynamics AX
2 3 (0.79%) Dynamics CRM
3 1 (0.26%) Markit EDM
3 1 (0.26%) SAP MM
3 1 (0.26%) SAP R/3
Cloud Services
1 115 (30.10%) Amazon AWS
2 55 (14.40%) Microsoft Azure
3 47 (12.30%) Google Cloud Platform
4 33 (8.64%) Amazon EMR
5 30 (7.85%) Amazon EC2
6 23 (6.02%) Amazon Kinesis
7 21 (5.50%) GitHub
8 16 (4.19%) Amazon S3
9 6 (1.57%) IBM Cloud
10 5 (1.31%) Azure Data Factory
11 4 (1.05%) AWS Lambda
11 4 (1.05%) Serverless
12 2 (0.52%) Cloud Computing
12 2 (0.52%) PaaS
12 2 (0.52%) Virtual Private Cloud
13 1 (0.26%) AWS CloudFormation
13 1 (0.26%) AWS Elastic Beanstalk
13 1 (0.26%) Azure Machine Learning
13 1 (0.26%) OpenShift
13 1 (0.26%) SaaS
Communications & Networking
1 21 (5.50%) Kerberos
2 5 (1.31%) Firewall
2 5 (1.31%) Internet
2 5 (1.31%) TCP/IP
3 4 (1.05%) SSH
4 3 (0.79%) DNS
5 2 (0.52%) Reverse Proxy
5 2 (0.52%) VPN
6 1 (0.26%) HTTP
6 1 (0.26%) HTTPS
6 1 (0.26%) ICMP
6 1 (0.26%) Intranet
6 1 (0.26%) SSL
Database & Business Intelligence
1 382 (100.00%) Big Data
2 297 (77.75%) Hadoop
3 165 (43.19%) Apache Hive
4 82 (21.47%) NoSQL
5 79 (20.68%) HBase
6 70 (18.32%) Data Warehouse
7 57 (14.92%) SQL Server
8 50 (13.09%) Relational Database
9 42 (10.99%) Apache Cassandra
10 36 (9.42%) OLAP
10 36 (9.42%) Star Schema
11 35 (9.16%) PostgreSQL
12 34 (8.90%) Enterprise Data Warehouse
13 33 (8.64%) OLAP Cube
14 32 (8.38%) Data Lake
15 29 (7.59%) MongoDB
15 29 (7.59%) Sqoop
16 28 (7.33%) Amazon Redshift
17 25 (6.54%) RDBMS
18 24 (6.28%) Amazon RDS
Development Applications
1 63 (16.49%) Git (software)
2 54 (14.14%) Jenkins
3 37 (9.69%) Maven
4 23 (6.02%) JIRA
4 23 (6.02%) Visual Studio Team System
5 16 (4.19%) Artifactory
6 14 (3.66%) Bitbucket
7 12 (3.14%) Sonatype Nexus
8 11 (2.88%) Cucumber
9 8 (2.09%) Eclipse
9 8 (2.09%) IntelliJ
10 6 (1.57%) JUnit
11 5 (1.31%) Ant
11 5 (1.31%) Selenium
11 5 (1.31%) Visual Studio
12 4 (1.05%) CruiseControl
12 4 (1.05%) IBM UrbanCode
12 4 (1.05%) Subversion
13 3 (0.79%) gulp
13 3 (0.79%) Protractor
General
1 118 (30.89%) Finance
2 19 (4.97%) Banking
3 16 (4.19%) Legal
4 12 (3.14%) Telecoms
5 11 (2.88%) Publishing
6 10 (2.62%) Retail
7 7 (1.83%) Advertising
8 6 (1.57%) Marketing
9 3 (0.79%) Games
9 3 (0.79%) Manufacturing
10 2 (0.52%) Retail Banking
11 1 (0.26%) French Language
11 1 (0.26%) Front Office
11 1 (0.26%) Investment Banking
Libraries, Frameworks & Software Standards
1 193 (50.52%) Kafka
2 35 (9.16%) AngularJS
3 26 (6.81%) Spring
4 25 (6.54%) Spark Streaming
5 22 (5.76%) RESTful
6 19 (4.97%) Elastic Stack
7 18 (4.71%) LDAP
7 18 (4.71%) OpenCV
8 16 (4.19%) MLlib
8 16 (4.19%) Unity (Game Engine)
9 15 (3.93%) Spring Boot
10 10 (2.62%) REST
11 7 (1.83%) .NET
11 7 (1.83%) RabbitMQ
12 6 (1.57%) CSS
12 6 (1.57%) Node.js
13 5 (1.31%) HTML
13 5 (1.31%) HTML5
13 5 (1.31%) J2EE
13 5 (1.31%) LAMP
Miscellaneous
1 34 (8.90%) Algorithms
2 25 (6.54%) Analytical Skills
3 20 (5.24%) Greenfield Project
4 17 (4.45%) Computer Science
5 16 (4.19%) HoloLens
6 8 (2.09%) Data Structures
7 7 (1.83%) Virtual Team
8 4 (1.05%) Fintech
8 4 (1.05%) Linux Command Line
9 3 (0.79%) Management Information System
9 3 (0.79%) Mobile App
9 3 (0.79%) NHS
9 3 (0.79%) Self-Motivation
10 2 (0.52%) Cyber Defence
10 2 (0.52%) iPad
10 2 (0.52%) RIA
10 2 (0.52%) User Experience
10 2 (0.52%) Wiki
11 1 (0.26%) Internet of Things
11 1 (0.26%) Public Cloud
Operating Systems
1 84 (21.99%) Linux
2 52 (13.61%) Unix
3 13 (3.40%) Ubuntu
4 8 (2.09%) Red Hat Enterprise Linux
5 6 (1.57%) Windows
6 5 (1.31%) CentOS
7 2 (0.52%) Android
7 2 (0.52%) Apple iOS
8 1 (0.26%) CoreOS
8 1 (0.26%) DC/OS
8 1 (0.26%) Mesosphere
8 1 (0.26%) Windows Server
8 1 (0.26%) Windows Server 2008
Processes & Methodologies
1 135 (35.34%) Analytics
2 122 (31.94%) Agile Software Development
3 120 (31.41%) Data Science
4 90 (23.56%) ETL
5 71 (18.59%) DevOps
6 64 (16.75%) Scrum
7 61 (15.97%) Machine Learning
8 54 (14.14%) Data Modelling
9 47 (12.30%) Data Engineering
9 47 (12.30%) Open Source
10 43 (11.26%) Software Engineering
11 40 (10.47%) OO
12 34 (8.90%) MapReduce
13 33 (8.64%) SQL Optimisation
13 33 (8.64%) TDD
14 30 (7.85%) Data Analytics
15 27 (7.07%) Business Intelligence
15 27 (7.07%) Test Automation
16 23 (6.02%) Deep Learning
17 22 (5.76%) Visualisation
Programming Languages
1 192 (50.26%) Java
2 179 (46.86%) Python
3 149 (39.01%) SQL
4 147 (38.48%) Scala
5 29 (7.59%) C++
5 29 (7.59%) Java 8
6 23 (6.02%) JavaScript
7 22 (5.76%) R
8 14 (3.66%) Bash Shell
8 14 (3.66%) C#
8 14 (3.66%) MATLAB
9 13 (3.40%) Shell Script
10 8 (2.09%) T-SQL
11 6 (1.57%) Groovy
11 6 (1.57%) PL/SQL
11 6 (1.57%) Ruby
12 5 (1.31%) Perl
13 4 (1.05%) PowerShell
14 3 (0.79%) AWK
14 3 (0.79%) HiveQL
Qualifications
1 16 (4.19%) SC Cleared
1 16 (4.19%) Security Cleared
2 13 (3.40%) Degree
3 8 (2.09%) Computer Science Degree
4 3 (0.79%) PhD
5 2 (0.52%) Master's Degree
6 1 (0.26%) CCA
6 1 (0.26%) CISMP
6 1 (0.26%) CISSP
6 1 (0.26%) MCSA
6 1 (0.26%) MCSE
6 1 (0.26%) Microsoft Certification
6 1 (0.26%) Numerate Degree
Quality Assurance & Compliance
1 46 (12.04%) Data Quality
2 8 (2.09%) GDPR
3 5 (1.31%) QA
4 1 (0.26%) ISO/IEC 27001
4 1 (0.26%) PMO
System Software
1 51 (13.35%) HDFS
2 46 (12.04%) Docker
3 23 (6.02%) Apache Flume
4 8 (2.09%) Apache ZooKeeper
5 4 (1.05%) Active Directory
6 3 (0.79%) Virtual Servers
Systems Management
1 61 (15.97%) Oozie
2 51 (13.35%) Ansible
3 32 (8.38%) Hadoop YARN
4 20 (5.24%) Kubernetes
5 19 (4.97%) Kibana
6 13 (3.40%) logstash
6 13 (3.40%) Terraform
7 11 (2.88%) Puppet
8 8 (2.09%) Ambari
8 8 (2.09%) Opscode Chef
9 3 (0.79%) Mesos
9 3 (0.79%) Prometheus
10 2 (0.52%) HAProxy
10 2 (0.52%) Host Intrusion Detection System
10 2 (0.52%) Network Intrusion Detection System
10 2 (0.52%) Packer
10 2 (0.52%) Sysdig
11 1 (0.26%) Consul
11 1 (0.26%) Grafana
11 1 (0.26%) Salt
Vendors
1 41 (10.73%) Cloudera
2 39 (10.21%) Google
3 37 (9.69%) Oracle
4 29 (7.59%) Hortonworks
5 21 (5.50%) Microsoft
6 19 (4.97%) Tableau
7 18 (4.71%) Informatica
8 17 (4.45%) Splunk
9 15 (3.93%) Alteryx
10 14 (3.66%) Centrify
10 14 (3.66%) Red Hat
10 14 (3.66%) Talend
11 13 (3.40%) MapR
12 11 (2.88%) SAP
13 10 (2.62%) IBM
14 7 (1.83%) Databricks
15 4 (1.05%) Ab Initio
16 2 (0.52%) Cognos
16 2 (0.52%) Salesforce.com
16 2 (0.52%) Teradata