Period
to 17 June 2018

The median contractor rate for a Big Data Engineer was £550 per day in advertised job vacancies in England during the 6 months to 17 June 2018.

The first table below provides contractor rate benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Big Data Engineer
UK > England
6 months to
17 Jun 2018
Same period 2017 Same period 2016
Rank 495 587 741
Rank change year-on-year +92 +154 +57
Contract jobs requiring a Big Data Engineer 365 224 113
As % of all contract IT jobs advertised in England 0.40% 0.27% 0.12%
As % of the Job Titles category 0.42% 0.28% 0.12%
Number of daily rates quoted 272 183 91
England median daily rate £550 £550 £600
Median daily rate % change year-on-year - -8.33% +9.09%
10th Percentile £416 £415 £465
90th Percentile £675 £675 £800
UK median daily rate £550 £550 £600
% change year-on-year - -8.33% +9.09%
Number of hourly rates quoted 1 0 0
England median hourly rate £15.50 - -
UK median hourly rate £15.50 - -

The following table is for comparison with the above and includes summary statistics for all contract IT job vacancies advertised in England. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of contract jobs in our overall sample.

All Contract IT Job Vacancies
England
Contract vacancies in England with a recognized job title 86,079 79,573 93,343
% of contract IT jobs with a recognized job title 94.80% 94.97% 95.23%
Number of daily rates quoted 56,831 51,854 61,353
England median daily rate £450 £434 £425
Median daily rate % change year-on-year +3.75% +2.06% +6.25%
10th Percentile £275 £263 £258
90th Percentile £638 £625 £600
UK median daily rate £450 £425 £420
% change year-on-year +5.88% +1.19% +5.00%
Number of hourly rates quoted 2,775 2,619 2,837
England median hourly rate £18.60 £18.75 £20.00
Median hourly rate % change year-on-year -0.80% -6.25% +14.29%
10th Percentile £11.11 £10.75 £10.50
90th Percentile £48.75 £48.75 £47.91
UK median hourly rate £18.00 £18.96 £20.00
% change year-on-year -5.06% -5.20% +14.29%

Big Data Engineer
Job Vacancy Trend in England

Job postings that featured Big Data Engineer in the job title as a percentage of all IT jobs advertised in England.

Job vacancy trend for Big Data Engineer in England

Big Data Engineer
Contractor Daily Rate Trend in England

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Big Data Engineer in England.

Contractor daily rate trend for Big Data Engineer in England

Big Data Engineer
Contractor Daily Rate Histogram in England

The daily rate distribution of IT jobs citing Big Data Engineer in England over the 6 months to 17 June 2018.

Contractor daily rate histogram for Big Data Engineer in England

Big Data Engineer
Contractor Hourly Rate Trend in England

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing Big Data Engineer in England.

Contractor hourly rate trend for Big Data Engineer in England

Big Data Engineer
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Big Data Engineer within the England region over the 6 months to 17 June 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London +16 274 £563 +2.27% 46
South East +16 39 £481 -1.28% 11
East of England +27 22 £475 -11.63% 1
North of England -16 18 £538 +2.38% 2
Yorkshire - 12 £525 - 1
Midlands - 9 £466 - 3
East Midlands - 8 £466 - 2
North West -22 6 £575 +9.52% 1
South West +8 3 £525 +2.44% 2
West Midlands - 1 - - 1
Big Data Engineer
UK

Big Data Engineer Skill Set
Top 30 Co-occurring IT Skills in England

For the 6 months to 17 June 2018, Big Data Engineer contract jobs required the following IT skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region featuring Big Data Engineer in the job title.

1 364 (99.73%) Big Data
2 273 (74.79%) Hadoop
3 263 (72.05%) Apache Spark
4 179 (49.04%) Python
5 175 (47.95%) Java
6 162 (44.38%) Scala
7 160 (43.84%) Kafka
8 156 (42.74%) Apache Hive
9 145 (39.73%) SQL
10 122 (33.42%) Analytics
11 113 (30.96%) Agile Software Development
12 111 (30.41%) Finance
13 110 (30.14%) ETL
14 104 (28.49%) Amazon AWS
15 97 (26.58%) Data Science
16 88 (24.11%) NoSQL
16 88 (24.11%) Linux
17 72 (19.73%) DevOps
18 65 (17.81%) Data Warehouse
19 63 (17.26%) HBase
20 60 (16.44%) Scrum
21 58 (15.89%) Cloudera
21 58 (15.89%) Data Engineering
22 56 (15.34%) Machine Learning
22 56 (15.34%) HDFS
23 52 (14.25%) Apache
24 50 (13.70%) Data Modelling
24 50 (13.70%) Jenkins
25 47 (12.88%) SQL Server
26 46 (12.60%) Apache Cassandra

Big Data Engineer Skill Set
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 263 (72.05%) Apache Spark
2 52 (14.25%) Apache
3 37 (10.14%) Elasticsearch
4 19 (5.21%) Apache Pig
5 13 (3.56%) nginx
6 12 (3.29%) Tomcat
7 10 (2.74%) Apache Airflow
8 9 (2.47%) Luigi
9 8 (2.19%) Cloud Foundry
9 8 (2.19%) Oracle Workflow
10 6 (1.64%) Apache Solr
11 2 (0.55%) Jupyter
11 2 (0.55%) SAS
11 2 (0.55%) SharePoint
12 1 (0.27%) BizTalk Server
12 1 (0.27%) OpenStack
Business Applications
1 4 (1.10%) SAP Oil and Gas
2 1 (0.27%) Dynamics AX
2 1 (0.27%) Dynamics CRM
2 1 (0.27%) Markit EDM
Cloud Services
1 104 (28.49%) Amazon AWS
2 43 (11.78%) Microsoft Azure
3 25 (6.85%) Amazon Kinesis
4 23 (6.30%) Google Cloud Platform
5 22 (6.03%) Amazon EMR
6 20 (5.48%) Amazon EC2
7 11 (3.01%) GitHub
8 10 (2.74%) Amazon S3
9 7 (1.92%) PaaS
10 5 (1.37%) Serverless
11 4 (1.10%) AWS Lambda
12 2 (0.55%) Azure Data Factory
13 1 (0.27%) Amazon SQS
13 1 (0.27%) AWS CloudFormation
13 1 (0.27%) Bluemix
13 1 (0.27%) Data Lake Analytics
13 1 (0.27%) Google Compute Engine
13 1 (0.27%) IBM Cloud
13 1 (0.27%) Office 365
13 1 (0.27%) OpenShift
Communications & Networking
1 19 (5.21%) Kerberos
2 6 (1.64%) DNS
2 6 (1.64%) ICMP
2 6 (1.64%) TCP/IP
2 6 (1.64%) VPN
3 4 (1.10%) DSL
4 1 (0.27%) Reverse Proxy
Database & Business Intelligence
1 364 (99.73%) Big Data
2 273 (74.79%) Hadoop
3 156 (42.74%) Apache Hive
4 88 (24.11%) NoSQL
5 65 (17.81%) Data Warehouse
6 63 (17.26%) HBase
7 47 (12.88%) SQL Server
8 46 (12.60%) Apache Cassandra
9 44 (12.05%) Relational Database
10 42 (11.51%) PostgreSQL
11 35 (9.59%) Impala
12 34 (9.32%) Amazon Redshift
13 32 (8.77%) MongoDB
14 29 (7.95%) Star Schema
15 27 (7.40%) Data Lake
15 27 (7.40%) MySQL
16 25 (6.85%) OLAP
17 23 (6.30%) Enterprise Data Warehouse
17 23 (6.30%) RDBMS
18 22 (6.03%) OLAP Cube
Development Applications
1 50 (13.70%) Jenkins
2 42 (11.51%) Git (software)
3 31 (8.49%) Maven
4 23 (6.30%) Visual Studio Team System
5 15 (4.11%) Artifactory
5 15 (4.11%) Cucumber
6 12 (3.29%) Subversion
7 11 (3.01%) Sonatype Nexus
8 8 (2.19%) Eclipse
8 8 (2.19%) gulp
8 8 (2.19%) Protractor
8 8 (2.19%) Selenium
9 7 (1.92%) Bitbucket
9 7 (1.92%) Visual Studio
10 6 (1.64%) IntelliJ
10 6 (1.64%) SonarQube
11 5 (1.37%) GitLab
11 5 (1.37%) JIRA
12 3 (0.82%) Team Foundation Server
12 3 (0.82%) TeamCity
General
1 111 (30.41%) Finance
2 19 (5.21%) Banking
3 11 (3.01%) Games
3 11 (3.01%) Marketing
4 8 (2.19%) Retail
5 6 (1.64%) Advertising
5 6 (1.64%) Manufacturing
6 5 (1.37%) Telecoms
7 4 (1.10%) Investment Banking
8 1 (0.27%) Legal
8 1 (0.27%) Publishing
Libraries, Frameworks & Software Standards
1 160 (43.84%) Kafka
2 37 (10.14%) AngularJS
3 22 (6.03%) MLlib
4 18 (4.93%) Spark Streaming
5 16 (4.38%) LDAP
5 16 (4.38%) OpenCV
5 16 (4.38%) Unity (Game Engine)
6 15 (4.11%) Spring
7 13 (3.56%) Spring Boot
8 12 (3.29%) Elastic Stack
9 11 (3.01%) LEMP Stack
9 11 (3.01%) RESTful
10 10 (2.74%) Node.js
11 9 (2.47%) JSON
11 9 (2.47%) RabbitMQ
11 9 (2.47%) TensorFlow
11 9 (2.47%) Web Services
12 8 (2.19%) WireMock
13 7 (1.92%) Samza
13 7 (1.92%) Theano
Miscellaneous
1 29 (7.95%) Algorithms
2 27 (7.40%) Analytical Skills
3 18 (4.93%) Computer Science
4 16 (4.38%) HoloLens
5 12 (3.29%) Data Structures
6 10 (2.74%) Greenfield Project
7 7 (1.92%) Public Cloud
8 5 (1.37%) Fintech
8 5 (1.37%) Grid Computing
9 3 (0.82%) Linux Command Line
9 3 (0.82%) NHS
10 2 (0.55%) Data Centre
10 2 (0.55%) Derivative
10 2 (0.55%) Enterprise Software
10 2 (0.55%) Equity Derivative
10 2 (0.55%) Equity Risk
10 2 (0.55%) User Experience
10 2 (0.55%) Virtual Team
11 1 (0.27%) Clustering
11 1 (0.27%) Mobile App
Operating Systems
1 88 (24.11%) Linux
2 33 (9.04%) Unix
3 13 (3.56%) Red Hat Enterprise Linux
4 7 (1.92%) Ubuntu
5 6 (1.64%) Windows
6 5 (1.37%) Windows Server
6 5 (1.37%) Windows Server 2008
7 3 (0.82%) CentOS
8 1 (0.27%) CoreOS
8 1 (0.27%) DC/OS
Processes & Methodologies
1 122 (33.42%) Analytics
2 113 (30.96%) Agile Software Development
3 110 (30.14%) ETL
4 97 (26.58%) Data Science
5 72 (19.73%) DevOps
6 60 (16.44%) Scrum
7 58 (15.89%) Data Engineering
8 56 (15.34%) Machine Learning
9 50 (13.70%) Data Modelling
10 45 (12.33%) Software Engineering
11 42 (11.51%) Open Source
12 32 (8.77%) OO
13 28 (7.67%) Problem-Solving
14 26 (7.12%) Data Analytics
14 26 (7.12%) Test Automation
15 25 (6.85%) Continuous Integration
16 22 (6.03%) Deep Learning
16 22 (6.03%) MapReduce
16 22 (6.03%) SQL Optimisation
17 21 (5.75%) Technical Architecture
Programming Languages
1 179 (49.04%) Python
2 175 (47.95%) Java
3 162 (44.38%) Scala
4 145 (39.73%) SQL
5 28 (7.67%) Java 8
6 23 (6.30%) C++
7 19 (5.21%) JavaScript
8 14 (3.84%) Ruby
8 14 (3.84%) Shell Script
9 13 (3.56%) Bash Shell
10 12 (3.29%) R
11 11 (3.01%) Perl
12 9 (2.47%) T-SQL
13 7 (1.92%) C
13 7 (1.92%) Go
13 7 (1.92%) HiveQL
14 5 (1.37%) PL/SQL
15 4 (1.10%) C#
15 4 (1.10%) Groovy
16 2 (0.55%) PowerShell
Qualifications
1 27 (7.40%) Security Cleared
2 23 (6.30%) SC Cleared
3 18 (4.93%) Degree
4 9 (2.47%) Computer Science Degree
4 9 (2.47%) DV Cleared
5 6 (1.64%) Master's Degree
6 5 (1.37%) Numerate Degree
7 3 (0.82%) PhD
Quality Assurance & Compliance
1 38 (10.41%) Data Quality
2 11 (3.01%) GDPR
3 8 (2.19%) QA
4 4 (1.10%) GCP
5 2 (0.55%) PMO
System Software
1 56 (15.34%) HDFS
2 46 (12.60%) Docker
3 25 (6.85%) Apache Flume
4 8 (2.19%) Virtual Servers
5 6 (1.64%) Apache ZooKeeper
6 1 (0.27%) Active Directory
Systems Management
1 44 (12.05%) Oozie
2 34 (9.32%) Ansible
3 26 (7.12%) Hadoop YARN
4 21 (5.75%) Puppet
5 20 (5.48%) Kubernetes
6 15 (4.11%) Opscode Chef
7 12 (3.29%) Ambari
7 12 (3.29%) Terraform
8 11 (3.01%) Kibana
9 7 (1.92%) Host Intrusion Detection System
9 7 (1.92%) Network Intrusion Detection System
9 7 (1.92%) Prometheus
9 7 (1.92%) Sysdig
10 6 (1.64%) Consul
10 6 (1.64%) HAProxy
10 6 (1.64%) Salt
11 5 (1.37%) logstash
12 2 (0.55%) Packer
13 1 (0.27%) Mesos
13 1 (0.27%) Nagios
Vendors
1 58 (15.89%) Cloudera
2 31 (8.49%) Oracle
3 28 (7.67%) Hortonworks
4 23 (6.30%) Informatica
4 23 (6.30%) Microsoft
5 21 (5.75%) Talend
6 20 (5.48%) Google
7 15 (4.11%) MapR
8 14 (3.84%) Splunk
9 13 (3.56%) Tableau
10 11 (3.01%) Alteryx
11 8 (2.19%) Centrify
12 7 (1.92%) Red Hat
12 7 (1.92%) SaltStack
13 5 (1.37%) SAP
14 4 (1.10%) Ab Initio
14 4 (1.10%) IBM
14 4 (1.10%) Salesforce.com
15 2 (0.55%) Atlassian
15 2 (0.55%) Cadence