Period
to 11 December 2017

The median contractor rate for a Big Data Engineer was £500 per day in advertised job vacancies in England during the 6 months to 11 December 2017.

The first table below provides contractor rate benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Big Data Engineer
UK > England
6 months to
11 Dec 2017
Same period 2016 Same period 2015
Rank 533 601 807
Rank change year-on-year +68 +206 -19
Contract jobs requiring a Big Data Engineer 333 265 63
As % of all contract IT jobs advertised in England 0.35% 0.28% 0.061%
As % of the Job Titles category 0.37% 0.29% 0.064%
Number of daily rates quoted 264 178 51
England median daily rate £500 £550 £550
Median daily rate % change year-on-year -9.09% - +10.00%
10th Percentile £411 £419 £438
90th Percentile £639 £716 £675
UK median daily rate £500 £550 £550
% change year-on-year -9.09% - +10.00%
Number of hourly rates quoted 1 1 1
England median hourly rate £27.50 £100.00 £37.50
Median hourly rate % change year-on-year -72.50% +166.67% -
10th Percentile £26.25 £90.00 £36.25
90th Percentile £28.75 £110.00 £38.75
UK median hourly rate £27.50 £100.00 £37.50
% change year-on-year -72.50% +166.67% -

The following table is for comparison with the above and includes summary statistics for all contract IT job vacancies advertised in England. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of contract jobs in our overall sample.

All Contract IT Job Vacancies
England
Contract vacancies in England with a recognized job title 90105 90412 98619
% of contract IT jobs with a recognized job title 94.81% 94.71% 95.05%
Number of daily rates quoted 60301 59526 63905
England median daily rate £450 £425 £413
Median daily rate % change year-on-year +5.88% +3.03% +3.13%
10th Percentile £263 £255 £250
90th Percentile £638 £600 £600
UK median daily rate £450 £425 £405
% change year-on-year +5.88% +4.94% +1.25%
Number of hourly rates quoted 2613 2991 3781
England median hourly rate £18.70 £18.38 £18.00
Median hourly rate % change year-on-year +1.77% +2.08% +9.09%
10th Percentile £10.82 £10.50 £10.50
90th Percentile £47.83 £47.00 £45.00
UK median hourly rate £18.70 £18.00 £18.00
% change year-on-year +3.89% - +9.09%

Big Data Engineer
Job Vacancy Trend in England

Job postings that featured Big Data Engineer in the job title as a percentage of all IT jobs advertised in England.

Job vacancy trend for Big Data Engineer in England

Big Data Engineer
Contractor Daily Rate Trend in England

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Big Data Engineer in England.

Contractor daily rate trend for Big Data Engineer in England

Big Data Engineer
Contractor Daily Rate Histogram in England

The daily rate distribution of IT jobs citing Big Data Engineer in England over the 6 months to 11 December 2017.

Contractor daily rate histogram for Big Data Engineer in England

Big Data Engineer
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Big Data Engineer within the England region over the 6 months to 11 December 2017. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London -6 247 £525 -6.25% 26
South East +38 58 £500 +17.65% 3
North of England +44 10 £488 -11.36% 2
East of England -23 9 £488 +14.71% 2
West Midlands - 7 £450 -
Midlands - 7 £450 -
North West +11 5 £488 -11.36% 2
Yorkshire - 5 £475 -
South West -4 2 £631 +20.24%
Big Data Engineer
UK

Big Data Engineer Skill Set
Top 30 Co-occurring IT Skills in England

For the 6 months to 11 December 2017, Big Data Engineer contract jobs required the following IT skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all contract ads across the England region featuring Big Data Engineer in the job title.

1 333 (100.00%) Big Data
2 298 (89.49%) Apache Spark
3 270 (81.08%) Hadoop
4 179 (53.75%) Java
5 177 (53.15%) Python
6 156 (46.85%) Scala
7 141 (42.34%) Apache Hive
8 138 (41.44%) Kafka
9 102 (30.63%) HDFS
10 96 (28.83%) Amazon AWS
11 92 (27.63%) Agile Software Development
12 90 (27.03%) Analytics
12 90 (27.03%) HBase
13 73 (21.92%) ETL
14 71 (21.32%) TDD
15 70 (21.02%) Docker
16 67 (20.12%) Oozie
17 66 (19.82%) SQL
18 64 (19.22%) Apache Flume
19 62 (18.62%) PostgreSQL
20 61 (18.32%) Apache
20 61 (18.32%) NoSQL
20 61 (18.32%) Data Engineering
21 60 (18.02%) MapReduce
21 60 (18.02%) Scrum
22 57 (17.12%) DevOps
23 53 (15.92%) Software Engineering
24 52 (15.62%) Cloudera
24 52 (15.62%) Impala
24 52 (15.62%) Sqoop

Big Data Engineer Skill Set
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 298 (89.49%) Apache Spark
2 61 (18.32%) Apache
3 36 (10.81%) Apache Solr
4 26 (7.81%) Elasticsearch
5 24 (7.21%) Apache Pig
6 21 (6.31%) Confluence
7 5 (1.50%) Tomcat
8 4 (1.20%) nginx
9 3 (0.90%) GeoServer
10 1 (0.30%) Apache Airflow
10 1 (0.30%) Jupyter
10 1 (0.30%) OpenStack
Applications
1 1 (0.30%) Microsoft PowerPoint
Cloud Services
1 96 (28.83%) Amazon AWS
2 30 (9.01%) GitHub
3 24 (7.21%) Google Cloud Platform
4 20 (6.01%) Bluemix
4 20 (6.01%) IBM Cloud
5 19 (5.71%) Microsoft Azure
6 14 (4.20%) Amazon EMR
7 12 (3.60%) Amazon EC2
7 12 (3.60%) Amazon S3
8 8 (2.40%) Google Analytics
8 8 (2.40%) Virtual Private Cloud
9 5 (1.50%) Amazon Glacier
9 5 (1.50%) Amazon Kinesis
10 4 (1.20%) AWS Data Pipeline
10 4 (1.20%) AWS Lambda
10 4 (1.20%) Azure Data Factory
10 4 (1.20%) Serverless
11 3 (0.90%) Cloud Computing
12 2 (0.60%) Amazon SQS
12 2 (0.60%) PaaS
Communications & Networking
1 41 (12.31%) Kerberos
2 5 (1.50%) LDAP
3 2 (0.60%) Network Security
3 2 (0.60%) TCP/IP
4 1 (0.30%) Internet
4 1 (0.30%) Multicast
4 1 (0.30%) SSL
Database & Business Intelligence
1 333 (100.00%) Big Data
2 270 (81.08%) Hadoop
3 141 (42.34%) Apache Hive
4 90 (27.03%) HBase
5 62 (18.62%) PostgreSQL
6 61 (18.32%) NoSQL
7 52 (15.62%) Impala
7 52 (15.62%) Sqoop
8 47 (14.11%) Apache Cassandra
9 45 (13.51%) Data Warehouse
10 30 (9.01%) MongoDB
11 28 (8.41%) SQL Server
12 20 (6.01%) SQL Server 2008
13 17 (5.11%) Amazon Redshift
13 17 (5.11%) Flink
14 16 (4.80%) Data Lake
14 16 (4.80%) Relational Database
15 15 (4.50%) SQL Server Integration Services
16 12 (3.60%) Presto
17 10 (3.00%) RDBMS
Development Applications
1 28 (8.41%) Jenkins
2 27 (8.11%) Cucumber
2 27 (8.11%) JIRA
3 14 (4.20%) Git (software)
4 9 (2.70%) Talend Open Studio
5 4 (1.20%) GitLab
5 4 (1.20%) IntelliJ
5 4 (1.20%) Subversion
6 2 (0.60%) ERwin
6 2 (0.60%) Vagrant
6 2 (0.60%) Visual Studio
6 2 (0.60%) Visual Studio Team System
7 1 (0.30%) Atlassian Bamboo
7 1 (0.30%) JUnit
7 1 (0.30%) Maven
7 1 (0.30%) Mockito
7 1 (0.30%) Team Foundation Server
General
1 43 (12.91%) Finance
2 21 (6.31%) Publishing
3 20 (6.01%) Marketing
4 14 (4.20%) Banking
4 14 (4.20%) Telecoms
5 9 (2.70%) Electronics
5 9 (2.70%) Manufacturing
6 7 (2.10%) Retail
7 4 (1.20%) Games
8 2 (0.60%) Investment Banking
9 1 (0.30%) Automotive
9 1 (0.30%) Law
9 1 (0.30%) Legal
9 1 (0.30%) Music Industry
9 1 (0.30%) Retail Banking
Libraries, Frameworks & Software Standards
1 138 (41.44%) Kafka
2 41 (12.31%) REST
3 39 (11.71%) Spring
4 36 (10.81%) PySpark
5 32 (9.61%) Spring Boot
6 29 (8.71%) Web Services
7 28 (8.41%) Drools
8 18 (5.41%) Apache Camel
9 16 (4.80%) Spark Streaming
10 13 (3.90%) JSON
11 12 (3.60%) Apache Tez
12 11 (3.30%) Akka
13 8 (2.40%) LAMP
13 8 (2.40%) RESTful
13 8 (2.40%) SOAP
14 5 (1.50%) Apache Drill
14 5 (1.50%) LAPP Stack
14 5 (1.50%) Node.js
15 4 (1.20%) gRPC
15 4 (1.20%) Sensu
Miscellaneous
1 37 (11.11%) Algorithms
2 20 (6.01%) Computer Science
3 17 (5.11%) Greenfield Project
4 10 (3.00%) Analytical Skills
4 10 (3.00%) Semantic Web
5 9 (2.70%) User Experience
6 7 (2.10%) Data Structures
6 7 (2.10%) Fintech
7 4 (1.20%) Online Games
8 2 (0.60%) Data Centre
8 2 (0.60%) Grid Computing
8 2 (0.60%) Linked Data
8 2 (0.60%) Management Information System
8 2 (0.60%) Smart City
8 2 (0.60%) Smart Energy
9 1 (0.30%) Enterprise Software
9 1 (0.30%) Linux Command Line
9 1 (0.30%) Replication
9 1 (0.30%) Video\Live Streaming
9 1 (0.30%) Virtual Team
Operating Systems
1 51 (15.32%) Linux
2 20 (6.01%) Unix
3 10 (3.00%) Red Hat Enterprise Linux
4 4 (1.20%) Ubuntu
5 2 (0.60%) CentOS
6 1 (0.30%) Windows
6 1 (0.30%) Windows Server
6 1 (0.30%) Windows Server 2008
Processes & Methodologies
1 92 (27.63%) Agile Software Development
2 90 (27.03%) Analytics
3 73 (21.92%) ETL
4 71 (21.32%) TDD
5 61 (18.32%) Data Engineering
6 60 (18.02%) MapReduce
6 60 (18.02%) Scrum
7 57 (17.12%) DevOps
8 53 (15.92%) Software Engineering
9 51 (15.32%) Data Architecture
10 45 (13.51%) Open Source
11 44 (13.21%) Machine Learning
12 37 (11.11%) BDD
13 35 (10.51%) Pair Programming
14 34 (10.21%) Microservices
15 31 (9.31%) Business Intelligence
15 31 (9.31%) Data Modelling
16 27 (8.11%) Data Analytics
17 25 (7.51%) Kanban
18 22 (6.61%) Mentoring
Programming Languages
1 179 (53.75%) Java
2 177 (53.15%) Python
3 156 (46.85%) Scala
4 66 (19.82%) SQL
5 20 (6.01%) R
6 14 (4.20%) Shell Script
7 10 (3.00%) JavaScript
8 8 (2.40%) PHP
9 7 (2.10%) Bash Shell
10 5 (1.50%) PL/SQL
11 4 (1.20%) Perl
11 4 (1.20%) SPARQL
12 3 (0.90%) Ruby
13 2 (0.60%) C
13 2 (0.60%) C#
13 2 (0.60%) C++
13 2 (0.60%) PowerShell
14 1 (0.30%) GraphQL
Qualifications
1 23 (6.91%) Security Cleared
2 18 (5.41%) SC Cleared
3 9 (2.70%) Degree
4 5 (1.50%) Computer Science Degree
5 3 (0.90%) Master's Degree
6 2 (0.60%) DBS Check
7 1 (0.30%) Numerate Degree
Quality Assurance & Compliance
1 10 (3.00%) QA
2 9 (2.70%) Data Quality
3 8 (2.40%) Actionable Recommendations
4 4 (1.20%) GCP
5 3 (0.90%) GDPR
6 1 (0.30%) FATCA
6 1 (0.30%) SLA
System Software
1 102 (30.63%) HDFS
2 70 (21.02%) Docker
3 64 (19.22%) Apache Flume
4 30 (9.01%) Apache ZooKeeper
5 1 (0.30%) Active Directory
5 1 (0.30%) Oracle RAC
5 1 (0.30%) vCloud
5 1 (0.30%) VMware Infrastructure
Systems Management
1 67 (20.12%) Oozie
2 36 (10.81%) Ambari
3 34 (10.21%) Hadoop YARN
4 33 (9.91%) Ansible
4 33 (9.91%) Terraform
5 30 (9.01%) Kubernetes
6 22 (6.61%) Mesos
7 9 (2.70%) Puppet
8 7 (2.10%) Opscode Chef
9 4 (1.20%) Salt
10 3 (0.90%) Packer
11 1 (0.30%) Control-M
11 1 (0.30%) Prometheus
Vendors
1 52 (15.62%) Cloudera
2 40 (12.01%) Google
3 33 (9.91%) Hortonworks
4 27 (8.11%) Tableau
5 21 (6.31%) IBM
6 14 (4.20%) Oracle
7 13 (3.90%) MapR
8 12 (3.60%) Microsoft
9 10 (3.00%) SAP
10 9 (2.70%) Talend
11 8 (2.40%) Informatica
11 8 (2.40%) Salesforce.com
11 8 (2.40%) Teradata
12 6 (1.80%) Splunk
13 4 (1.20%) SaltStack
14 2 (0.60%) Alteryx
14 2 (0.60%) Red Hat
15 1 (0.30%) VMware