Period
to 13 December 2018

The median contractor rate for a Big Data Engineer was £550 per day in advertised job vacancies in England during the 6 months to 13 December 2018.

The first table below provides contractor rate benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Big Data Engineer
UK > England
6 months to
13 Dec 2018
Same period 2017 Same period 2016
Rank 523 544 612
Rank change year-on-year +21 +68 +202
Contract jobs requiring a Big Data Engineer 371 333 265
As % of all contract IT jobs advertised in England 0.38% 0.35% 0.28%
As % of the Job Titles category 0.41% 0.37% 0.29%
Number of daily rates quoted 279 264 178
England median daily rate £550 £500 £550
Median daily rate % change year-on-year +10.00% -9.09% -
10th Percentile £450 £411 £419
90th Percentile £735 £639 £716
UK median daily rate £550 £500 £550
% change year-on-year +10.00% -9.09% -
Number of hourly rates quoted 0 1 1
England median hourly rate - £27.50 £100.00
Median hourly rate % change year-on-year - -72.50% +166.67%
UK median hourly rate - £27.50 £100.00
% change year-on-year - -72.50% +166.67%

The following table is for comparison with the above and includes summary statistics for all contract IT job vacancies advertised in England. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of contract jobs in our overall sample.

All Contract IT Job Vacancies
England
Contract vacancies in England with a recognized job title 91,359 90,187 90,477
% of contract IT jobs with a recognized job title 94.75% 94.89% 94.77%
Number of daily rates quoted 61,235 60,341 59,569
England median daily rate £459 £450 £425
Median daily rate % change year-on-year +1.89% +5.88% +3.03%
10th Percentile £275 £263 £255
90th Percentile £638 £638 £600
UK median daily rate £450 £450 £425
% change year-on-year - +5.88% +4.94%
Number of hourly rates quoted 2,773 2,621 2,997
England median hourly rate £18.12 £18.70 £18.38
Median hourly rate % change year-on-year -3.13% +1.77% +2.08%
10th Percentile £11.00 £10.79 £10.50
90th Percentile £48.75 £47.83 £47.00
UK median hourly rate £18.00 £18.70 £18.00
% change year-on-year -3.74% +3.89% -

Big Data Engineer
Job Vacancy Trend in England

Job postings that featured Big Data Engineer in the job title as a percentage of all IT jobs advertised in England.

Job vacancy trend for Big Data Engineer in England

Big Data Engineer
Contractor Daily Rate Trend in England

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Big Data Engineer in England.

Contractor daily rate trend for Big Data Engineer in England

Big Data Engineer
Contractor Daily Rate Histogram in England

The daily rate distribution of IT jobs citing Big Data Engineer in England over the 6 months to 13 December 2018.

Contractor daily rate histogram for Big Data Engineer in England

Big Data Engineer
Contractor Hourly Rate Trend in England

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing Big Data Engineer in England.

Contractor hourly rate trend for Big Data Engineer in England

Big Data Engineer
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Big Data Engineer within the England region over the 6 months to 13 December 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London +23 260 £575 +9.52% 40
South East -11 57 £525 +5.00% 4
North of England +2 23 £530 +8.72% 4
South West -24 14 £525 -16.83% 1
North West -12 13 £530 +8.72% 1
East of England +8 12 £475 -2.56% 2
Yorkshire -8 5 £550 +15.79% 2
North East - 5 £488 - 1
Midlands -13 4 £495 +9.89%
West Midlands -26 2 £495 +9.89%
East Midlands - 2 £500 -
Big Data Engineer
UK

Big Data Engineer Skill Set
Top 30 Co-occurring IT Skills in England

For the 6 months to 13 December 2018, Big Data Engineer contract jobs required the following IT skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region featuring Big Data Engineer in the job title.

1 371 (100.00%) Big Data
2 263 (70.89%) Hadoop
3 245 (66.04%) Apache Spark
4 193 (52.02%) Kafka
4 193 (52.02%) Python
5 186 (50.13%) Java
6 157 (42.32%) Apache Hive
7 126 (33.96%) Scala
7 126 (33.96%) Agile Software Development
8 124 (33.42%) SQL
9 123 (33.15%) Analytics
10 117 (31.54%) Amazon AWS
11 97 (26.15%) Data Science
12 93 (25.07%) HBase
13 90 (24.26%) ETL
14 83 (22.37%) Finance
15 80 (21.56%) NoSQL
16 79 (21.29%) Microsoft Azure
17 74 (19.95%) DevOps
18 69 (18.60%) Apache
19 66 (17.79%) Linux
20 61 (16.44%) Apache Cassandra
20 61 (16.44%) Elasticsearch
21 60 (16.17%) Oozie
21 60 (16.17%) HDFS
22 58 (15.63%) Google Cloud Platform
23 57 (15.36%) Ansible
24 56 (15.09%) Jenkins
25 55 (14.82%) Machine Learning
25 55 (14.82%) Scrum

Big Data Engineer Skill Set
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 245 (66.04%) Apache Spark
2 69 (18.60%) Apache
3 61 (16.44%) Elasticsearch
4 18 (4.85%) Apache Pig
5 15 (4.04%) SAS
6 11 (2.96%) Apache Solr
7 7 (1.89%) Jupyter
8 6 (1.62%) Apache Qpid
9 5 (1.35%) Apache Airflow
9 5 (1.35%) Confluence
10 4 (1.08%) Luigi
11 3 (0.81%) BizTalk Server
11 3 (0.81%) Cloud Foundry
11 3 (0.81%) nginx
11 3 (0.81%) SharePoint
12 2 (0.54%) Oracle Workflow
12 2 (0.54%) Tomcat
Applications
1 3 (0.81%) Microsoft Excel
1 3 (0.81%) MS Visio
2 1 (0.27%) Chatbot
Business Applications
1 13 (3.50%) SAP Oil and Gas
2 3 (0.81%) Dynamics AX
2 3 (0.81%) Dynamics CRM
3 1 (0.27%) SAP MM
3 1 (0.27%) SAP R/3
Cloud Services
1 117 (31.54%) Amazon AWS
2 79 (21.29%) Microsoft Azure
3 58 (15.63%) Google Cloud Platform
4 24 (6.47%) Amazon EMR
5 18 (4.85%) Amazon EC2
6 16 (4.31%) Azure Data Factory
6 16 (4.31%) GitHub
7 15 (4.04%) Amazon S3
8 11 (2.96%) Amazon Kinesis
9 8 (2.16%) Data Lake Analytics
9 8 (2.16%) IaaS
10 6 (1.62%) IBM Cloud
11 5 (1.35%) Cloud Computing
12 4 (1.08%) AWS Lambda
12 4 (1.08%) Cortana Intelligence Suite
12 4 (1.08%) Serverless
13 3 (0.81%) SaaS
13 3 (0.81%) Virtual Private Cloud
14 2 (0.54%) AWS Elastic Beanstalk
14 2 (0.54%) PaaS
Communications & Networking
1 17 (4.58%) Kerberos
2 6 (1.62%) TCP/IP
3 5 (1.35%) Firewall
3 5 (1.35%) Internet
3 5 (1.35%) LAN
3 5 (1.35%) WAN
4 4 (1.08%) DNS
4 4 (1.08%) SSH
5 3 (0.81%) VPN
6 2 (0.54%) ICMP
7 1 (0.27%) DSL
7 1 (0.27%) HTTP
7 1 (0.27%) HTTPS
7 1 (0.27%) Intranet
7 1 (0.27%) Network Security
7 1 (0.27%) Reverse Proxy
7 1 (0.27%) SSL
Database & Business Intelligence
1 371 (100.00%) Big Data
2 263 (70.89%) Hadoop
3 157 (42.32%) Apache Hive
4 93 (25.07%) HBase
5 80 (21.56%) NoSQL
6 61 (16.44%) Apache Cassandra
7 50 (13.48%) Sqoop
8 49 (13.21%) Data Warehouse
9 45 (12.13%) MongoDB
10 44 (11.86%) Data Lake
11 41 (11.05%) SQL Server
12 37 (9.97%) Relational Database
13 28 (7.55%) Impala
14 25 (6.74%) PostgreSQL
15 20 (5.39%) Power BI
15 20 (5.39%) Presto
16 19 (5.12%) Flink
17 18 (4.85%) Cosmos DB
17 18 (4.85%) Redis
18 17 (4.58%) Azure SQL Data Warehouse
Development Applications
1 56 (15.09%) Jenkins
2 50 (13.48%) Git (software)
3 24 (6.47%) JIRA
3 24 (6.47%) Maven
4 13 (3.50%) Visual Studio Team System
5 11 (2.96%) Artifactory
6 10 (2.70%) Cucumber
7 9 (2.43%) Sonatype Nexus
8 8 (2.16%) Bitbucket
9 7 (1.89%) Selenium
10 6 (1.62%) Gerrit
10 6 (1.62%) JBehave
10 6 (1.62%) JUnit
11 5 (1.35%) Ant
11 5 (1.35%) Eclipse
11 5 (1.35%) SonarQube
12 4 (1.08%) CruiseControl
12 4 (1.08%) IntelliJ
12 4 (1.08%) RStudio
12 4 (1.08%) Visual Studio
General
1 83 (22.37%) Finance
2 21 (5.66%) Banking
2 21 (5.66%) Legal
3 18 (4.85%) Retail
3 18 (4.85%) Telecoms
4 12 (3.23%) Publishing
5 6 (1.62%) Marketing
6 3 (0.81%) Advertising
7 2 (0.54%) Front Office
7 2 (0.54%) Investment Banking
7 2 (0.54%) Retail Banking
8 1 (0.27%) Corporate Banking
8 1 (0.27%) Dutch Language
8 1 (0.27%) Financial Institution
8 1 (0.27%) French Language
8 1 (0.27%) Manufacturing
Libraries, Frameworks & Software Standards
1 193 (52.02%) Kafka
2 28 (7.55%) Spark Streaming
3 24 (6.47%) AngularJS
3 24 (6.47%) Elastic Stack
3 24 (6.47%) Spring
4 19 (5.12%) RESTful
5 13 (3.50%) Spring Boot
6 12 (3.23%) PySpark
7 10 (2.70%) Apache Avro
7 10 (2.70%) Apache Camel
7 10 (2.70%) LDAP
7 10 (2.70%) REST
8 9 (2.43%) CSS
8 9 (2.43%) HTML
9 8 (2.16%) .NET
9 8 (2.16%) OpenCV
10 7 (1.89%) GraphQL
10 7 (1.89%) React
10 7 (1.89%) scikit-learn
10 7 (1.89%) XML
Miscellaneous
1 38 (10.24%) Algorithms
2 28 (7.55%) Analytical Skills
3 19 (5.12%) Greenfield Project
4 11 (2.96%) Data Structures
5 6 (1.62%) Data Centre
5 6 (1.62%) HoloLens
5 6 (1.62%) Virtual Team
6 5 (1.35%) Fintech
6 5 (1.35%) Self-Motivation
7 3 (0.81%) iPad
7 3 (0.81%) Replication
7 3 (0.81%) Wiki
8 2 (0.54%) Cyber Defence
8 2 (0.54%) Management Information System
8 2 (0.54%) Mobile App
8 2 (0.54%) Public Cloud
8 2 (0.54%) RIA
9 1 (0.27%) Analytical Mindset
9 1 (0.27%) Internet of Things
9 1 (0.27%) Linux Command Line
Operating Systems
1 66 (17.79%) Linux
2 43 (11.59%) Unix
3 7 (1.89%) Ubuntu
4 6 (1.62%) Windows
5 3 (0.81%) CentOS
5 3 (0.81%) Red Hat Enterprise Linux
6 2 (0.54%) Android
6 2 (0.54%) Apple iOS
7 1 (0.27%) DC/OS
7 1 (0.27%) Mesosphere
Processes & Methodologies
1 126 (33.96%) Agile Software Development
2 123 (33.15%) Analytics
3 97 (26.15%) Data Science
4 90 (24.26%) ETL
5 74 (19.95%) DevOps
6 55 (14.82%) Machine Learning
6 55 (14.82%) Scrum
7 50 (13.48%) MapReduce
8 45 (12.13%) Open Source
8 45 (12.13%) Software Engineering
9 44 (11.86%) Data Modelling
10 43 (11.59%) Data Engineering
11 38 (10.24%) TDD
12 36 (9.70%) Business Intelligence
13 32 (8.63%) Computer Science
14 29 (7.82%) Test Automation
15 27 (7.28%) Visualisation
16 25 (6.74%) Data Analytics
17 24 (6.47%) OO
18 22 (5.93%) Continuous Integration
Programming Languages
1 193 (52.02%) Python
2 186 (50.13%) Java
3 126 (33.96%) Scala
4 124 (33.42%) SQL
5 35 (9.43%) R
6 31 (8.36%) JavaScript
7 26 (7.01%) C++
8 19 (5.12%) Bash Shell
8 19 (5.12%) Java 8
9 18 (4.85%) MATLAB
10 17 (4.58%) C#
11 13 (3.50%) Groovy
11 13 (3.50%) Ruby
12 9 (2.43%) Shell Script
13 6 (1.62%) AWK
13 6 (1.62%) C
13 6 (1.62%) PowerShell
14 4 (1.08%) Go
15 3 (0.81%) ES6
15 3 (0.81%) sed
Qualifications
1 34 (9.16%) Degree
2 23 (6.20%) Security Cleared
3 22 (5.93%) Computer Science Degree
4 21 (5.66%) SC Cleared
5 11 (2.96%) Master's Degree
6 1 (0.27%) CCA
6 1 (0.27%) CISMP
6 1 (0.27%) CISSP
6 1 (0.27%) CREST Certified
6 1 (0.27%) DV Cleared
6 1 (0.27%) MCSA
6 1 (0.27%) MCSE
6 1 (0.27%) Microsoft Certification
6 1 (0.27%) Numerate Degree
6 1 (0.27%) PhD
Quality Assurance & Compliance
1 23 (6.20%) Data Quality
2 7 (1.89%) QA
3 6 (1.62%) GDPR
3 6 (1.62%) SLA
4 1 (0.27%) ISO/IEC 27001
System Software
1 60 (16.17%) HDFS
2 46 (12.40%) Docker
3 38 (10.24%) Apache Flume
4 24 (6.47%) Apache ZooKeeper
5 5 (1.35%) Oracle RAC
6 3 (0.81%) Active Directory
7 1 (0.27%) Virtual Machines
Systems Management
1 60 (16.17%) Oozie
2 57 (15.36%) Ansible
3 51 (13.75%) Hadoop YARN
4 28 (7.55%) Kubernetes
5 17 (4.58%) Kibana
5 17 (4.58%) Terraform
6 15 (4.04%) Ambari
7 14 (3.77%) Puppet
8 12 (3.23%) logstash
9 7 (1.89%) Opscode Chef
10 4 (1.08%) Consul
10 4 (1.08%) Mesos
11 3 (0.81%) Grafana
11 3 (0.81%) Prometheus
12 2 (0.54%) HAProxy
12 2 (0.54%) Host Intrusion Detection System
12 2 (0.54%) Network Intrusion Detection System
12 2 (0.54%) Salt
12 2 (0.54%) Sysdig
13 1 (0.27%) StatsD
Vendors
1 47 (12.67%) Cloudera
2 45 (12.13%) Google
3 43 (11.59%) Oracle
4 36 (9.70%) Hortonworks
5 30 (8.09%) Talend
6 20 (5.39%) Alteryx
6 20 (5.39%) Informatica
6 20 (5.39%) Tableau
7 16 (4.31%) SAP
8 15 (4.04%) Databricks
9 14 (3.77%) Microsoft
10 13 (3.50%) Splunk
11 9 (2.43%) MapR
12 8 (2.16%) Red Hat
13 7 (1.89%) Centrify
13 7 (1.89%) IBM
14 5 (1.35%) Cisco
15 4 (1.08%) Teradata
16 2 (0.54%) Cognos
16 2 (0.54%) New Relic