Period
to 23 February 2018

The median contractor rate for a Big Data Engineer was £525 per day in advertised job vacancies in England during the 6 months to 23 February 2018.

The first table below provides contractor rate benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Big Data Engineer
UK > England
6 months to
23 Feb 2018
Same period 2017 Same period 2016
Rank 532 549 758
Rank change year-on-year +17 +209 +9
Contract jobs requiring a Big Data Engineer 329 289 94
As % of all contract IT jobs advertised in England 0.35% 0.32% 0.095%
As % of the Job Titles category 0.37% 0.34% 0.099%
Number of daily rates quoted 249 209 77
England median daily rate £525 £550 £550
Median daily rate % change year-on-year -4.55% - +4.76%
10th Percentile £413 £425 £433
90th Percentile £650 £675 £675
UK median daily rate £525 £550 £550
% change year-on-year -4.55% - +6.80%
Number of hourly rates quoted 1 1 1
England median hourly rate £27.50 £100.00 £37.50
Median hourly rate % change year-on-year -72.50% +166.67% -
10th Percentile £26.25 £90.00 £36.25
90th Percentile £28.75 £110.00 £38.75
UK median hourly rate £27.50 £100.00 £37.50
% change year-on-year -72.50% +166.67% -

The following table is for comparison with the above and includes summary statistics for all contract IT job vacancies advertised in England. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of contract jobs in our overall sample.

All Contract IT Job Vacancies
England
Contract vacancies in England with a recognized job title 88,262 84,277 94,530
% of contract IT jobs with a recognized job title 94.72% 94.75% 95.14%
Number of daily rates quoted 59,200 55,364 61,413
England median daily rate £450 £425 £420
Median daily rate % change year-on-year +5.88% +1.19% +5.00%
10th Percentile £263 £257 £255
90th Percentile £638 £613 £600
UK median daily rate £450 £425 £413
% change year-on-year +5.88% +3.03% +3.13%
Number of hourly rates quoted 2,607 2,965 3,469
England median hourly rate £18.70 £18.50 £18.50
Median hourly rate % change year-on-year +1.08% - +11.16%
10th Percentile £11.00 £10.63 £10.50
90th Percentile £48.57 £47.50 £47.50
UK median hourly rate £18.70 £18.00 £19.00
% change year-on-year +3.89% -5.26% +15.15%

Big Data Engineer
Job Vacancy Trend in England

Job postings that featured Big Data Engineer in the job title as a percentage of all IT jobs advertised in England.

Job vacancy trend for Big Data Engineer in England

Big Data Engineer
Contractor Daily Rate Trend in England

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Big Data Engineer in England.

Contractor daily rate trend for Big Data Engineer in England

Big Data Engineer
Contractor Daily Rate Histogram in England

The daily rate distribution of IT jobs citing Big Data Engineer in England over the 6 months to 23 February 2018.

Contractor daily rate histogram for Big Data Engineer in England

Big Data Engineer
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Big Data Engineer within the England region over the 6 months to 23 February 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London -43 233 £550 - 46
South East +41 59 £500 - 12
North of England +32 16 £525 - 6
Midlands - 12 £450 - 1
North West -1 8 £519 -1.19% 2
East of England -5 8 £460 -3.92%
West Midlands - 8 £450 - 1
Yorkshire - 8 £525 - 4
East Midlands - 4 £463 -
South West +4 1 £575 +9.52%
Big Data Engineer
UK

Big Data Engineer Skill Set
Top 30 Co-occurring IT Skills in England

For the 6 months to 23 February 2018, Big Data Engineer contract jobs required the following IT skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region featuring Big Data Engineer in the job title.

1 328 (99.70%) Big Data
2 276 (83.89%) Apache Spark
3 247 (75.08%) Hadoop
4 188 (57.14%) Python
5 159 (48.33%) Scala
6 138 (41.95%) Apache Hive
6 138 (41.95%) Java
7 134 (40.73%) Kafka
8 112 (34.04%) Agile Software Development
9 85 (25.84%) Amazon AWS
10 84 (25.53%) ETL
11 77 (23.40%) Analytics
12 70 (21.28%) SQL
13 68 (20.67%) DevOps
14 66 (20.06%) Docker
15 65 (19.76%) Data Engineering
16 60 (18.24%) Data Science
16 60 (18.24%) HBase
17 57 (17.33%) Scrum
18 56 (17.02%) Apache
18 56 (17.02%) HDFS
19 55 (16.72%) Cloudera
20 53 (16.11%) Apache Flume
21 51 (15.50%) Open Source
22 50 (15.20%) Oozie
23 48 (14.59%) Software Engineering
24 47 (14.29%) NoSQL
24 47 (14.29%) Linux
24 47 (14.29%) Data Warehouse
25 44 (13.37%) MapReduce

Big Data Engineer Skill Set
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 276 (83.89%) Apache Spark
2 56 (17.02%) Apache
3 23 (6.99%) Elasticsearch
4 22 (6.69%) Apache Pig
5 8 (2.43%) Apache Solr
5 8 (2.43%) Tomcat
6 7 (2.13%) nginx
7 4 (1.22%) Cloud Foundry
8 3 (0.91%) GeoServer
9 2 (0.61%) Confluence
9 2 (0.61%) Jupyter
10 1 (0.30%) Apache Airflow
10 1 (0.30%) OpenStack
10 1 (0.30%) SAS
10 1 (0.30%) SharePoint
Applications
1 1 (0.30%) Microsoft PowerPoint
Cloud Services
1 85 (25.84%) Amazon AWS
2 33 (10.03%) Microsoft Azure
3 29 (8.81%) Google Cloud Platform
4 25 (7.60%) GitHub
5 13 (3.95%) Bluemix
5 13 (3.95%) IBM Cloud
6 10 (3.04%) Amazon Kinesis
6 10 (3.04%) Amazon S3
7 8 (2.43%) Amazon EC2
8 7 (2.13%) Serverless
9 6 (1.82%) AWS Lambda
10 5 (1.52%) Azure Data Factory
11 4 (1.22%) Amazon EMR
11 4 (1.22%) AWS Data Pipeline
11 4 (1.22%) Virtual Private Cloud
12 3 (0.91%) Amazon SQS
13 2 (0.61%) AWS CloudFormation
13 2 (0.61%) Data Lake Analytics
13 2 (0.61%) PaaS
14 1 (0.30%) Google Compute Engine
Communications & Networking
1 15 (4.56%) Kerberos
2 4 (1.22%) DSL
2 4 (1.22%) TCP/IP
3 2 (0.61%) DNS
3 2 (0.61%) ICMP
3 2 (0.61%) Network Security
3 2 (0.61%) VPN
4 1 (0.30%) Internet
4 1 (0.30%) Multicast
4 1 (0.30%) SSL
Database & Business Intelligence
1 328 (99.70%) Big Data
2 247 (75.08%) Hadoop
3 138 (41.95%) Apache Hive
4 60 (18.24%) HBase
5 47 (14.29%) Data Warehouse
5 47 (14.29%) NoSQL
6 41 (12.46%) Apache Cassandra
6 41 (12.46%) Impala
7 40 (12.16%) PostgreSQL
8 36 (10.94%) Sqoop
9 31 (9.42%) MongoDB
10 25 (7.60%) SQL Server
11 18 (5.47%) Relational Database
12 16 (4.86%) Data Lake
13 15 (4.56%) Amazon Redshift
13 15 (4.56%) Flink
14 14 (4.26%) MySQL
15 10 (3.04%) BigQuery
15 10 (3.04%) RDBMS
15 10 (3.04%) SQL Server Integration Services
Development Applications
1 42 (12.77%) Jenkins
2 23 (6.99%) Git (software)
3 10 (3.04%) Maven
3 10 (3.04%) Subversion
3 10 (3.04%) Visual Studio Team System
4 8 (2.43%) Cucumber
4 8 (2.43%) JIRA
4 8 (2.43%) Visual Studio
5 5 (1.52%) Sonatype Nexus
6 4 (1.22%) ERwin
6 4 (1.22%) gulp
6 4 (1.22%) Protractor
6 4 (1.22%) Selenium
7 3 (0.91%) GitLab
7 3 (0.91%) JUnit
7 3 (0.91%) TeamCity
8 2 (0.61%) Artifactory
8 2 (0.61%) SonarQube
8 2 (0.61%) Team Foundation Server
8 2 (0.61%) Vagrant
General
1 41 (12.46%) Finance
2 22 (6.69%) Publishing
3 16 (4.86%) Banking
4 10 (3.04%) Games
5 8 (2.43%) Marketing
5 8 (2.43%) Retail
6 5 (1.52%) Investment Banking
6 5 (1.52%) Telecoms
7 3 (0.91%) Manufacturing
8 1 (0.30%) Automotive
8 1 (0.30%) Law
8 1 (0.30%) Legal
8 1 (0.30%) Music Industry
8 1 (0.30%) Retail Banking
Libraries, Frameworks & Software Standards
1 134 (40.73%) Kafka
2 32 (9.73%) PySpark
3 23 (6.99%) Spark Streaming
4 11 (3.34%) TensorFlow
5 10 (3.04%) AngularJS
6 9 (2.74%) Node.js
6 9 (2.74%) Theano
6 9 (2.74%) Velocity
6 9 (2.74%) Web Services
7 8 (2.43%) Apache Tez
7 8 (2.43%) CNTK
7 8 (2.43%) MXNet
7 8 (2.43%) REST
8 7 (2.13%) LDAP
8 7 (2.13%) Spring
9 6 (1.82%) Elastic Stack
10 5 (1.52%) Sensu
10 5 (1.52%) Spring Boot
11 4 (1.22%) RabbitMQ
11 4 (1.22%) RESTful
Miscellaneous
1 24 (7.29%) Algorithms
2 18 (5.47%) Computer Science
3 9 (2.74%) Analytical Skills
3 9 (2.74%) Greenfield Project
4 8 (2.43%) Fintech
5 7 (2.13%) Semantic Web
5 7 (2.13%) User Experience
6 5 (1.52%) Data Structures
7 3 (0.91%) Data Centre
7 3 (0.91%) Online Games
8 2 (0.61%) Enterprise Software
8 2 (0.61%) Public Cloud
8 2 (0.61%) Smart City
8 2 (0.61%) Smart Energy
9 1 (0.30%) Grid Computing
9 1 (0.30%) HoloLens
9 1 (0.30%) Management Information System
9 1 (0.30%) Mobile Games
9 1 (0.30%) Replication
9 1 (0.30%) Video\Live Streaming
Operating Systems
1 47 (14.29%) Linux
2 16 (4.86%) Unix
3 12 (3.65%) Red Hat Enterprise Linux
4 3 (0.91%) Ubuntu
5 2 (0.61%) CentOS
6 1 (0.30%) DC/OS
6 1 (0.30%) Windows
6 1 (0.30%) Windows Server
6 1 (0.30%) Windows Server 2008
Processes & Methodologies
1 112 (34.04%) Agile Software Development
2 84 (25.53%) ETL
3 77 (23.40%) Analytics
4 68 (20.67%) DevOps
5 65 (19.76%) Data Engineering
6 60 (18.24%) Data Science
7 57 (17.33%) Scrum
8 51 (15.50%) Open Source
9 48 (14.59%) Software Engineering
10 44 (13.37%) MapReduce
11 41 (12.46%) TDD
12 40 (12.16%) Machine Learning
13 33 (10.03%) Kanban
14 29 (8.81%) Continuous Delivery
15 28 (8.51%) Continuous Integration
15 28 (8.51%) Data Analytics
16 26 (7.90%) Data Architecture
16 26 (7.90%) Pair Programming
17 24 (7.29%) Data Modelling
18 20 (6.08%) Business Intelligence
Programming Languages
1 188 (57.14%) Python
2 159 (48.33%) Scala
3 138 (41.95%) Java
4 70 (21.28%) SQL
5 19 (5.78%) R
6 18 (5.47%) JavaScript
7 13 (3.95%) Shell Script
8 11 (3.34%) Ruby
9 10 (3.04%) Bash Shell
10 9 (2.74%) Perl
11 8 (2.43%) C
12 5 (1.52%) PL/SQL
13 4 (1.22%) C#
13 4 (1.22%) HiveQL
14 3 (0.91%) C++
15 2 (0.61%) Go
15 2 (0.61%) Groovy
15 2 (0.61%) PowerShell
16 1 (0.30%) Clojure
16 1 (0.30%) Kotlin
Qualifications
1 20 (6.08%) Security Cleared
2 16 (4.86%) Degree
3 15 (4.56%) SC Cleared
4 9 (2.74%) Master's Degree
5 5 (1.52%) Computer Science Degree
5 5 (1.52%) DV Cleared
6 1 (0.30%) Numerate Degree
6 1 (0.30%) PhD
Quality Assurance & Compliance
1 10 (3.04%) Data Quality
2 9 (2.74%) GDPR
3 7 (2.13%) GCP
4 6 (1.82%) QA
5 1 (0.30%) FATCA
System Software
1 66 (20.06%) Docker
2 56 (17.02%) HDFS
3 53 (16.11%) Apache Flume
4 4 (1.22%) Apache ZooKeeper
4 4 (1.22%) Virtual Servers
5 1 (0.30%) vCloud
5 1 (0.30%) VMware Infrastructure
Systems Management
1 50 (15.20%) Oozie
2 33 (10.03%) Hadoop YARN
3 31 (9.42%) Ambari
4 23 (6.99%) Mesos
5 22 (6.69%) Kubernetes
6 19 (5.78%) Ansible
6 19 (5.78%) Puppet
7 14 (4.26%) Opscode Chef
8 13 (3.95%) Terraform
9 5 (1.52%) Kibana
9 5 (1.52%) Salt
10 3 (0.91%) logstash
10 3 (0.91%) Prometheus
11 2 (0.61%) Consul
11 2 (0.61%) HAProxy
11 2 (0.61%) Host Intrusion Detection System
11 2 (0.61%) Network Intrusion Detection System
11 2 (0.61%) Packer
11 2 (0.61%) Sysdig
12 1 (0.30%) Nagios
Vendors
1 55 (16.72%) Cloudera
2 30 (9.12%) Google
3 29 (8.81%) Hortonworks
4 24 (7.29%) Microsoft
5 22 (6.69%) Tableau
6 20 (6.08%) Oracle
7 17 (5.17%) Talend
8 13 (3.95%) IBM
9 11 (3.34%) Informatica
9 11 (3.34%) MapR
10 8 (2.43%) Alteryx
11 7 (2.13%) Splunk
12 5 (1.52%) SaltStack
13 4 (1.22%) Teradata
14 2 (0.61%) Atlassian
14 2 (0.61%) Cadence
14 2 (0.61%) Salesforce.com
15 1 (0.30%) Hitachi
15 1 (0.30%) SAP
15 1 (0.30%) VMware