Period
to 20 March 2019

The median contractor rate for a Big Data Engineer was £550 per day in advertised job vacancies in London during the 6 months to 20 March 2019.

The first table below provides contractor rate benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Big Data Engineer
England > London
6 months to
20 Mar 2019
Same period 2018 Same period 2017
Rank 358 400 372
Rank change year-on-year +42 -28 +143
Contract jobs requiring a Big Data Engineer 261 242 230
As % of all contract IT jobs advertised in London 0.61% 0.52% 0.57%
As % of the Job Titles category 0.65% 0.55% 0.60%
Number of daily rates quoted 195 189 171
London median daily rate £550 £550 £550
10th Percentile £470 £450 £425
90th Percentile £724 £675 £675
England median daily rate £525 £550 £550
% change year-on-year -4.55% - -8.33%
Number of hourly rates quoted 0 0 1
London median hourly rate - - £100.00
England median hourly rate - £27.50 £100.00
% change year-on-year - -72.50% -

The following table is for comparison with the above and includes summary statistics for all contract IT job vacancies advertised in London. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of contract jobs in our overall sample.

All Contract IT Job Vacancies
London
Contract vacancies in London with a recognized job title 40,456 44,137 38,289
% of contract IT jobs with a recognized job title 94.64% 94.86% 94.98%
Number of daily rates quoted 29,024 32,018 27,260
London median daily rate £500 £500 £475
Median daily rate % change year-on-year - +5.26% +5.56%
10th Percentile £313 £305 £300
90th Percentile £675 £675 £650
England median daily rate £475 £450 £425
% change year-on-year +5.56% +5.88% +0.48%
Number of hourly rates quoted 542 660 528
London median hourly rate £21.63 £21.50 £22.00
Median hourly rate % change year-on-year +0.60% -2.27% +15.79%
10th Percentile £12.50 £12.00 £11.99
90th Percentile £49.98 £49.53 £52.50
England median hourly rate £18.56 £18.70 £18.50
% change year-on-year -0.75% +1.08% -

Big Data Engineer
Job Vacancy Trend in London

Job postings that featured Big Data Engineer in the job title as a percentage of all IT jobs advertised in London.

Job vacancy trend for Big Data Engineer in London

Big Data Engineer
Contractor Daily Rate Trend in London

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Big Data Engineer in London.

Contractor daily rate trend for Big Data Engineer in London

Big Data Engineer
Contractor Daily Rate Histogram in London

The daily rate distribution of IT jobs citing Big Data Engineer in London over the 6 months to 20 March 2019.

Contractor daily rate histogram for Big Data Engineer in London

Big Data Engineer
Contract Job Locations in London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Big Data Engineer within the London region over the 6 months to 20 March 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +33 71 £525 -8.70% 9
East London -11 36 £525 -6.67% 4
South London +18 19 £650 +18.18% 3
West London - 12 £550 - 2
Big Data Engineer
England

Big Data Engineer Skill Set
Top 30 Co-occurring IT Skills in London

For the 6 months to 20 March 2019, Big Data Engineer contract jobs required the following IT skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the London region featuring Big Data Engineer in the job title.

1 261 (100.00%) Big Data
2 166 (63.60%) Hadoop
3 165 (63.22%) Python
4 154 (59.00%) Apache Spark
5 146 (55.94%) Amazon AWS
6 109 (41.76%) Kafka
7 94 (36.02%) Microsoft Azure
8 90 (34.48%) Finance
9 86 (32.95%) Java
10 84 (32.18%) Apache Hive
11 79 (30.27%) Agile Software Development
12 77 (29.50%) Data Science
13 68 (26.05%) PySpark
14 65 (24.90%) Google
15 63 (24.14%) Google Cloud Platform
16 62 (23.75%) Scala
17 60 (22.99%) DevOps
17 60 (22.99%) Data Engineering
18 59 (22.61%) Analytics
19 57 (21.84%) HBase
20 52 (19.92%) Docker
21 47 (18.01%) Apache
22 46 (17.62%) Amazon EMR
22 46 (17.62%) Hortonworks
22 46 (17.62%) Cloudera
23 44 (16.86%) Machine Learning
23 44 (16.86%) Degree
24 43 (16.48%) ETL
24 43 (16.48%) Jenkins
25 42 (16.09%) SQL

Big Data Engineer Skill Set
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 154 (59.00%) Apache Spark
2 47 (18.01%) Apache
3 36 (13.79%) Apache Airflow
4 27 (10.34%) Elasticsearch
5 13 (4.98%) Apache Solr
6 8 (3.07%) Apache Pig
6 8 (3.07%) SAS
7 7 (2.68%) Confluence
8 4 (1.53%) Cloud Foundry
9 3 (1.15%) nginx
10 2 (0.77%) Jupyter
11 1 (0.38%) Luigi
Applications
1 7 (2.68%) Microsoft Excel
2 3 (1.15%) Chatbot
3 1 (0.38%) MS Visio
Business Applications
1 17 (6.51%) SAP Oil and Gas
2 1 (0.38%) Distributed Ledger
2 1 (0.38%) SAP ERP
2 1 (0.38%) SAP S/4HANA
Cloud Services
1 146 (55.94%) Amazon AWS
2 94 (36.02%) Microsoft Azure
3 63 (24.14%) Google Cloud Platform
4 46 (17.62%) Amazon EMR
5 18 (6.90%) Amazon S3
6 13 (4.98%) GitHub
7 10 (3.83%) AWS Lambda
7 10 (3.83%) Serverless
8 9 (3.45%) Azure Data Factory
9 8 (3.07%) Amazon Kinesis
9 8 (3.07%) Data Lake Analytics
10 6 (2.30%) Amazon EC2
11 5 (1.92%) IaaS
12 4 (1.53%) IBM Cloud
13 2 (0.77%) SaaS
13 2 (0.77%) Virtual Private Cloud
14 1 (0.38%) Amazon CloudWatch
14 1 (0.38%) CloudFront
14 1 (0.38%) PaaS
14 1 (0.38%) Route 53
Communications & Networking
1 13 (4.98%) Kerberos
2 3 (1.15%) DNS
2 3 (1.15%) VPN
3 2 (0.77%) ICMP
3 2 (0.77%) TCP/IP
4 1 (0.38%) DSL
4 1 (0.38%) Firewall
4 1 (0.38%) LAN
4 1 (0.38%) WAN
Database & Business Intelligence
1 261 (100.00%) Big Data
2 166 (63.60%) Hadoop
3 84 (32.18%) Apache Hive
4 57 (21.84%) HBase
5 42 (16.09%) Apache Cassandra
6 36 (13.79%) NoSQL
7 31 (11.88%) MongoDB
8 25 (9.58%) Data Warehouse
9 21 (8.05%) Data Lake
10 16 (6.13%) SAP HANA
10 16 (6.13%) SQL Server
11 14 (5.36%) Power BI
11 14 (5.36%) Redis
11 14 (5.36%) Sqoop
12 12 (4.60%) Flink
12 12 (4.60%) PostgreSQL
13 11 (4.21%) Impala
14 10 (3.83%) HDInsight
14 10 (3.83%) Relational Database
15 9 (3.45%) Amazon Redshift
Development Applications
1 43 (16.48%) Jenkins
2 22 (8.43%) Git (software)
3 12 (4.60%) Sonatype Nexus
4 10 (3.83%) JIRA
5 8 (3.07%) Maven
5 8 (3.07%) Visual Studio Team System
6 6 (2.30%) Visual Studio
7 5 (1.92%) Cucumber
7 5 (1.92%) Gerrit
7 5 (1.92%) GitLab
7 5 (1.92%) Gradle
7 5 (1.92%) JBehave
7 5 (1.92%) Selenium
8 4 (1.53%) Artifactory
8 4 (1.53%) SonarQube
9 1 (0.38%) Bitbucket
9 1 (0.38%) Eclipse
9 1 (0.38%) IntelliJ
9 1 (0.38%) Octopus Deploy
9 1 (0.38%) TeamCity
General
1 90 (34.48%) Finance
2 19 (7.28%) Banking
3 12 (4.60%) Legal
4 11 (4.21%) Retail
5 7 (2.68%) Front Office
6 4 (1.53%) Marketing
6 4 (1.53%) Telecoms
7 2 (0.77%) Advertising
7 2 (0.77%) Investment Banking
7 2 (0.77%) Publishing
8 1 (0.38%) Corporate Banking
8 1 (0.38%) Dutch Language
8 1 (0.38%) Financial Institution
Libraries, Frameworks & Software Standards
1 109 (41.76%) Kafka
2 68 (26.05%) PySpark
3 15 (5.75%) Elastic Stack
4 11 (4.21%) Apache NiFi
4 11 (4.21%) Pandas
5 10 (3.83%) AngularJS
5 10 (3.83%) Spark Streaming
6 9 (3.45%) RESTful
6 9 (3.45%) Spring
7 7 (2.68%) Spring Boot
8 6 (2.30%) Apache Camel
8 6 (2.30%) Drools
9 5 (1.92%) Gherkin
10 4 (1.53%) Django
10 4 (1.53%) Web Services
11 3 (1.15%) Natural Language Processing
11 3 (1.15%) RabbitMQ
11 3 (1.15%) REST
11 3 (1.15%) Samza
12 2 (0.77%) React
Miscellaneous
1 40 (15.33%) Analytical Skills
2 23 (8.81%) Algorithms
3 8 (3.07%) Fintech
3 8 (3.07%) Greenfield Project
3 8 (3.07%) Public Cloud
4 7 (2.68%) Data Structures
5 4 (1.53%) iPad
6 3 (1.15%) Replication
7 2 (0.77%) Data Centre
7 2 (0.77%) Hedge funds
7 2 (0.77%) Self-Motivation
8 1 (0.38%) Analytical Mindset
8 1 (0.38%) Enterprise Cloud
8 1 (0.38%) HoloLens
8 1 (0.38%) Linux Command Line
8 1 (0.38%) Management Information System
8 1 (0.38%) NHS
8 1 (0.38%) Wiki
Operating Systems
1 29 (11.11%) Linux
2 19 (7.28%) Unix
3 2 (0.77%) Windows
Processes & Methodologies
1 79 (30.27%) Agile Software Development
2 77 (29.50%) Data Science
3 60 (22.99%) Data Engineering
3 60 (22.99%) DevOps
4 59 (22.61%) Analytics
5 44 (16.86%) Machine Learning
6 43 (16.48%) ETL
7 33 (12.64%) Data Analysis
7 33 (12.64%) Statistical Modelling
8 32 (12.26%) Mentoring
9 31 (11.88%) Data Cleansing
10 30 (11.49%) Coaching
11 27 (10.34%) Deep Learning
12 26 (9.96%) Scrum
13 25 (9.58%) Performance Optimisation
13 25 (9.58%) Predictive Analytics
13 25 (9.58%) Software Engineering
14 21 (8.05%) Data Modelling
15 20 (7.66%) Visualisation
16 19 (7.28%) Computer Science
Programming Languages
1 165 (63.22%) Python
2 86 (32.95%) Java
3 62 (23.75%) Scala
4 42 (16.09%) SQL
5 19 (7.28%) Bash Shell
6 13 (4.98%) R
7 12 (4.60%) JavaScript
8 11 (4.21%) Groovy
9 9 (3.45%) Java 8
9 9 (3.45%) Ruby
10 8 (3.07%) Julia
11 6 (2.30%) VBA
12 4 (1.53%) C#
13 3 (1.15%) AWK
13 3 (1.15%) C++
13 3 (1.15%) Go
14 2 (0.77%) MATLAB
14 2 (0.77%) Perl
14 2 (0.77%) PowerShell
14 2 (0.77%) T-SQL
Qualifications
1 44 (16.86%) Degree
2 29 (11.11%) Master's Degree
3 19 (7.28%) Security Cleared
4 17 (6.51%) SC Cleared
5 15 (5.75%) Computer Science Degree
6 1 (0.38%) AWS Certification
6 1 (0.38%) Numerate Degree
Quality Assurance & Compliance
1 6 (2.30%) Data Quality
2 2 (0.77%) QA
2 2 (0.77%) SLA
System Software
1 52 (19.92%) Docker
2 33 (12.64%) HDFS
3 19 (7.28%) Apache Flume
4 18 (6.90%) Apache ZooKeeper
5 4 (1.53%) vCloud
5 4 (1.53%) VMware Infrastructure
6 1 (0.38%) Oracle RAC
6 1 (0.38%) Virtual Machines
Systems Management
1 36 (13.79%) Ansible
2 31 (11.88%) Kubernetes
3 26 (9.96%) Terraform
4 24 (9.20%) Hadoop YARN
5 15 (5.75%) Puppet
6 14 (5.36%) Oozie
7 9 (3.45%) logstash
8 7 (2.68%) Kibana
9 6 (2.30%) Ambari
10 5 (1.92%) Nagios
10 5 (1.92%) ZABBIX
11 4 (1.53%) Consul
11 4 (1.53%) Opscode Chef
12 2 (0.77%) Grafana
12 2 (0.77%) HAProxy
12 2 (0.77%) Host Intrusion Detection System
12 2 (0.77%) Network Intrusion Detection System
12 2 (0.77%) Prometheus
12 2 (0.77%) Salt
12 2 (0.77%) Sysdig
Vendors
1 65 (24.90%) Google
2 46 (17.62%) Cloudera
2 46 (17.62%) Hortonworks
3 34 (13.03%) Talend
4 22 (8.43%) Oracle
5 21 (8.05%) Databricks
5 21 (8.05%) Informatica
6 18 (6.90%) SAP
7 10 (3.83%) Alteryx
7 10 (3.83%) Microsoft
8 8 (3.07%) Tableau
9 6 (2.30%) SaltStack
10 4 (1.53%) IBM
10 4 (1.53%) Splunk
10 4 (1.53%) VMware
11 1 (0.38%) Cisco
11 1 (0.38%) Greenplum
11 1 (0.38%) MapR
11 1 (0.38%) New Relic
11 1 (0.38%) Teradata