Period
to 22 February 2018

The median annual salary for a Big Data Engineer was £70,000 in advertised job vacancies during the 6 months to 22 February 2018.

The first table below provides salary benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Big Data Engineer
UK
6 months to
22 Feb 2018
Same period 2017 Same period 2016
Rank 783 921 1058
Rank change year-on-year +138 +137 +71
Permanent jobs requiring a Big Data Engineer 420 266 234
As % of all permanent IT jobs advertised in the UK 0.24% 0.15% 0.11%
As % of the Job Titles category 0.25% 0.16% 0.12%
Number of salaries quoted 357 243 198
UK median annual salary £70,000 £72,500 £67,500
Median salary % change year-on-year -3.45% +7.41% +8.00%
10th Percentile £45,750 £47,500 £40,000
90th Percentile £97,500 £95,000 £93,750
UK excluding London median annual salary £60,000 £55,000 £42,500
% change year-on-year +9.09% +29.41% -29.17%

The following table is for comparison with the above and includes summary statistics for all permanent IT job vacancies. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of permanent jobs in our overall sample.

All Permanent IT Job Vacancies
UK
Permanent vacancies in the UK with a recognized job title 169,468 169,423 202,837
% of permanent IT jobs with a recognized job title 96.25% 96.48% 96.65%
Number of salaries quoted 138,985 140,739 169,592
UK median annual salary £50,000 £47,500 £46,309
Median salary % change year-on-year +5.26% +2.57% +2.91%
10th Percentile £27,500 £27,000 £26,750
90th Percentile £80,000 £77,500 £76,000
UK excluding London median annual salary £42,500 £42,500 £42,000
% change year-on-year - +1.19% +5.00%

Big Data Engineer
Job Vacancy Trend

Job postings that featured Big Data Engineer in the job title as a percentage of all IT jobs advertised.

Job vacancy trend for Big Data Engineer in the UK

Big Data Engineer
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Big Data Engineer.

Salary trend for Big Data Engineer in the UK

Big Data Engineer
Salary Histogram

The salary distribution of IT jobs citing Big Data Engineer over the 6 months to 22 February 2018.

Salary histogram for Big Data Engineer in the UK

Big Data Engineer
Top 13 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Big Data Engineer within the UK over the 6 months to 22 February 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +124 395 £70,000 - 67
London +83 288 £75,000 - 47
UK excluding London +55 115 £60,000 +9.09% 23
South East +36 55 £60,000 +20.00% 12
East of England +31 27 £75,000 +50.00% 1
North of England +13 14 £45,000 - 6
North West +8 10 £45,000 +12.50% 2
South West -12 6 £52,500 -25.00%
West Midlands +18 5 £65,000 - 1
Midlands +8 5 £65,000 - 1
Yorkshire +23 4 £57,500 +15.00% 4
Wales +2 4 £60,000 -25.00% 1
Scotland -30 4 £57,500 +27.78% 2

Big Data Engineer Skill Set
Top 30 Co-occurring IT Skills

For the 6 months to 22 February 2018, Big Data Engineer job roles required the following IT skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads featuring Big Data Engineer in the job title.

1 420 (100.00%) Big Data
2 328 (78.10%) Hadoop
3 302 (71.90%) Apache Spark
4 228 (54.29%) Python
5 224 (53.33%) Java
6 183 (43.57%) Scala
7 169 (40.24%) Agile Software Development
8 167 (39.76%) Apache Hive
9 155 (36.90%) Kafka
10 148 (35.24%) NoSQL
11 143 (34.05%) Analytics
12 137 (32.62%) HBase
13 131 (31.19%) Cloudera
14 117 (27.86%) Apache Cassandra
15 116 (27.62%) SQL
15 116 (27.62%) Amazon AWS
16 115 (27.38%) Apache Flume
17 102 (24.29%) Machine Learning
18 100 (23.81%) Data Science
19 98 (23.33%) MongoDB
20 93 (22.14%) ETL
21 92 (21.90%) Software Engineering
22 88 (20.95%) Linux
23 86 (20.48%) DevOps
24 74 (17.62%) Elasticsearch
24 74 (17.62%) Hortonworks
25 68 (16.19%) Hadoop YARN
26 67 (15.95%) Impala
27 64 (15.24%) Data Engineering
27 64 (15.24%) TDD

Big Data Engineer Skill Set
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 302 (71.90%) Apache Spark
2 74 (17.62%) Elasticsearch
3 53 (12.62%) Apache Solr
4 40 (9.52%) Apache Pig
5 34 (8.10%) Apache
6 19 (4.52%) SAS
7 10 (2.38%) Apache Airflow
8 6 (1.43%) OpenStack
9 4 (0.95%) Confluence
10 2 (0.48%) GeoServer
10 2 (0.48%) Jetty
10 2 (0.48%) nginx
11 1 (0.24%) Luigi
Applications
1 1 (0.24%) Microsoft Excel
1 1 (0.24%) Weka
Cloud Services
1 116 (27.62%) Amazon AWS
2 53 (12.62%) Microsoft Azure
3 29 (6.90%) GitHub
4 21 (5.00%) Google Cloud Platform
5 19 (4.52%) Amazon S3
6 15 (3.57%) Amazon EC2
7 12 (2.86%) Amazon EMR
8 11 (2.62%) Amazon Kinesis
9 9 (2.14%) AWS Lambda
10 8 (1.90%) Serverless
11 6 (1.43%) SaaS
12 5 (1.19%) Amazon SQS
12 5 (1.19%) AWS Elastic Beanstalk
12 5 (1.19%) Azure Data Factory
12 5 (1.19%) Cloud Computing
12 5 (1.19%) Virtual Private Cloud
13 2 (0.48%) AWS CloudTrail
13 2 (0.48%) Azure Active Directory
13 2 (0.48%) CloudFront
13 2 (0.48%) Route 53
Communications & Networking
1 9 (2.14%) Internet
2 2 (0.48%) DAB
2 2 (0.48%) SAN
3 1 (0.24%) HTTP
Database & Business Intelligence
1 420 (100.00%) Big Data
2 328 (78.10%) Hadoop
3 167 (39.76%) Apache Hive
4 148 (35.24%) NoSQL
5 137 (32.62%) HBase
6 117 (27.86%) Apache Cassandra
7 98 (23.33%) MongoDB
8 67 (15.95%) Impala
9 55 (13.10%) Data Warehouse
10 46 (10.95%) Sqoop
11 41 (9.76%) Relational Database
12 40 (9.52%) Flink
13 31 (7.38%) Neo4j
14 25 (5.95%) Data Lake
15 24 (5.71%) RDBMS
16 21 (5.00%) PostgreSQL
16 21 (5.00%) SQL Server
17 18 (4.29%) HAWQ
17 18 (4.29%) MySQL
17 18 (4.29%) Pentaho
Development Applications
1 60 (14.29%) Jenkins
2 36 (8.57%) Git (software)
3 24 (5.71%) Maven
4 20 (4.76%) TeamCity
5 19 (4.52%) Cucumber
6 12 (2.86%) JIRA
7 9 (2.14%) Sonatype Nexus
8 7 (1.67%) JUnit
9 6 (1.43%) Vagrant
10 5 (1.19%) GitLab
11 4 (0.95%) Artifactory
11 4 (0.95%) Selenium
12 3 (0.71%) Atlassian Bamboo
12 3 (0.71%) ProC
12 3 (0.71%) Team Foundation Server
12 3 (0.71%) Travis CI
12 3 (0.71%) Visual Studio
12 3 (0.71%) WebDriver
13 2 (0.48%) git-flow
13 2 (0.48%) Octopus Deploy
General
1 54 (12.86%) Finance
2 27 (6.43%) Retail
3 26 (6.19%) Telecoms
4 17 (4.05%) Banking
4 17 (4.05%) Marketing
5 11 (2.62%) Law
6 8 (1.90%) Pharmaceutical
7 6 (1.43%) Advertising
7 6 (1.43%) Electronics
7 6 (1.43%) Games
7 6 (1.43%) Legal
7 6 (1.43%) Manufacturing
8 4 (0.95%) Publishing
9 3 (0.71%) Investment Banking
10 1 (0.24%) iGaming
10 1 (0.24%) Military
10 1 (0.24%) Music Industry
Libraries, Frameworks & Software Standards
1 155 (36.90%) Kafka
2 40 (9.52%) Spark Streaming
3 24 (5.71%) RabbitMQ
3 24 (5.71%) XML
4 17 (4.05%) Middleware
4 17 (4.05%) Web Services
5 15 (3.57%) JSON
5 15 (3.57%) Natural Language Processing
6 12 (2.86%) Node.js
6 12 (2.86%) TensorFlow
7 9 (2.14%) Keras
7 9 (2.14%) MLlib
7 9 (2.14%) YAML
8 8 (1.90%) HTML
8 8 (1.90%) Spring
9 7 (1.67%) Akka
9 7 (1.67%) NumPy
9 7 (1.67%) REST
10 6 (1.43%) Pandas
10 6 (1.43%) PySpark
Miscellaneous
1 59 (14.05%) Analytical Skills
2 52 (12.38%) Computer Science
3 24 (5.71%) Algorithms
3 24 (5.71%) Fintech
4 20 (4.76%) Greenfield Project
5 8 (1.90%) Data Structures
5 8 (1.90%) User Experience
6 6 (1.43%) Data Centre
6 6 (1.43%) Enterprise Software
7 5 (1.19%) Clustering
7 5 (1.19%) Hedge funds
8 3 (0.71%) Linux Command Line
8 3 (0.71%) Smart Energy
9 2 (0.48%) Client/Server
9 2 (0.48%) Equities
9 2 (0.48%) Foreign Exchange (FX)
9 2 (0.48%) Internet of Things
9 2 (0.48%) Online Games
10 1 (0.24%) Blog
10 1 (0.24%) Market Risk
Operating Systems
1 88 (20.95%) Linux
2 58 (13.81%) Unix
3 12 (2.86%) Windows
4 5 (1.19%) Mac OS X
5 4 (0.95%) Red Hat Enterprise Linux
6 2 (0.48%) Ubuntu
7 1 (0.24%) Android
7 1 (0.24%) CentOS
7 1 (0.24%) DC/OS
7 1 (0.24%) Solaris
Processes & Methodologies
1 169 (40.24%) Agile Software Development
2 143 (34.05%) Analytics
3 102 (24.29%) Machine Learning
4 100 (23.81%) Data Science
5 93 (22.14%) ETL
6 92 (21.90%) Software Engineering
7 86 (20.48%) DevOps
8 64 (15.24%) Data Engineering
8 64 (15.24%) TDD
9 62 (14.76%) MapReduce
10 61 (14.52%) Open Source
11 55 (13.10%) Continuous Integration
12 46 (10.95%) Business Intelligence
12 46 (10.95%) Data Analytics
13 39 (9.29%) Visualisation
14 32 (7.62%) Data Ingestion
15 31 (7.38%) Data Modelling
15 31 (7.38%) Data Transformation
16 30 (7.14%) BDD
17 29 (6.90%) Continuous Delivery
Programming Languages
1 228 (54.29%) Python
2 224 (53.33%) Java
3 183 (43.57%) Scala
4 116 (27.62%) SQL
5 42 (10.00%) Bash Shell
6 32 (7.62%) R
7 26 (6.19%) Go
8 23 (5.48%) C#
9 18 (4.29%) C
10 17 (4.05%) Shell Script
11 14 (3.33%) JavaScript
12 11 (2.62%) C++
13 7 (1.67%) MATLAB
13 7 (1.67%) Ruby
14 6 (1.43%) Perl
15 5 (1.19%) Groovy
16 4 (0.95%) PowerShell
16 4 (0.95%) T-SQL
17 3 (0.71%) HiveQL
17 3 (0.71%) PROC SQL
Qualifications
1 50 (11.90%) Degree
2 25 (5.95%) Security Cleared
3 24 (5.71%) Computer Science Degree
4 20 (4.76%) DV Cleared
5 14 (3.33%) Master's Degree
6 11 (2.62%) PhD
7 3 (0.71%) AWS Certification
7 3 (0.71%) Numerate Degree
8 2 (0.48%) EMC Certification
9 1 (0.24%) SC Cleared
Quality Assurance & Compliance
1 13 (3.10%) QA
2 6 (1.43%) Data Quality
3 5 (1.19%) GCP
4 4 (0.95%) SLA
System Software
1 115 (27.38%) Apache Flume
2 61 (14.52%) HDFS
3 55 (13.10%) Docker
4 19 (4.52%) Apache ZooKeeper
5 13 (3.10%) Virtual Machines
Systems Management
1 68 (16.19%) Hadoop YARN
2 41 (9.76%) Oozie
3 33 (7.86%) Puppet
4 25 (5.95%) Ansible
4 25 (5.95%) Kubernetes
5 18 (4.29%) Opscode Chef
6 14 (3.33%) Terraform
7 7 (1.67%) Mesos
8 6 (1.43%) Kibana
9 3 (0.71%) Salt
10 2 (0.48%) Ambari
11 1 (0.24%) Control-M
11 1 (0.24%) Docker Swarm
11 1 (0.24%) Marathon
11 1 (0.24%) Single Sign-On
Vendors
1 131 (31.19%) Cloudera
2 74 (17.62%) Hortonworks
3 47 (11.19%) Google
4 43 (10.24%) MapR
5 41 (9.76%) Tableau
6 26 (6.19%) Oracle
6 26 (6.19%) Talend
7 21 (5.00%) Microsoft
8 20 (4.76%) Informatica
9 18 (4.29%) Greenplum
10 16 (3.81%) SAP
11 14 (3.33%) IBM
12 13 (3.10%) MuleSoft
13 10 (2.38%) Facebook
14 9 (2.14%) Atlassian
14 9 (2.14%) Splunk
15 8 (1.90%) Teradata
16 5 (1.19%) Red Hat
16 5 (1.19%) Salesforce.com
17 4 (0.95%) MicroStrategy