Period
to 21 June 2018

The median annual salary for a Big Data Engineer was £70,000 in advertised job vacancies during the 6 months to 21 June 2018.

The first table below provides salary benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Big Data Engineer
UK
6 months to
21 Jun 2018
Same period 2017 Same period 2016
Rank 725 914 1016
Rank change year-on-year +189 +102 +111
Permanent jobs requiring a Big Data Engineer 499 281 260
As % of all permanent IT jobs advertised in the UK 0.29% 0.16% 0.13%
As % of the Job Titles category 0.30% 0.17% 0.13%
Number of salaries quoted 396 225 214
UK median annual salary £70,000 £67,500 £63,750
Median salary % change year-on-year +3.70% +5.88% +6.25%
10th Percentile £47,500 £50,000 £41,625
90th Percentile £94,375 £90,000 £99,625
UK excluding London median annual salary £60,000 £58,750 £47,500
% change year-on-year +2.13% +23.68% -20.83%

The following table is for comparison with the above and includes summary statistics for all permanent IT job vacancies. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of permanent jobs in our overall sample.

All Permanent IT Job Vacancies
UK
Permanent vacancies in the UK with a recognized job title 166,189 166,064 194,291
% of permanent IT jobs with a recognized job title 96.33% 96.24% 96.61%
Number of salaries quoted 135,115 136,769 163,202
UK median annual salary £50,000 £47,500 £47,500
Median salary % change year-on-year +5.26% - +5.56%
10th Percentile £27,500 £27,500 £26,500
90th Percentile £80,000 £77,500 £76,250
UK excluding London median annual salary £45,000 £42,500 £42,000
% change year-on-year +5.88% +1.19% +5.00%

Big Data Engineer
Job Vacancy Trend

Job postings that featured Big Data Engineer in the job title as a percentage of all IT jobs advertised.

Job vacancy trend for Big Data Engineer in the UK

Big Data Engineer
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Big Data Engineer.

Salary trend for Big Data Engineer in the UK

Big Data Engineer
Salary Histogram

The salary distribution of IT jobs citing Big Data Engineer over the 6 months to 21 June 2018.

Salary histogram for Big Data Engineer in the UK

Big Data Engineer
Top 14 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Big Data Engineer within the UK over the 6 months to 21 June 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +174 472 £70,000 +3.70% 55
London +98 320 £70,000 - 44
UK excluding London +83 157 £60,000 +2.13% 14
South East +56 98 £60,000 -2.44% 8
North of England +37 26 £52,500 -27.59% 2
East of England +13 17 £70,000 +40.00%
North West +12 16 £53,750 -18.87% 1
Yorkshire - 10 £45,000 - 1
Midlands +23 8 £62,500 - 1
East Midlands - 5 £60,000 - 1
Scotland -29 4 - - 3
West Midlands +13 3 £67,500 -
South West -15 3 £55,000 -
Wales - 1 £55,000 -

Big Data Engineer Skill Set
Top 30 Co-occurring IT Skills

For the 6 months to 21 June 2018, Big Data Engineer job roles required the following IT skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads featuring Big Data Engineer in the job title.

1 499 (100.00%) Big Data
2 372 (74.55%) Hadoop
3 317 (63.53%) Apache Spark
4 271 (54.31%) Python
5 259 (51.90%) Java
6 239 (47.90%) Scala
7 212 (42.48%) Apache Hive
8 209 (41.88%) Agile Software Development
9 191 (38.28%) Kafka
10 183 (36.67%) Analytics
11 162 (32.46%) Cloudera
12 156 (31.26%) SQL
13 150 (30.06%) NoSQL
14 148 (29.66%) ETL
15 146 (29.26%) Amazon AWS
16 131 (26.25%) Apache Flume
17 123 (24.65%) Impala
18 119 (23.85%) HBase
18 119 (23.85%) Linux
18 119 (23.85%) Data Science
19 117 (23.45%) Apache Cassandra
20 114 (22.85%) Continuous Integration
21 113 (22.65%) Machine Learning
22 102 (20.44%) Hortonworks
23 98 (19.64%) Data Warehouse
24 92 (18.44%) Software Engineering
24 92 (18.44%) Jenkins
25 90 (18.04%) Data Engineering
26 87 (17.43%) HDFS
26 87 (17.43%) Hadoop YARN

Big Data Engineer Skill Set
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 317 (63.53%) Apache Spark
2 73 (14.63%) Elasticsearch
3 50 (10.02%) Apache Solr
4 39 (7.82%) Apache Pig
5 32 (6.41%) Apache
6 14 (2.81%) Confluence
7 6 (1.20%) SAS
8 3 (0.60%) Apache Airflow
8 3 (0.60%) Tomcat
9 2 (0.40%) GemFire
9 2 (0.40%) JBoss
9 2 (0.40%) OpenStack
9 2 (0.40%) WildFly
10 1 (0.20%) Cloud Foundry
10 1 (0.20%) IIS
10 1 (0.20%) Luigi
10 1 (0.20%) nginx
Applications
1 5 (1.00%) GNU Octave
Business Applications
1 2 (0.40%) Dynamics AX
1 2 (0.40%) Dynamics CRM
1 2 (0.40%) Dynamics NAV
Cloud Services
1 146 (29.26%) Amazon AWS
2 52 (10.42%) Microsoft Azure
3 27 (5.41%) Amazon EMR
3 27 (5.41%) Google Cloud Platform
4 26 (5.21%) GitHub
5 22 (4.41%) Amazon S3
6 20 (4.01%) Amazon Kinesis
7 15 (3.01%) AWS Lambda
7 15 (3.01%) Serverless
8 11 (2.20%) Amazon EC2
9 9 (1.80%) IBM Cloud
10 6 (1.20%) Cloud Computing
11 5 (1.00%) AWS Elastic Beanstalk
11 5 (1.00%) SaaS
12 4 (0.80%) Amazon SQS
13 3 (0.60%) PaaS
14 2 (0.40%) Azure Data Factory
15 1 (0.20%) Amazon Glacier
15 1 (0.20%) CloudBees
15 1 (0.20%) Data Lake Analytics
Communications & Networking
1 7 (1.40%) Internet
2 2 (0.40%) HTTP
3 1 (0.20%) 5G
3 1 (0.20%) HTTPS
3 1 (0.20%) Kerberos
Database & Business Intelligence
1 499 (100.00%) Big Data
2 372 (74.55%) Hadoop
3 212 (42.48%) Apache Hive
4 150 (30.06%) NoSQL
5 123 (24.65%) Impala
6 119 (23.85%) HBase
7 117 (23.45%) Apache Cassandra
8 98 (19.64%) Data Warehouse
9 80 (16.03%) MongoDB
10 57 (11.42%) Sqoop
11 52 (10.42%) Neo4j
12 48 (9.62%) Relational Database
13 38 (7.62%) Flink
14 36 (7.21%) PostgreSQL
15 30 (6.01%) RDBMS
16 28 (5.61%) Amazon Redshift
16 28 (5.61%) SQL Server
17 24 (4.81%) Data Lake
18 22 (4.41%) MySQL
19 20 (4.01%) SQL Server Integration Services
Development Applications
1 92 (18.44%) Jenkins
2 62 (12.42%) Git (software)
3 41 (8.22%) TeamCity
4 26 (5.21%) JIRA
5 21 (4.21%) Cucumber
6 13 (2.61%) Atlassian Bamboo
6 13 (2.61%) JUnit
7 10 (2.00%) GitLab
8 9 (1.80%) Vagrant
9 8 (1.60%) Maven
10 7 (1.40%) SQL Server Management Studio (SSMS)
11 5 (1.00%) Sonatype Nexus
12 4 (0.80%) RStudio
12 4 (0.80%) Selenium
13 3 (0.60%) CA Release Automation
14 2 (0.40%) Octopus Deploy
14 2 (0.40%) Subversion
14 2 (0.40%) YouTrack
15 1 (0.20%) SonarQube
15 1 (0.20%) tSQLt
General
1 51 (10.22%) Finance
2 37 (7.41%) Marketing
3 22 (4.41%) Telecoms
4 20 (4.01%) Retail
5 17 (3.41%) Advertising
6 16 (3.21%) Banking
7 10 (2.00%) Games
7 10 (2.00%) Pharmaceutical
8 9 (1.80%) Law
9 4 (0.80%) Legal
9 4 (0.80%) Manufacturing
10 2 (0.40%) Financial Institution
11 1 (0.20%) Aerospace
11 1 (0.20%) Electronics
11 1 (0.20%) Investment Banking
11 1 (0.20%) Music Industry
Libraries, Frameworks & Software Standards
1 191 (38.28%) Kafka
2 41 (8.22%) Spark Streaming
3 38 (7.62%) TensorFlow
4 35 (7.01%) MLlib
5 34 (6.81%) Keras
6 26 (5.21%) Middleware
7 24 (4.81%) RabbitMQ
7 24 (4.81%) Web Services
8 22 (4.41%) XML
9 21 (4.21%) NumPy
9 21 (4.21%) Pandas
10 19 (3.81%) Flask
11 17 (3.41%) Django
12 15 (3.01%) Akka
13 14 (2.81%) JSON
14 11 (2.20%) PySpark
15 9 (1.80%) Elastic Stack
16 8 (1.60%) REST
17 7 (1.40%) Lucene
18 6 (1.20%) Spring
Miscellaneous
1 63 (12.63%) Computer Science
2 45 (9.02%) Analytical Skills
3 29 (5.81%) Algorithms
3 29 (5.81%) User Experience
4 28 (5.61%) Greenfield Project
5 16 (3.21%) Fintech
6 15 (3.01%) Data Structures
7 14 (2.81%) Self-Motivation
8 8 (1.60%) Clustering
8 8 (1.60%) Enterprise Software
9 5 (1.00%) Smart Energy
10 3 (0.60%) Blog
11 2 (0.40%) Client/Server
11 2 (0.40%) Hedge funds
11 2 (0.40%) Hybrid Cloud
11 2 (0.40%) Internet of Things
11 2 (0.40%) Management Information System
11 2 (0.40%) Marketing Technology
11 2 (0.40%) Social Media
11 2 (0.40%) Video\Live Streaming
Operating Systems
1 119 (23.85%) Linux
2 45 (9.02%) Unix
3 23 (4.61%) Windows
4 8 (1.60%) Red Hat Enterprise Linux
5 6 (1.20%) Windows Server
6 5 (1.00%) Windows Server 2012
7 2 (0.40%) CentOS
7 2 (0.40%) Mac OS X
7 2 (0.40%) Ubuntu
8 1 (0.20%) Windows Server 2008
Processes & Methodologies
1 209 (41.88%) Agile Software Development
2 183 (36.67%) Analytics
3 148 (29.66%) ETL
4 119 (23.85%) Data Science
5 114 (22.85%) Continuous Integration
6 113 (22.65%) Machine Learning
7 92 (18.44%) Software Engineering
8 90 (18.04%) Data Engineering
9 83 (16.63%) MapReduce
10 81 (16.23%) DevOps
11 80 (16.03%) TDD
12 56 (11.22%) Open Source
13 45 (9.02%) Visualisation
14 43 (8.62%) Configuration Management
14 43 (8.62%) Data Modelling
15 40 (8.02%) Data Analytics
15 40 (8.02%) Scrum
16 37 (7.41%) Deep Learning
17 36 (7.21%) Data Ingestion
18 35 (7.01%) Problem-Solving
Programming Languages
1 271 (54.31%) Python
2 259 (51.90%) Java
3 239 (47.90%) Scala
4 156 (31.26%) SQL
5 47 (9.42%) Go
6 43 (8.62%) Shell Script
7 39 (7.82%) Bash Shell
8 38 (7.62%) R
9 26 (5.21%) C#
10 18 (3.61%) C++
11 17 (3.41%) T-SQL
12 16 (3.21%) C
13 13 (2.61%) Java 8
14 11 (2.20%) JavaScript
15 7 (1.40%) Perl
16 6 (1.20%) PowerShell
17 5 (1.00%) Groovy
17 5 (1.00%) PL/SQL
18 4 (0.80%) Ruby
19 3 (0.60%) Clojure
Qualifications
1 53 (10.62%) Degree
2 27 (5.41%) Computer Science Degree
3 25 (5.01%) Security Cleared
4 15 (3.01%) PhD
5 13 (2.61%) DV Cleared
6 9 (1.80%) Master's Degree
6 9 (1.80%) Numerate Degree
7 5 (1.00%) AWS Certification
8 3 (0.60%) ISEB
8 3 (0.60%) ISTQB
9 2 (0.40%) EMC Certification
9 2 (0.40%) MCSA
9 2 (0.40%) MCSD
9 2 (0.40%) Microsoft Certification
10 1 (0.20%) SC Cleared
Quality Assurance & Compliance
1 15 (3.01%) Data Quality
2 11 (2.20%) QA
3 6 (1.20%) GCP
4 2 (0.40%) SLA
5 1 (0.20%) GDPR
System Software
1 131 (26.25%) Apache Flume
2 87 (17.43%) HDFS
3 46 (9.22%) Docker
4 15 (3.01%) Virtual Machines
5 14 (2.81%) Apache ZooKeeper
6 9 (1.80%) VirtualBox
7 3 (0.60%) Oracle RAC
8 1 (0.20%) Cronacle
8 1 (0.20%) VMware ESXi
8 1 (0.20%) VMware Infrastructure
Systems Management
1 87 (17.43%) Hadoop YARN
2 37 (7.41%) Puppet
3 36 (7.21%) Oozie
4 29 (5.81%) Ansible
5 19 (3.81%) Kibana
5 19 (3.81%) Kubernetes
6 18 (3.61%) Mesos
7 11 (2.20%) Terraform
8 10 (2.00%) Docker Swarm
9 6 (1.20%) logstash
9 6 (1.20%) Opscode Chef
10 5 (1.00%) Grafana
10 5 (1.00%) Prometheus
11 4 (0.80%) Salt
12 2 (0.40%) Ambari
12 2 (0.40%) Control-M
12 2 (0.40%) Host Intrusion Detection System
12 2 (0.40%) Xymon
12 2 (0.40%) ZABBIX
13 1 (0.20%) BMC PATROL
Vendors
1 162 (32.46%) Cloudera
2 102 (20.44%) Hortonworks
3 59 (11.82%) Google
4 43 (8.62%) Informatica
5 42 (8.42%) Talend
6 28 (5.61%) Tableau
7 26 (5.21%) IBM
7 26 (5.21%) Microsoft
8 25 (5.01%) MapR
9 18 (3.61%) Greenplum
10 16 (3.21%) MuleSoft
11 15 (3.01%) Oracle
12 14 (2.81%) Atlassian
13 10 (2.00%) Facebook
13 10 (2.00%) Red Hat
14 6 (1.20%) Salesforce.com
14 6 (1.20%) Twitter
15 5 (1.00%) SAP
16 4 (0.80%) Cognizant
16 4 (0.80%) MicroStrategy