Period
to 23 November 2017

The median annual salary for a Big Data Engineer was £70,000 in advertised job vacancies during the 6 months to 23 November 2017.

The first table below provides salary benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Big Data Engineer
UK
6 months to
23 Nov 2017
Same period 2016 Same period 2015
Rank 882 1006 1088
Rank change year-on-year +124 +82 +57
Permanent jobs requiring a Big Data Engineer 308 233 208
As % of all permanent IT jobs advertised in the UK 0.17% 0.12% 0.096%
As % of the Job Titles category 0.18% 0.13% 0.099%
Number of salaries quoted 258 199 173
UK median annual salary £70,000 £70,000 £65,000
Median salary % change year-on-year - +7.69% -
10th Percentile £46,250 £45,000 £38,750
90th Percentile £91,250 £95,000 £91,250
UK excluding London median annual salary £60,000 £55,000 £42,500
% change year-on-year +9.09% +29.41% -15.00%

The following table is for comparison with the above and includes summary statistics for all permanent IT job vacancies. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of permanent jobs in our overall sample.

All Permanent IT Job Vacancies
UK
Permanent vacancies in the UK with a recognized job title 172559 183130 209788
% of permanent IT jobs with a recognized job title 95.79% 96.48% 96.46%
Number of salaries quoted 138476 153185 174290
UK median annual salary £48,000 £47,500 £45,000
Median salary % change year-on-year +1.05% +5.56% -
10th Percentile £27,500 £27,000 £26,250
90th Percentile £78,750 £77,500 £75,000
UK excluding London median annual salary £42,500 £42,500 £41,000
% change year-on-year - +3.66% +2.50%

Big Data Engineer
Job Vacancy Trend

Job postings that featured Big Data Engineer in the job title as a percentage of all IT jobs advertised.

Job vacancy trend for Big Data Engineer in the UK

Big Data Engineer
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Big Data Engineer.

Salary trend for Big Data Engineer in the UK

Big Data Engineer
Salary Histogram

The salary distribution of IT jobs citing Big Data Engineer over the 6 months to 23 November 2017.

Salary histogram for Big Data Engineer in the UK

Big Data Engineer
Top 13 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Big Data Engineer within the UK over the 6 months to 23 November 2017. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +113 300 £70,000 - 41
London +53 210 £75,000 +3.45% 35
UK excluding London +68 88 £60,000 +9.09% 7
South East +59 50 £65,000 +30.00% 2
East of England - 16 £60,000 - 2
South West -2 9 £60,000 +9.09% 1
North of England +7 5 £45,000 -
North West -16 5 £45,000 +12.50%
Midlands +27 4 £55,000 +50.68% 1
Wales +12 3 £60,000 -25.00%
West Midlands - 3 £55,000 - 1
East Midlands +32 1 £55,000 +50.68%
Scotland -11 1 £57,500 +24.32% 1

Big Data Engineer Skill Set
Top 30 Co-occurring IT Skills

For the 6 months to 23 November 2017, Big Data Engineer job roles required the following IT skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all job ads featuring Big Data Engineer in the job title.

1 308 (100.00%) Big Data
2 245 (79.55%) Hadoop
3 230 (74.68%) Apache Spark
4 173 (56.17%) Java
5 157 (50.97%) Agile Software Development
6 153 (49.68%) Python
7 147 (47.73%) Scala
8 107 (34.74%) Analytics
9 105 (34.09%) Apache Hive
10 94 (30.52%) NoSQL
10 94 (30.52%) Kafka
11 75 (24.35%) HBase
12 73 (23.70%) SQL
13 70 (22.73%) Amazon AWS
14 67 (21.75%) Machine Learning
15 63 (20.45%) Software Engineering
16 60 (19.48%) Cloudera
17 59 (19.16%) Apache Flume
17 59 (19.16%) TDD
17 59 (19.16%) ETL
18 56 (18.18%) Linux
18 56 (18.18%) Analytical Skills
19 53 (17.21%) MongoDB
20 52 (16.88%) DevOps
21 49 (15.91%) Apache Cassandra
22 46 (14.94%) Data Analytics
22 46 (14.94%) Business Intelligence
23 45 (14.61%) Docker
23 45 (14.61%) Sqoop
24 44 (14.29%) Open Source

Big Data Engineer Skill Set
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 230 (74.68%) Apache Spark
2 38 (12.34%) Apache
3 32 (10.39%) Elasticsearch
4 27 (8.77%) Apache Solr
5 17 (5.52%) SAS
6 11 (3.57%) Apache Pig
7 8 (2.60%) OpenStack
8 6 (1.95%) Confluence
9 4 (1.30%) Apache Airflow
9 4 (1.30%) GeoServer
10 3 (0.97%) Tomcat
11 2 (0.65%) Jetty
11 2 (0.65%) nginx
Applications
1 1 (0.32%) Microsoft Excel
1 1 (0.32%) Weka
Cloud Services
1 70 (22.73%) Amazon AWS
2 32 (10.39%) Microsoft Azure
3 16 (5.19%) Amazon EC2
3 16 (5.19%) Amazon S3
3 16 (5.19%) Cloud Computing
4 15 (4.87%) Google Cloud Platform
5 9 (2.92%) Amazon EMR
6 8 (2.60%) Amazon Kinesis
7 5 (1.62%) Azure Data Factory
7 5 (1.62%) GitHub
8 4 (1.30%) AWS Lambda
8 4 (1.30%) SaaS
8 4 (1.30%) Serverless
8 4 (1.30%) Virtual Private Cloud
9 2 (0.65%) AWS CloudFormation
9 2 (0.65%) AWS CloudTrail
9 2 (0.65%) Azure Active Directory
9 2 (0.65%) CloudFront
9 2 (0.65%) Route 53
9 2 (0.65%) Slack
Communications & Networking
1 4 (1.30%) Internet
2 3 (0.97%) Kerberos
3 2 (0.65%) DAB
3 2 (0.65%) SAN
4 1 (0.32%) CDMA
4 1 (0.32%) FTP
4 1 (0.32%) GSM
4 1 (0.32%) LDAP
4 1 (0.32%) LTE
4 1 (0.32%) Network Security
4 1 (0.32%) UMTS
Database & Business Intelligence
1 308 (100.00%) Big Data
2 245 (79.55%) Hadoop
3 105 (34.09%) Apache Hive
4 94 (30.52%) NoSQL
5 75 (24.35%) HBase
6 53 (17.21%) MongoDB
7 49 (15.91%) Apache Cassandra
8 45 (14.61%) Sqoop
9 37 (12.01%) Data Warehouse
10 32 (10.39%) Impala
11 26 (8.44%) Relational Database
12 21 (6.82%) SQL Server
13 17 (5.52%) Data Lake
13 17 (5.52%) Flink
14 16 (5.19%) Redis
15 14 (4.55%) RDBMS
16 12 (3.90%) PostgreSQL
16 12 (3.90%) Power BI
17 10 (3.25%) Data Mining
17 10 (3.25%) MySQL
Development Applications
1 24 (7.79%) Jenkins
1 24 (7.79%) Maven
2 21 (6.82%) Git (software)
3 7 (2.27%) JIRA
4 6 (1.95%) Cucumber
4 6 (1.95%) Sonatype Nexus
5 5 (1.62%) JUnit
5 5 (1.62%) Talend Open Studio
5 5 (1.62%) TeamCity
6 4 (1.30%) Artifactory
6 4 (1.30%) Selenium
6 4 (1.30%) Travis CI
6 4 (1.30%) Vagrant
7 3 (0.97%) ProC
7 3 (0.97%) Team Foundation Server
7 3 (0.97%) Visual Studio
8 2 (0.65%) GNU Toolchain
8 2 (0.65%) Gradle
8 2 (0.65%) Mercurial
8 2 (0.65%) Subversion
General
1 40 (12.99%) Finance
2 33 (10.71%) Retail
3 26 (8.44%) Banking
4 16 (5.19%) Telecoms
5 13 (4.22%) Marketing
6 9 (2.92%) Legal
7 8 (2.60%) Electronics
7 8 (2.60%) Investment Banking
8 7 (2.27%) Manufacturing
8 7 (2.27%) Publishing
9 6 (1.95%) Games
9 6 (1.95%) Pharmaceutical
10 5 (1.62%) Advertising
11 2 (0.65%) Law
12 1 (0.32%) Financial Institution
12 1 (0.32%) iGaming
12 1 (0.32%) Military
12 1 (0.32%) Retail Banking
Libraries, Frameworks & Software Standards
1 94 (30.52%) Kafka
2 16 (5.19%) Spark Streaming
3 13 (4.22%) Natural Language Processing
4 12 (3.90%) JSON
5 11 (3.57%) XML
6 10 (3.25%) RabbitMQ
6 10 (3.25%) Spring
7 7 (2.27%) REST
8 6 (1.95%) HTML
8 6 (1.95%) Node.js
8 6 (1.95%) Spring Boot
9 5 (1.62%) Web Services
10 4 (1.30%) CSS
10 4 (1.30%) GeoTools
10 4 (1.30%) J2EE
10 4 (1.30%) JMS
10 4 (1.30%) JTS
10 4 (1.30%) Pandas
10 4 (1.30%) React
10 4 (1.30%) RESTful
Miscellaneous
1 56 (18.18%) Analytical Skills
2 33 (10.71%) Computer Science
3 25 (8.12%) Greenfield Project
4 21 (6.82%) Fintech
5 16 (5.19%) Algorithms
6 13 (4.22%) Data Structures
7 5 (1.62%) Enterprise Software
8 4 (1.30%) Data Centre
9 3 (0.97%) Hedge funds
9 3 (0.97%) Internet of Things
9 3 (0.97%) Market Risk
9 3 (0.97%) Online Games
10 2 (0.65%) Credit Risk
10 2 (0.65%) Derivative
10 2 (0.65%) Equities
10 2 (0.65%) Foreign Exchange (FX)
10 2 (0.65%) User Experience
10 2 (0.65%) Virtual Team
11 1 (0.32%) Linux Command Line
11 1 (0.32%) mHealth
Operating Systems
1 56 (18.18%) Linux
2 29 (9.42%) Unix
3 12 (3.90%) Windows
4 4 (1.30%) Mac OS X
5 3 (0.97%) Ubuntu
6 2 (0.65%) CentOS
7 1 (0.32%) Android
7 1 (0.32%) DC/OS
7 1 (0.32%) Red Hat Enterprise Linux
7 1 (0.32%) Solaris
Processes & Methodologies
1 157 (50.97%) Agile Software Development
2 107 (34.74%) Analytics
3 67 (21.75%) Machine Learning
4 63 (20.45%) Software Engineering
5 59 (19.16%) ETL
5 59 (19.16%) TDD
6 52 (16.88%) DevOps
7 46 (14.94%) Business Intelligence
7 46 (14.94%) Data Analytics
8 44 (14.29%) Open Source
9 39 (12.66%) MapReduce
10 34 (11.04%) Scrum
11 33 (10.71%) Big Data Analytics
12 30 (9.74%) Data Engineering
13 29 (9.42%) Continuous Delivery
14 26 (8.44%) Continuous Integration
14 26 (8.44%) Kanban
15 25 (8.12%) Data Architecture
16 22 (7.14%) Data Modelling
16 22 (7.14%) Test Automation
Programming Languages
1 173 (56.17%) Java
2 153 (49.68%) Python
3 147 (47.73%) Scala
4 73 (23.70%) SQL
5 34 (11.04%) R
6 29 (9.42%) C
7 19 (6.17%) Bash Shell
8 14 (4.55%) Go
8 14 (4.55%) JavaScript
9 9 (2.92%) C++
10 8 (2.60%) MATLAB
10 8 (2.60%) Perl
11 7 (2.27%) C#
12 6 (1.95%) Shell Script
13 5 (1.62%) HiveQL
13 5 (1.62%) Ruby
14 4 (1.30%) PowerShell
15 3 (0.97%) PHP
15 3 (0.97%) PL/SQL
15 3 (0.97%) PROC SQL
Qualifications
1 43 (13.96%) Degree
2 28 (9.09%) DV Cleared
2 28 (9.09%) Security Cleared
3 8 (2.60%) Computer Science Degree
3 8 (2.60%) Master's Degree
4 7 (2.27%) PhD
4 7 (2.27%) SC Cleared
Quality Assurance & Compliance
1 10 (3.25%) QA
2 7 (2.27%) Data Quality
3 4 (1.30%) SLA
4 3 (0.97%) GCP
5 1 (0.32%) Actionable Recommendations
System Software
1 59 (19.16%) Apache Flume
2 45 (14.61%) Docker
3 37 (12.01%) HDFS
4 12 (3.90%) Apache ZooKeeper
5 6 (1.95%) Virtual Machines
Systems Management
1 27 (8.77%) Hadoop YARN
2 21 (6.82%) Terraform
3 19 (6.17%) Oozie
4 18 (5.84%) Opscode Chef
5 16 (5.19%) Puppet
6 14 (4.55%) Mesos
7 12 (3.90%) Kubernetes
8 9 (2.92%) Ansible
9 3 (0.97%) Ambari
9 3 (0.97%) Packer
10 2 (0.65%) Kibana
11 1 (0.32%) Marathon
Vendors
1 60 (19.48%) Cloudera
2 28 (9.09%) Hortonworks
3 23 (7.47%) MapR
3 23 (7.47%) Oracle
4 21 (6.82%) Tableau
5 16 (5.19%) Google
6 13 (4.22%) Teradata
7 11 (3.57%) Microsoft
7 11 (3.57%) SAP
8 8 (2.60%) Atlassian
8 8 (2.60%) Informatica
9 7 (2.27%) Splunk
10 5 (1.62%) Talend
11 2 (0.65%) Databricks
11 2 (0.65%) Greenplum
11 2 (0.65%) IBM
11 2 (0.65%) MuleSoft
11 2 (0.65%) Red Hat
12 1 (0.32%) MicroStrategy
12 1 (0.32%) Netezza