Period
to 19 September 2017

The median annual salary for a Big Data Engineer was £70,000 in advertised job vacancies during the 6 months to 19 September 2017.

The first table below provides salary benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Big Data Engineer
UK
6 months to
19 Sep 2017
Same period 2016 Same period 2015
Rank 918 1040 1109
Rank change year-on-year +122 +69 -12
Permanent jobs requiring a Big Data Engineer 263 209 202
As % of all permanent IT jobs advertised in the UK 0.15% 0.11% 0.092%
As % of the Job Titles category 0.15% 0.11% 0.095%
Number of salaries quoted 213 169 172
UK median annual salary £70,000 £62,500 £65,000
Median salary % change year-on-year +12.00% -3.85% -
10th Percentile £50,000 £42,500 £40,250
90th Percentile £88,500 £96,250 £85,000
UK excluding London median annual salary £63,000 £50,000 £42,500
% change year-on-year +26.00% +17.65% -15.00%

The following table is for comparison with the above and includes summary statistics for all permanent IT job vacancies. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of permanent jobs in our overall sample.

All Permanent IT Job Vacancies
UK
Permanent vacancies in the UK with a recognized job title 169773 187120 211546
% of permanent IT jobs with a recognized job title 95.82% 96.47% 96.45%
Number of salaries quoted 136138 157303 175314
UK median annual salary £47,500 £47,500 £45,000
Median salary % change year-on-year - +5.56% -
10th Percentile £27,500 £26,750 £26,250
90th Percentile £78,750 £77,500 £75,000
UK excluding London median annual salary £42,500 £42,500 £40,000
% change year-on-year - +6.25% -

Big Data Engineer
Job Vacancy Trend

Job postings that featured Big Data Engineer in the job title as a percentage of all IT jobs advertised.

Job vacancy trend for Big Data Engineer in the UK

Big Data Engineer
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Big Data Engineer.

Salary trend for Big Data Engineer in the UK

Big Data Engineer
Salary Histogram

The salary distribution of IT jobs citing Big Data Engineer over the 6 months to 19 September 2017.

Salary histogram for Big Data Engineer in the UK

Big Data Engineer
Top 13 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Big Data Engineer within the UK over the 6 months to 19 September 2017. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +114 261 £70,000 +7.69% 31
London +57 188 £70,000 -3.45% 19
UK excluding London +70 67 £63,000 +26.00% 13
South East +87 44 £65,000 +30.00% 6
South West +2 11 £57,500 +6.98% 1
East of England +15 5 £60,000 +4.35% 4
North of England -17 4 £82,500 +61.76%
North West -15 3 £75,000 +47.06%
Midlands +31 2 £45,000 -1.64% 1
East Midlands +28 1 £55,000 +20.22%
Scotland -3 1 £57,500 +27.78%
West Midlands - 1 £35,000 - 1
North East - 1 £90,000 -

Big Data Engineer Skill Set
Top 30 Co-occurring IT Skills

For the 6 months to 19 September 2017, Big Data Engineer job roles required the following IT skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all job ads featuring Big Data Engineer in the job title.

1 263 (100.00%) Big Data
2 201 (76.43%) Hadoop
3 199 (75.67%) Apache Spark
4 166 (63.12%) Java
5 145 (55.13%) Agile Software Development
6 129 (49.05%) Scala
7 120 (45.63%) Python
8 97 (36.88%) Analytics
9 87 (33.08%) NoSQL
10 84 (31.94%) Apache Hive
11 75 (28.52%) Kafka
12 66 (25.10%) ETL
13 63 (23.95%) SQL
13 63 (23.95%) Software Engineering
14 60 (22.81%) Amazon AWS
15 54 (20.53%) TDD
16 53 (20.15%) Cloudera
17 49 (18.63%) Machine Learning
17 49 (18.63%) Scrum
18 48 (18.25%) MongoDB
18 48 (18.25%) HBase
19 45 (17.11%) Open Source
20 38 (14.45%) DevOps
21 37 (14.07%) Docker
21 37 (14.07%) MapReduce
22 35 (13.31%) HDFS
22 35 (13.31%) Analytical Skills
22 35 (13.31%) Business Intelligence
23 34 (12.93%) Data Warehouse
23 34 (12.93%) Kanban

Big Data Engineer Skill Set
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 199 (75.67%) Apache Spark
2 32 (12.17%) Apache
3 23 (8.75%) Apache Solr
4 20 (7.60%) Elasticsearch
5 11 (4.18%) OpenStack
6 10 (3.80%) SAS
7 9 (3.42%) Apache Pig
8 5 (1.90%) Confluence
9 3 (1.14%) Tomcat
10 2 (0.76%) Apache Airflow
10 2 (0.76%) GeoServer
10 2 (0.76%) Mule
11 1 (0.38%) Luigi
11 1 (0.38%) nginx
Applications
1 1 (0.38%) SPSS
1 1 (0.38%) SPSS Modeler
1 1 (0.38%) Weka
Cloud Services
1 60 (22.81%) Amazon AWS
2 20 (7.60%) Cloud Computing
2 20 (7.60%) Microsoft Azure
3 19 (7.22%) Google Cloud Platform
4 12 (4.56%) Amazon EC2
4 12 (4.56%) Amazon S3
5 5 (1.90%) Azure Data Factory
5 5 (1.90%) SaaS
6 4 (1.52%) Amazon EMR
6 4 (1.52%) IaaS
6 4 (1.52%) OpenShift
6 4 (1.52%) PaaS
7 2 (0.76%) Amazon Kinesis
7 2 (0.76%) GitHub
8 1 (0.38%) AWS CloudFormation
8 1 (0.38%) AWS Elastic Beanstalk
8 1 (0.38%) AWS Lambda
8 1 (0.38%) CloudBees
8 1 (0.38%) Cortana Intelligence Suite
Communications & Networking
1 6 (2.28%) Internet
2 4 (1.52%) Kerberos
2 4 (1.52%) Wireless
3 1 (0.38%) CDMA
3 1 (0.38%) Firewall
3 1 (0.38%) FTP
3 1 (0.38%) GSM
3 1 (0.38%) LDAP
3 1 (0.38%) LTE
3 1 (0.38%) MQTT
3 1 (0.38%) Network Security
3 1 (0.38%) UMTS
Database & Business Intelligence
1 263 (100.00%) Big Data
2 201 (76.43%) Hadoop
3 87 (33.08%) NoSQL
4 84 (31.94%) Apache Hive
5 48 (18.25%) HBase
5 48 (18.25%) MongoDB
6 34 (12.93%) Data Warehouse
7 32 (12.17%) Relational Database
8 30 (11.41%) Sqoop
9 28 (10.65%) Apache Cassandra
10 20 (7.60%) PostgreSQL
11 19 (7.22%) RDBMS
12 16 (6.08%) Impala
13 14 (5.32%) SQL Server
14 13 (4.94%) Redis
15 11 (4.18%) Data Lake
16 9 (3.42%) Data Mining
17 8 (3.04%) Amazon Redshift
17 8 (3.04%) BigQuery
17 8 (3.04%) MySQL
Development Applications
1 19 (7.22%) Jenkins
2 18 (6.84%) Git (software)
3 14 (5.32%) Maven
4 12 (4.56%) JIRA
5 8 (3.04%) Sonatype Nexus
6 7 (2.66%) Talend Open Studio
7 4 (1.52%) Bitbucket
7 4 (1.52%) Cucumber
7 4 (1.52%) JUnit
7 4 (1.52%) Selenium
8 2 (0.76%) GNU Toolchain
8 2 (0.76%) Mercurial
8 2 (0.76%) Subversion
8 2 (0.76%) Travis CI
9 1 (0.38%) FishEye
9 1 (0.38%) Mockito
9 1 (0.38%) ProC
9 1 (0.38%) SonarQube
9 1 (0.38%) Valgrind
General
1 31 (11.79%) Finance
2 24 (9.13%) Retail
3 21 (7.98%) Banking
4 11 (4.18%) Legal
5 9 (3.42%) Games
6 8 (3.04%) Investment Banking
6 8 (3.04%) Marketing
7 7 (2.66%) Pharmaceutical
7 7 (2.66%) Publishing
8 6 (2.28%) Advertising
9 5 (1.90%) Electronics
9 5 (1.90%) Telecoms
10 3 (1.14%) Manufacturing
11 1 (0.38%) Financial Institution
11 1 (0.38%) Front Office
11 1 (0.38%) iGaming
11 1 (0.38%) Retail Banking
Libraries, Frameworks & Software Standards
1 75 (28.52%) Kafka
2 12 (4.56%) JSON
3 11 (4.18%) Spring
4 10 (3.80%) REST
5 9 (3.42%) RabbitMQ
5 9 (3.42%) Web Services
5 9 (3.42%) XML
6 7 (2.66%) Drools
6 7 (2.66%) Natural Language Processing
7 6 (2.28%) Spring Boot
8 5 (1.90%) MLlib
8 5 (1.90%) RESTful
8 5 (1.90%) Spark Streaming
9 4 (1.52%) AngularJS
9 4 (1.52%) J2EE
10 3 (1.14%) ActiveMQ
10 3 (1.14%) Apache Camel
10 3 (1.14%) JMS
11 2 (0.76%) JTS
11 2 (0.76%) LAMP
Miscellaneous
1 35 (13.31%) Analytical Skills
2 27 (10.27%) Greenfield Project
3 25 (9.51%) Computer Science
4 18 (6.84%) Algorithms
5 15 (5.70%) Fintech
6 7 (2.66%) Data Structures
7 6 (2.28%) Virtual Team
8 4 (1.52%) Enterprise Software
8 4 (1.52%) Social Media
9 3 (1.14%) Grid Computing
9 3 (1.14%) Internet of Things
9 3 (1.14%) Online Games
10 2 (0.76%) Credit Risk
10 2 (0.76%) Market Risk
11 1 (0.38%) Blog
11 1 (0.38%) Derivative
11 1 (0.38%) Management Information System
11 1 (0.38%) mHealth
11 1 (0.38%) Reference Data
11 1 (0.38%) Replication
Operating Systems
1 33 (12.55%) Linux
2 8 (3.04%) Unix
3 7 (2.66%) Windows
4 4 (1.52%) Mac OS X
5 2 (0.76%) Ubuntu
6 1 (0.38%) Android
6 1 (0.38%) CentOS
6 1 (0.38%) Red Hat Enterprise Linux
Processes & Methodologies
1 145 (55.13%) Agile Software Development
2 97 (36.88%) Analytics
3 66 (25.10%) ETL
4 63 (23.95%) Software Engineering
5 54 (20.53%) TDD
6 49 (18.63%) Machine Learning
6 49 (18.63%) Scrum
7 45 (17.11%) Open Source
8 38 (14.45%) DevOps
9 37 (14.07%) MapReduce
10 35 (13.31%) Business Intelligence
11 34 (12.93%) Kanban
12 30 (11.41%) Data Analytics
13 27 (10.27%) Continuous Integration
14 23 (8.75%) Data Engineering
15 22 (8.37%) Big Data Analytics
15 22 (8.37%) Data Modelling
16 21 (7.98%) Data Architecture
17 19 (7.22%) E-Commerce
18 16 (6.08%) Test Automation
Programming Languages
1 166 (63.12%) Java
2 129 (49.05%) Scala
3 120 (45.63%) Python
4 63 (23.95%) SQL
5 31 (11.79%) R
6 21 (7.98%) C
7 13 (4.94%) Go
8 12 (4.56%) JavaScript
9 10 (3.80%) C#
9 10 (3.80%) C++
10 7 (2.66%) Bash Shell
11 6 (2.28%) Perl
12 4 (1.52%) Shell Script
13 3 (1.14%) HiveQL
13 3 (1.14%) MATLAB
13 3 (1.14%) PL/SQL
13 3 (1.14%) Ruby
14 1 (0.38%) Erlang
14 1 (0.38%) Groovy
14 1 (0.38%) PROC SQL
Qualifications
1 34 (12.93%) Degree
2 20 (7.60%) Security Cleared
3 19 (7.22%) DV Cleared
4 10 (3.80%) Master's Degree
5 9 (3.42%) SC Cleared
6 7 (2.66%) Computer Science Degree
7 5 (1.90%) PhD
8 2 (0.76%) EMC Certification
Quality Assurance & Compliance
1 7 (2.66%) QA
2 6 (2.28%) Data Quality
3 3 (1.14%) GCP
4 2 (0.76%) SLA
5 1 (0.38%) Actionable Recommendations
System Software
1 37 (14.07%) Docker
2 35 (13.31%) HDFS
3 33 (12.55%) Apache Flume
4 6 (2.28%) Apache ZooKeeper
5 2 (0.76%) Virtual Machines
6 1 (0.38%) Cronacle
6 1 (0.38%) etcd
6 1 (0.38%) flannel
6 1 (0.38%) Oracle RAC
Systems Management
1 16 (6.08%) Mesos
2 14 (5.32%) Opscode Chef
2 14 (5.32%) Terraform
3 13 (4.94%) Kubernetes
4 12 (4.56%) Oozie
5 11 (4.18%) Ansible
6 10 (3.80%) Puppet
7 6 (2.28%) Hadoop YARN
8 4 (1.52%) Packer
9 3 (1.14%) Ambari
10 1 (0.38%) Control-M
10 1 (0.38%) Salt
Vendors
1 53 (20.15%) Cloudera
2 31 (11.79%) Hortonworks
3 22 (8.37%) MapR
4 18 (6.84%) Google
5 17 (6.46%) Oracle
6 16 (6.08%) Tableau
7 10 (3.80%) Informatica
8 9 (3.42%) Teradata
9 7 (2.66%) Red Hat
9 7 (2.66%) Talend
10 6 (2.28%) Microsoft
11 4 (1.52%) Alias
11 4 (1.52%) Netezza
11 4 (1.52%) SAP
11 4 (1.52%) Vertica
12 3 (1.14%) Atlassian
12 3 (1.14%) MicroStrategy
12 3 (1.14%) Splunk
13 2 (0.76%) EMC
13 2 (0.76%) Intel