Period
to 14 December 2018

The following table provides summary statistics for permanent job vacancies with a requirement for Data Ingestion skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Ingestion over the 6 months to 14 December 2018 with a comparison to the same period in the previous 2 years.

Data Ingestion
UK
6 months to
14 Dec 2018
Same period 2017 Same period 2016
Rank 964 1098 1165
Rank change year-on-year +134 +67 +141
Permanent jobs citing Data Ingestion 176 108 53
As % of all permanent IT jobs advertised in the UK 0.11% 0.060% 0.029%
As % of the Processes & Methodologies category 0.12% 0.065% 0.031%
Number of salaries quoted 131 88 40
UK median annual salary £70,000 £70,000 £65,000
Median salary % change year-on-year - +7.69% -
10th Percentile £50,000 £52,500 £49,875
90th Percentile £115,000 £91,500 £115,000
UK excluding London median annual salary £65,000 £70,000 £65,000
% change year-on-year -7.14% +7.69% -21.21%

Data Ingestion is in the Processes and Methodologies category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for process or methodology skills.

All Process and Methodology Skills
UK
Permanent vacancies with a requirement for process or methodology skills 145,877 165,074 169,374
As % of all permanent IT jobs advertised in the UK 91.58% 91.89% 91.28%
Number of salaries quoted 115,394 131,109 140,507
UK median annual salary £52,500 £50,000 £48,500
Median salary % change year-on-year +5.00% +3.09% +2.11%
10th Percentile £29,750 £28,250 £27,696
90th Percentile £85,000 £80,000 £78,750
UK excluding London median annual salary £45,000 £43,500 £42,500
% change year-on-year +3.45% +2.35% -

Data Ingestion
Job Vacancy Trend

Job postings citing Data Ingestion as a percentage of all IT jobs advertised.

Job vacancy trend for Data Ingestion in the UK

Data Ingestion
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Data Ingestion.

Salary trend for Data Ingestion in the UK

Data Ingestion
Salary Histogram

The salary distribution of IT jobs citing Data Ingestion over the 6 months to 14 December 2018.

Salary histogram for Data Ingestion in the UK

Data Ingestion
Top 11 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Ingestion within the UK over the 6 months to 14 December 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +146 171 £70,000 - 31
London +99 98 £87,500 +18.64% 23
UK excluding London +81 77 £65,000 -7.14% 8
North of England +71 40 £70,000 +64.71%
North West +60 40 £70,000 +64.71%
South East +39 18 £65,000 +13.04% 3
West Midlands +21 9 £60,000 -14.29% 2
Midlands +11 9 £60,000 -14.29% 2
Scotland - 5 - -
East of England +31 4 £45,000 +5.88%
South West - 1 £55,000 - 3

For the 6 months to 14 December 2018, IT jobs citing Data Ingestion also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Data Ingestion.

1 114 (64.77%) Python
2 109 (61.93%) Apache Spark
3 107 (60.80%) Agile Software Development
4 103 (58.52%) Big Data
5 101 (57.39%) Hadoop
6 97 (55.11%) Amazon AWS
7 96 (54.55%) Analytics
8 95 (53.98%) Java
9 79 (44.89%) Kafka
10 78 (44.32%) Scala
11 74 (42.05%) NoSQL
11 74 (42.05%) Data Modelling
12 68 (38.64%) Data Engineering
13 64 (36.36%) DevOps
14 62 (35.23%) Microsoft Azure
15 61 (34.66%) Open Source
16 60 (34.09%) Computer Science
17 59 (33.52%) Data Architecture
18 58 (32.95%) Degree
18 58 (32.95%) Apache
19 57 (32.39%) Data Warehouse
19 57 (32.39%) MongoDB
20 53 (30.11%) Mathematics
21 50 (28.41%) Amazon Redshift
22 49 (27.84%) ETL
22 49 (27.84%) Microsoft
23 48 (27.27%) Apache Hive
23 48 (27.27%) Machine Learning
24 45 (25.57%) HBase
25 44 (25.00%) Configuration Management

Data Ingestion
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 109 (61.93%) Apache Spark
2 58 (32.95%) Apache
3 15 (8.52%) Apache Solr
4 13 (7.39%) Apache Pig
5 10 (5.68%) Elasticsearch
6 9 (5.11%) SAS
7 3 (1.70%) Oracle Workflow
7 3 (1.70%) Tomcat
7 3 (1.70%) WebLogic
8 1 (0.57%) Apache Airflow
8 1 (0.57%) Jupyter
Applications
1 2 (1.14%) Microsoft Excel
2 1 (0.57%) Microsoft PowerPoint
2 1 (0.57%) Microsoft Project
2 1 (0.57%) MS Visio
Business Applications
1 6 (3.41%) SAP HR
2 1 (0.57%) Eloqua
2 1 (0.57%) Oracle AML
Cloud Services
1 97 (55.11%) Amazon AWS
2 62 (35.23%) Microsoft Azure
3 42 (23.86%) Amazon EC2
3 42 (23.86%) Amazon EMR
3 42 (23.86%) Cloud Computing
4 33 (18.75%) Serverless
5 27 (15.34%) Google Cloud Platform
5 27 (15.34%) IaaS
5 27 (15.34%) PaaS
6 18 (10.23%) AWS Lambda
7 16 (9.09%) SaaS
8 8 (4.55%) Azure Data Factory
9 6 (3.41%) Amazon SQS
9 6 (3.41%) Cortana Intelligence Suite
10 4 (2.27%) GitHub
11 3 (1.70%) Azure Machine Learning
12 1 (0.57%) Amazon S3
12 1 (0.57%) Data Lake Analytics
Communications & Networking
1 18 (10.23%) Internet
2 4 (2.27%) HTTP
3 1 (0.57%) Kerberos
Database & Business Intelligence
1 103 (58.52%) Big Data
2 101 (57.39%) Hadoop
3 74 (42.05%) NoSQL
4 57 (32.39%) Data Warehouse
4 57 (32.39%) MongoDB
5 50 (28.41%) Amazon Redshift
6 48 (27.27%) Apache Hive
7 45 (25.57%) HBase
8 42 (23.86%) Data Lake
9 39 (22.16%) Flink
10 38 (21.59%) Amazon Athena
10 38 (21.59%) Apache Cassandra
11 33 (18.75%) Impala
12 25 (14.20%) RDBMS
12 25 (14.20%) Relational Database
13 21 (11.93%) SQL Server
14 13 (7.39%) MySQL
15 11 (6.25%) PostgreSQL
15 11 (6.25%) QlikView
15 11 (6.25%) Spotfire
Development Applications
1 43 (24.43%) JIRA
2 13 (7.39%) Git (software)
3 9 (5.11%) Jenkins
4 8 (4.55%) Eclipse
4 8 (4.55%) ERwin
5 7 (3.98%) Subversion
5 7 (3.98%) Visual Studio
6 6 (3.41%) Team Foundation Server
7 5 (2.84%) Maven
8 4 (2.27%) Axure RP
8 4 (2.27%) FindBugs
8 4 (2.27%) IntelliJ
8 4 (2.27%) JUnit
9 2 (1.14%) GitLab
9 2 (1.14%) TeamCity
10 1 (0.57%) Mercurial
General
1 33 (18.75%) Finance
2 21 (11.93%) Banking
3 12 (6.82%) Retail
4 9 (5.11%) Law
4 9 (5.11%) Marketing
5 8 (4.55%) Advertising
6 6 (3.41%) Military
7 4 (2.27%) Telecoms
8 3 (1.70%) Investment Banking
9 1 (0.57%) Financial Institution
Job Titles
1 64 (36.36%) Data Engineer
2 61 (34.66%) Architect
3 45 (25.57%) Data Architect
4 38 (21.59%) Big Data Engineer
5 30 (17.05%) Developer
6 26 (14.77%) Senior Architect
7 25 (14.20%) Senior Data Architect
7 25 (14.20%) Senior Data Engineer
8 17 (9.66%) Big Data Architect
9 15 (8.52%) AWS Architect
9 15 (8.52%) AWS Solutions Architect
9 15 (8.52%) Azure Architect
9 15 (8.52%) Cloud Architect
9 15 (8.52%) Cloud Solutions Architect
10 14 (7.95%) Big Data Developer
11 12 (6.82%) Solutions Architect
12 9 (5.11%) Analytics Consultant
12 9 (5.11%) Consultant
13 8 (4.55%) Data Analytics Consultant
13 8 (4.55%) Data Consultant
Libraries, Frameworks & Software Standards
1 79 (44.89%) Kafka
2 44 (25.00%) Web Services
3 43 (24.43%) Spark Streaming
4 10 (5.68%) JSON
5 8 (4.55%) Elastic Stack
5 8 (4.55%) HTML
6 7 (3.98%) XML
7 6 (3.41%) Apache Avro
7 6 (3.41%) Natural Language Processing
7 6 (3.41%) RESTful
8 5 (2.84%) AngularJS
8 5 (2.84%) CSS
9 4 (2.27%) HTML5
9 4 (2.27%) jQuery
9 4 (2.27%) React
9 4 (2.27%) Spring
10 3 (1.70%) J2EE
10 3 (1.70%) JRuby
10 3 (1.70%) scikit-learn
11 2 (1.14%) RabbitMQ
Miscellaneous
1 24 (13.64%) Analytical Skills
2 18 (10.23%) Internet of Things
3 17 (9.66%) Public Cloud
4 11 (6.25%) Algorithms
5 10 (5.68%) Enterprise Software
6 9 (5.11%) Reference Data
7 8 (4.55%) Hybrid Cloud
8 6 (3.41%) Smart City
9 4 (2.27%) Mainframe
10 2 (1.14%) Cloud Native
10 2 (1.14%) Data Structures
10 2 (1.14%) Fintech
11 1 (0.57%) Infotainment
11 1 (0.57%) Linux Command Line
11 1 (0.57%) Management Information System
11 1 (0.57%) PMI
11 1 (0.57%) Pricing Data
11 1 (0.57%) Private Cloud
11 1 (0.57%) Self-Motivation
11 1 (0.57%) User Experience
Operating Systems
1 22 (12.50%) Unix
2 6 (3.41%) Linux
3 4 (2.27%) Windows
4 1 (0.57%) DC/OS
Processes & Methodologies
1 107 (60.80%) Agile Software Development
2 96 (54.55%) Analytics
3 74 (42.05%) Data Modelling
4 68 (38.64%) Data Engineering
5 64 (36.36%) DevOps
6 61 (34.66%) Open Source
7 60 (34.09%) Computer Science
8 59 (33.52%) Data Architecture
9 53 (30.11%) Mathematics
10 49 (27.84%) ETL
11 48 (27.27%) Machine Learning
12 44 (25.00%) Configuration Management
13 42 (23.86%) Data Analytics
14 40 (22.73%) Change Management
15 39 (22.16%) TDD
16 31 (17.61%) Data Science
17 30 (17.05%) Data Analysis
18 24 (13.64%) Data Management
18 24 (13.64%) Software Engineering
19 23 (13.07%) Artificial Intelligence
Programming Languages
1 114 (64.77%) Python
2 95 (53.98%) Java
3 78 (44.32%) Scala
4 36 (20.45%) SQL
5 15 (8.52%) Shell Script
6 14 (7.95%) R
7 8 (4.55%) C
7 8 (4.55%) C++
7 8 (4.55%) Go
8 7 (3.98%) JavaScript
9 4 (2.27%) Clojure
9 4 (2.27%) MATLAB
10 3 (1.70%) Groovy
10 3 (1.70%) HiveQL
10 3 (1.70%) Jython
10 3 (1.70%) Kotlin
10 3 (1.70%) PL/SQL
11 1 (0.57%) C#
11 1 (0.57%) Cypher
Qualifications
1 58 (32.95%) Degree
2 42 (23.86%) Computer Science Degree
3 39 (22.16%) Numerate Degree
4 15 (8.52%) AWS Certification
5 1 (0.57%) PhD
5 1 (0.57%) PMI Certification
Quality Assurance & Compliance
1 13 (7.39%) QA
2 10 (5.68%) Data Quality
3 3 (1.70%) GDPR
4 2 (1.14%) PCI DSS
System Software
1 32 (18.18%) HDFS
2 14 (7.95%) Apache Flume
3 5 (2.84%) Apache ZooKeeper
4 4 (2.27%) Virtual Machines
5 2 (1.14%) Docker
Systems Management
1 15 (8.52%) Oozie
2 9 (5.11%) Kibana
3 8 (4.55%) Hadoop YARN
3 8 (4.55%) Mesos
4 7 (3.98%) Opscode Chef
4 7 (3.98%) Puppet
4 7 (3.98%) Terraform
5 4 (2.27%) Kubernetes
5 4 (2.27%) Rundeck
6 1 (0.57%) Ansible
Vendors
1 49 (27.84%) Microsoft
2 38 (21.59%) CA Agile Central
3 30 (17.05%) Google
4 24 (13.64%) Cloudera
5 23 (13.07%) Oracle
6 20 (11.36%) Cognizant
7 18 (10.23%) Tableau
8 16 (9.09%) Talend
9 10 (5.68%) Databricks
9 10 (5.68%) Hortonworks
9 10 (5.68%) Salesforce.com
9 10 (5.68%) SAP
10 9 (5.11%) IBM
11 7 (3.98%) Ab Initio
12 3 (1.70%) Informatica
12 3 (1.70%) MapR
13 1 (0.57%) MicroStrategy