Period
to 20 June 2019

The following table provides summary statistics for permanent job vacancies with a requirement for Data Ingestion skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Ingestion over the 6 months to 20 June 2019 with a comparison to the same period in the previous 2 years.

Data Ingestion
UK
6 months to
20 Jun 2019
Same period 2018 Same period 2017
Rank 857 1081 1114
Rank change year-on-year +224 +33 +142
Permanent jobs citing Data Ingestion 236 114 77
As % of all permanent IT jobs advertised in the UK 0.16% 0.066% 0.045%
As % of the Processes & Methodologies category 0.18% 0.072% 0.049%
Number of salaries quoted 171 73 64
UK median annual salary £65,000 £75,000 £65,000
Median salary % change year-on-year -13.33% +15.38% +18.18%
10th Percentile £45,000 £45,500 £52,875
90th Percentile £112,500 £116,000 £116,250
UK excluding London median annual salary £65,000 £57,500 £50,000
% change year-on-year +13.04% +15.00% -

Data Ingestion is in the Processes and Methodologies category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for process or methodology skills.

All Process and Methodology Skills
UK
Permanent vacancies with a requirement for process or methodology skills 132,276 158,555 158,247
As % of all permanent IT jobs advertised in the UK 92.04% 92.00% 91.76%
Number of salaries quoted 103,739 127,602 128,957
UK median annual salary £52,500 £50,000 £47,500
Median salary % change year-on-year +5.00% +5.26% -
10th Percentile £30,000 £29,000 £27,750
90th Percentile £86,250 £82,500 £78,750
UK excluding London median annual salary £45,000 £45,000 £42,500
% change year-on-year - +5.88% -

Data Ingestion
Job Vacancy Trend

Job postings citing Data Ingestion as a percentage of all IT jobs advertised.

Job vacancy trend for Data Ingestion in the UK

Data Ingestion
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Data Ingestion.

Salary trend for Data Ingestion in the UK

Data Ingestion
Salary Histogram

The salary distribution of IT jobs citing Data Ingestion over the 6 months to 20 June 2019.

Salary histogram for Data Ingestion in the UK

Data Ingestion
Top 11 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Ingestion within the UK over the 6 months to 20 June 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +233 232 £65,000 -13.33% 10
London +156 161 £65,750 -17.81% 6
UK excluding London +126 70 £65,000 +13.04% 4
South East +54 27 £57,500 -11.54% 2
South West +36 18 £65,000 +30.00% 1
North West +40 17 £65,000 +30.00% 1
North of England +24 17 £65,000 +30.00% 1
Scotland - 4 £56,250 -
East of England - 3 £57,500 -
Midlands +48 1 £65,000 -
West Midlands +34 1 £65,000 -

For the 6 months to 20 June 2019, IT jobs citing Data Ingestion also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Data Ingestion.

1 150 (63.56%) Analytics
2 140 (59.32%) SQL
3 133 (56.36%) Hadoop
4 130 (55.08%) Big Data
5 129 (54.66%) Python
6 128 (54.24%) Agile Software Development
7 117 (49.58%) ETL
8 115 (48.73%) Apache Spark
9 112 (47.46%) Data Warehouse
10 84 (35.59%) Data Lake
10 84 (35.59%) Microsoft Azure
11 81 (34.32%) Finance
12 78 (33.05%) Kafka
13 77 (32.63%) Data Analytics
14 75 (31.78%) Business Intelligence
15 71 (30.08%) Amazon AWS
16 70 (29.66%) Problem-Solving
17 68 (28.81%) DevOps
17 68 (28.81%) Microsoft
18 63 (26.69%) Scala
19 62 (26.27%) Data Modelling
20 58 (24.58%) Java
21 55 (23.31%) SQL Server
21 55 (23.31%) Data Engineering
22 54 (22.88%) Scrum
22 54 (22.88%) NoSQL
23 52 (22.03%) Machine Learning
24 51 (21.61%) Apache Hive
25 49 (20.76%) Azure Data Factory
26 48 (20.34%) Google

Data Ingestion
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 115 (48.73%) Apache Spark
2 33 (13.98%) Apache
3 16 (6.78%) Apache Pig
4 10 (4.24%) SAS
5 9 (3.81%) Confluence
6 8 (3.39%) Elasticsearch
7 7 (2.97%) Apache Airflow
7 7 (2.97%) Apache Solr
8 4 (1.69%) Jupyter
9 2 (0.85%) Luigi
10 1 (0.42%) nginx
Applications
1 2 (0.85%) Microsoft Excel
1 2 (0.85%) Spreadsheet
1 2 (0.85%) SPSS
2 1 (0.42%) Microsoft Office
2 1 (0.42%) Microsoft PowerPoint
2 1 (0.42%) MS Visio
Business Applications
1 2 (0.85%) SAP ERP
Cloud Services
1 84 (35.59%) Microsoft Azure
2 71 (30.08%) Amazon AWS
3 49 (20.76%) Azure Data Factory
4 46 (19.49%) Google Cloud Platform
5 32 (13.56%) Power Platform
6 22 (9.32%) Serverless
7 17 (7.20%) Cortana Intelligence Suite
8 14 (5.93%) AWS Lambda
9 12 (5.08%) IaaS
9 12 (5.08%) PaaS
10 10 (4.24%) Amazon EMR
11 8 (3.39%) Amazon SQS
12 5 (2.12%) Amazon EC2
13 4 (1.69%) Cloud Computing
13 4 (1.69%) GitHub
13 4 (1.69%) Stackdriver
14 3 (1.27%) Azure Functions
14 3 (1.27%) SaaS
15 2 (0.85%) Amazon S3
15 2 (0.85%) Google App Engine
Communications & Networking
1 5 (2.12%) Internet
2 1 (0.42%) Kerberos
Database & Business Intelligence
1 133 (56.36%) Hadoop
2 130 (55.08%) Big Data
3 112 (47.46%) Data Warehouse
4 84 (35.59%) Data Lake
5 55 (23.31%) SQL Server
6 54 (22.88%) NoSQL
7 51 (21.61%) Apache Hive
8 44 (18.64%) Azure SQL Database
9 38 (16.10%) SQL Server Analysis Services
10 37 (15.68%) BigQuery
10 37 (15.68%) SQL Server Integration Services
10 37 (15.68%) SQL Server Reporting Services
11 32 (13.56%) Power BI
12 28 (11.86%) Oracle Reports
13 27 (11.44%) Apache Cassandra
13 27 (11.44%) HBase
13 27 (11.44%) Relational Database
14 26 (11.02%) Microsoft Analysis Services
14 26 (11.02%) MongoDB
15 24 (10.17%) Amazon Redshift
Development Applications
1 25 (10.59%) JIRA
2 14 (5.93%) Visual Studio
3 11 (4.66%) Jenkins
4 8 (3.39%) Git (software)
5 5 (2.12%) Mercurial
5 5 (2.12%) RStudio
6 4 (1.69%) Bitbucket Server
7 2 (0.85%) Cucumber
7 2 (0.85%) Maven
7 2 (0.85%) Spinnaker
8 1 (0.42%) Oracle APEX
8 1 (0.42%) Selenium
General
1 81 (34.32%) Finance
2 14 (5.93%) Banking
2 14 (5.93%) Retail
3 11 (4.66%) Games
3 11 (4.66%) Law
4 5 (2.12%) Automotive
4 5 (2.12%) Marketing
5 4 (1.69%) Telecoms
6 1 (0.42%) Financial Institution
Job Titles
1 87 (36.86%) Data Engineer
2 60 (25.42%) Developer
3 49 (20.76%) Architect
4 44 (18.64%) Data Architect
5 32 (13.56%) Big Data Engineer
6 26 (11.02%) Azure Developer
7 25 (10.59%) Business Intelligence Developer
7 25 (10.59%) SQL Developer
8 22 (9.32%) Database Developer
8 22 (9.32%) SQL BI Developer
8 22 (9.32%) SQL Database Developer
9 20 (8.47%) Senior Data Engineer
10 17 (7.20%) Lead Developer
11 15 (6.36%) Big Data Architect
11 15 (6.36%) Data Warehouse Developer
12 14 (5.93%) Big Data Developer
12 14 (5.93%) Oracle Developer
12 14 (5.93%) Python Engineer
13 13 (5.51%) Solutions Architect
14 12 (5.08%) Oracle Data Warehouse Developer
Libraries, Frameworks & Software Standards
1 78 (33.05%) Kafka
2 26 (11.02%) MDX
3 25 (10.59%) Apache NiFi
4 13 (5.51%) Natural Language Processing
4 13 (5.51%) Web Services
5 9 (3.81%) scikit-learn
6 8 (3.39%) JRuby
6 8 (3.39%) Pandas
7 7 (2.97%) JSON
7 7 (2.97%) Samza
8 6 (2.54%) SciPy
9 5 (2.12%) Apache Avro
9 5 (2.12%) HTML
9 5 (2.12%) REST
10 4 (1.69%) gRPC
10 4 (1.69%) Node.js
10 4 (1.69%) Spark Streaming
11 3 (1.27%) Matplotlib
11 3 (1.27%) NumPy
11 3 (1.27%) Regular Expression
Miscellaneous
1 47 (19.92%) Analytical Skills
2 31 (13.14%) Data Structures
3 12 (5.08%) Clustering
4 11 (4.66%) Algorithms
5 6 (2.54%) Greenfield Project
6 5 (2.12%) Hybrid Cloud
6 5 (2.12%) User Experience
7 3 (1.27%) Mobile App
7 3 (1.27%) Online Banking
7 3 (1.27%) Public Cloud
7 3 (1.27%) Reinsurance
7 3 (1.27%) Self-Motivation
8 2 (0.85%) Bitcoin
8 2 (0.85%) Cloud Native
8 2 (0.85%) Internet of Things
8 2 (0.85%) Management Information System
9 1 (0.42%) Credit Risk
9 1 (0.42%) FMCG
9 1 (0.42%) Life Science
Operating Systems
1 21 (8.90%) Unix
2 10 (4.24%) Linux
3 7 (2.97%) CentOS
3 7 (2.97%) Red Hat Enterprise Linux
3 7 (2.97%) Ubuntu
4 1 (0.42%) Windows
Processes & Methodologies
1 150 (63.56%) Analytics
2 128 (54.24%) Agile Software Development
3 117 (49.58%) ETL
4 77 (32.63%) Data Analytics
5 75 (31.78%) Business Intelligence
6 70 (29.66%) Problem-Solving
7 68 (28.81%) DevOps
8 62 (26.27%) Data Modelling
9 55 (23.31%) Data Engineering
10 54 (22.88%) Scrum
11 52 (22.03%) Machine Learning
12 44 (18.64%) Visualisation
13 43 (18.22%) Software Engineering
14 40 (16.95%) Data Architecture
15 38 (16.10%) Kimball Methodology
16 33 (13.98%) Data Management
17 32 (13.56%) Kanban
18 28 (11.86%) Data Analysis
18 28 (11.86%) Open Source
19 25 (10.59%) Data Science
Programming Languages
1 140 (59.32%) SQL
2 129 (54.66%) Python
3 63 (26.69%) Scala
4 58 (24.58%) Java
5 26 (11.02%) Data Analysis Expressions (DAX)
6 23 (9.75%) R
7 19 (8.05%) Shell Script
8 17 (7.20%) PL/SQL
9 14 (5.93%) C#
10 13 (5.51%) Clojure
11 8 (3.39%) Groovy
11 8 (3.39%) HiveQL
11 8 (3.39%) JavaScript
11 8 (3.39%) Jython
11 8 (3.39%) Kotlin
12 6 (2.54%) T-SQL
13 5 (2.12%) Cypher
14 2 (0.85%) C
15 1 (0.42%) Bash Shell
15 1 (0.42%) PHP
Qualifications
1 16 (6.78%) Degree
2 7 (2.97%) MCP
2 7 (2.97%) SC Cleared
2 7 (2.97%) Security Cleared
3 6 (2.54%) Computer Science Degree
4 4 (1.69%) Numerate Degree
5 3 (1.27%) AWS Certification
6 1 (0.42%) PhD
Quality Assurance & Compliance
1 27 (11.44%) Data Quality
2 12 (5.08%) ISO/IEC 27001
3 7 (2.97%) GDPR
4 4 (1.69%) PCI DSS
5 3 (1.27%) IFRS
5 3 (1.27%) MiFID
5 3 (1.27%) Solvency II
6 2 (0.85%) QA
System Software
1 23 (9.75%) HDFS
2 8 (3.39%) Apache Flume
3 7 (2.97%) Docker
Systems Management
1 22 (9.32%) Terraform
2 20 (8.47%) Puppet
3 18 (7.63%) Opscode Chef
4 11 (4.66%) Hadoop YARN
5 5 (2.12%) Chukwa
5 5 (2.12%) Kubernetes
6 2 (0.85%) Oozie
7 1 (0.42%) Ansible
7 1 (0.42%) Kibana
7 1 (0.42%) logstash
Vendors
1 68 (28.81%) Microsoft
2 48 (20.34%) Google
3 46 (19.49%) Databricks
4 36 (15.25%) Oracle
5 23 (9.75%) Tableau
6 22 (9.32%) Hortonworks
7 13 (5.51%) Talend
8 12 (5.08%) Teradata
9 11 (4.66%) Cloudera
10 9 (3.81%) IBM
11 7 (2.97%) Red Hat
12 5 (2.12%) MapR
13 4 (1.69%) Alteryx
13 4 (1.69%) CA Agile Central
14 3 (1.27%) AppDynamics
14 3 (1.27%) Dynatrace
14 3 (1.27%) New Relic
14 3 (1.27%) SAP
14 3 (1.27%) Splunk
14 3 (1.27%) Twitter