Period
to 17 June 2021

The following table provides summary statistics for permanent job vacancies advertised in the City of London with a requirement for Data Pipeline skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Pipeline over the 6 months to 17 June 2021 with a comparison to the same period in the previous 2 years.

Data Pipelines
Central London > City of London
6 months to
17 Jun 2021
Same period 2020 Same period 2019
Rank 146 189 270
Rank change year-on-year +43 +81 +38
Permanent jobs citing Data Pipeline 89 65 60
As % of all permanent jobs advertised in the City of London 1.71% 1.04% 0.60%
As % of the Processes & Methodologies category 1.80% 1.10% 0.64%
Number of salaries quoted 85 54 56
10th Percentile £47,750 £52,500 £48,419
25th Percentile £53,750 £61,250 £53,438
Median annual salary (50th Percentile) £75,000 £72,500 £68,750
Median % change year-on-year +3.45% +5.45% +1.85%
75th Percentile £83,750 £92,500 £81,250
90th Percentile £94,500 £114,500 £90,625
Central London median annual salary £75,000 £72,500 £70,000
% change year-on-year +3.45% +3.57% +3.70%

Data Pipeline is in the Processes and Methodologies category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in the City of London with a requirement for process or methodology skills.

All Process and Methodology Skills
City of London
Permanent vacancies with a requirement for process or methodology skills 4,943 5,892 9,367
As % of all permanent IT jobs advertised in the City of London 95.15% 94.20% 93.36%
Number of salaries quoted 4,393 5,157 8,349
10th Percentile £43,750 £41,250 £37,500
25th Percentile £55,000 £52,500 £50,000
Median annual salary (50th Percentile) £72,500 £70,000 £65,000
Median % change year-on-year +3.57% +7.69% +4.00%
75th Percentile £90,000 £85,000 £82,500
90th Percentile £106,250 £100,000 £97,500
Central London median annual salary £70,000 £70,000 £65,000
% change year-on-year - +7.69% +8.33%

Data Pipeline
Job Vacancy Trend in the City of London

Job postings citing Data Pipeline as a proportion of all IT jobs advertised in the City of London.

Job vacancy trend for Data Pipeline in the City of London

Data Pipeline
Salary Trend in the City of London

3-month moving average salary quoted in jobs citing Data Pipeline in the City of London.

Salary trend for Data Pipeline in the City of London

Data Pipeline
Salary Histogram in the City of London

Salary distribution for jobs citing Data Pipeline in the City of London over the 6 months to 17 June 2021.

Salary histogram for Data Pipeline in the City of London

Data Pipeline
Job Locations in the City of London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Pipeline within the City of London region over the 6 months to 17 June 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
London Bridge - 1 £80,000 -
Data Pipeline
Central London

For the 6 months to 17 June 2021, IT jobs citing Data Pipeline also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the City of London region with a requirement for Data Pipeline.

1 68 (76.40%) SQL
2 66 (74.16%) Python
3 43 (48.31%) Data Science
4 42 (47.19%) Analytics
5 37 (41.57%) AWS
6 32 (35.96%) Apache Spark
7 29 (32.58%) Data Lake
8 28 (31.46%) Data Warehouse
9 26 (29.21%) Big Data
10 24 (26.97%) ETL
10 24 (26.97%) Hadoop
11 23 (25.84%) Data Modelling
11 23 (25.84%) NoSQL
11 23 (25.84%) Data Transformation
12 22 (24.72%) Amazon Redshift
13 21 (23.60%) Azure
14 20 (22.47%) Analytical Skills
15 19 (21.35%) Relational Database
15 19 (21.35%) Java
16 18 (20.22%) Apache
16 18 (20.22%) Amazon EC2
17 17 (19.10%) Data Engineering
17 17 (19.10%) Degree
17 17 (19.10%) Problem-Solving
18 16 (17.98%) Finance
18 16 (17.98%) Amazon RDS
19 15 (16.85%) Talend
19 15 (16.85%) Agile Software Development
20 14 (15.73%) Computer Science
20 14 (15.73%) Statistics

Data Pipeline
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 32 (35.96%) Apache Spark
2 18 (20.22%) Apache
3 12 (13.48%) Apache Airflow
4 4 (4.49%) Elasticsearch
5 2 (2.25%) Apache Pig
6 1 (1.12%) Confluence
Applications
1 4 (4.49%) Microsoft Excel
1 4 (4.49%) Microsoft Office
1 4 (4.49%) Microsoft PowerPoint
Cloud Services
1 37 (41.57%) AWS
2 21 (23.60%) Azure
3 18 (20.22%) Amazon EC2
4 9 (10.11%) Power Platform
5 8 (8.99%) GCP
6 7 (7.87%) Azure Data Factory
6 7 (7.87%) Serverless
7 6 (6.74%) AWS Data Pipeline
8 4 (4.49%) Azure Functions
8 4 (4.49%) Azure Logic Apps
9 3 (3.37%) Amazon Kinesis
9 3 (3.37%) Amazon S3
9 3 (3.37%) AWS CloudFormation
9 3 (3.37%) AWS Lambda
10 2 (2.25%) Amazon ELB
10 2 (2.25%) AWS Fargate
10 2 (2.25%) AWS Step Functions
10 2 (2.25%) Azure Synapse Analytics
10 2 (2.25%) Matillion
10 2 (2.25%) Virtual Private Cloud
Communications & Networking
1 3 (3.37%) 5G
1 3 (3.37%) Broadband
2 2 (2.25%) DNS
2 2 (2.25%) TCP/IP
3 1 (1.12%) Network Security
Database & Business Intelligence
1 29 (32.58%) Data Lake
2 28 (31.46%) Data Warehouse
3 26 (29.21%) Big Data
4 24 (26.97%) Hadoop
5 23 (25.84%) NoSQL
6 22 (24.72%) Amazon Redshift
7 19 (21.35%) Relational Database
8 16 (17.98%) Amazon RDS
9 12 (13.48%) Looker
10 10 (11.24%) Tableau
11 9 (10.11%) Power BI
11 9 (10.11%) Spark SQL
12 8 (8.99%) PostgreSQL
13 7 (7.87%) Delta Lake
14 6 (6.74%) Data Mining
15 5 (5.62%) MongoDB
15 5 (5.62%) Neo4j
16 4 (4.49%) Azure SQL Database
17 3 (3.37%) OLTP
17 3 (3.37%) Sqoop
Development Applications
1 11 (12.36%) JIRA
2 10 (11.24%) Git (software)
2 10 (11.24%) Jenkins
3 3 (3.37%) ER Studio
3 3 (3.37%) ERwin
3 3 (3.37%) Gradle
3 3 (3.37%) PowerDesigner
4 2 (2.25%) Terragrunt
5 1 (1.12%) Artifactory
5 1 (1.12%) Maven
5 1 (1.12%) Octopus Deploy
5 1 (1.12%) Sonatype Nexus
5 1 (1.12%) TeamCity
General
1 20 (22.47%) Analytical Skills
2 16 (17.98%) Finance
3 12 (13.48%) Games
3 12 (13.48%) Social Skills
4 10 (11.24%) Marketing
5 7 (7.87%) Telecoms
6 3 (3.37%) Advertising
7 1 (1.12%) Banking
7 1 (1.12%) Legal
Job Titles
1 54 (60.67%) Data Engineer
2 18 (20.22%) AWS Engineer
3 16 (17.98%) Senior Data Engineer
4 14 (15.73%) Data Scientist
5 12 (13.48%) Big Data Engineer
6 7 (7.87%) Azure Engineer
6 7 (7.87%) Hadoop Engineer
7 6 (6.74%) Java Engineer
7 6 (6.74%) Junior
7 6 (6.74%) Junior Data Scientist
8 5 (5.62%) Analyst
8 5 (5.62%) Data Analyst
8 5 (5.62%) Developer
9 4 (4.49%) Digital Analyst
9 4 (4.49%) Digital Transformation
9 4 (4.49%) Senior Data Scientist
9 4 (4.49%) Transformation Analyst
10 3 (3.37%) Architect
10 3 (3.37%) Data Manager
10 3 (3.37%) Engineering Manager
Libraries, Frameworks & Software Standards
1 13 (14.61%) Kafka
2 8 (8.99%) PySpark
3 3 (3.37%) AngularJS
4 2 (2.25%) JSON
5 1 (1.12%) .NET
5 1 (1.12%) .NET Core
5 1 (1.12%) Apache Avro
5 1 (1.12%) GraphQL
5 1 (1.12%) gRPC
5 1 (1.12%) Middleware
5 1 (1.12%) Protocol Buffers
5 1 (1.12%) RabbitMQ
5 1 (1.12%) React Native
5 1 (1.12%) Spark Streaming
5 1 (1.12%) YAML
Miscellaneous
1 12 (13.48%) Data Structures
1 12 (13.48%) Mobile Games
2 7 (7.87%) Algorithms
3 5 (5.62%) Data Centre
4 3 (3.37%) Derivative
4 3 (3.37%) Social Media
5 2 (2.25%) Analytical Mindset
5 2 (2.25%) Foreign Exchange (FX)
5 2 (2.25%) Greenfield Project
5 2 (2.25%) Public Cloud
6 1 (1.12%) Client/Server
6 1 (1.12%) e-Learning
6 1 (1.12%) Equities
6 1 (1.12%) Equity Derivative
6 1 (1.12%) Fixed Income
6 1 (1.12%) Housing Association
6 1 (1.12%) Management Information System
6 1 (1.12%) User Experience
Operating Systems
1 11 (12.36%) Linux
Processes & Methodologies
1 43 (48.31%) Data Science
2 42 (47.19%) Analytics
3 24 (26.97%) ETL
4 23 (25.84%) Data Modelling
4 23 (25.84%) Data Transformation
5 17 (19.10%) Data Engineering
5 17 (19.10%) Problem-Solving
6 15 (16.85%) Agile Software Development
7 14 (15.73%) Business Intelligence
7 14 (15.73%) Computer Science
7 14 (15.73%) Model Validation
7 14 (15.73%) Statistics
8 13 (14.61%) Data Integration
8 13 (14.61%) DevOps
9 12 (13.48%) Data Validation
9 12 (13.48%) Predictive Modelling
9 12 (13.48%) Statistical Analysis
9 12 (13.48%) Validation
10 11 (12.36%) Artificial Intelligence
10 11 (12.36%) Machine Learning
Programming Languages
1 68 (76.40%) SQL
2 66 (74.16%) Python
3 19 (21.35%) Java
4 12 (13.48%) R
5 9 (10.11%) Bash Shell
6 6 (6.74%) Scala
7 5 (5.62%) Cypher
8 2 (2.25%) Go
9 1 (1.12%) C#
9 1 (1.12%) Clojure
9 1 (1.12%) Haskell
9 1 (1.12%) PowerShell
Qualifications
1 17 (19.10%) Degree
2 3 (3.37%) Computer Science Degree
2 3 (3.37%) Master's Degree
3 2 (2.25%) AWS Certification
Quality Assurance & Compliance
1 10 (11.24%) QA
2 7 (7.87%) Data Quality
3 1 (1.12%) Accessibility
System Software
1 3 (3.37%) Apache Flume
1 3 (3.37%) Docker
Systems Management
1 4 (4.49%) Terraform
2 3 (3.37%) Kubernetes
3 1 (1.12%) Ansible
3 1 (1.12%) Puppet
Vendors
1 15 (16.85%) Talend
2 10 (11.24%) Microsoft
3 9 (10.11%) Databricks
4 8 (8.99%) Facebook
5 7 (7.87%) Oracle
6 1 (1.12%) Confluent
6 1 (1.12%) Google
6 1 (1.12%) Informatica
6 1 (1.12%) Sage
6 1 (1.12%) VMware