Period
to 1 August 2021

The following table provides summary statistics for permanent job vacancies advertised in Central London with a requirement for Data Pipeline skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Pipeline over the 6 months to 1 August 2021 with a comparison to the same period in the previous 2 years.

Data Pipelines
London > Central London
6 months to
1 Aug 2021
Same period 2020 Same period 2019
Rank 168 187 278
Rank change year-on-year +19 +91 +58
Permanent jobs citing Data Pipeline 113 75 91
As % of all permanent jobs advertised in Central London 1.71% 1.16% 0.70%
As % of the Processes & Methodologies category 1.80% 1.23% 0.74%
Number of salaries quoted 106 67 86
10th Percentile £46,250 £45,750 £43,750
25th Percentile £56,250 £55,250 £55,250
Median annual salary (50th Percentile) £75,000 £72,500 £68,750
Median % change year-on-year +3.45% +5.45% +1.85%
75th Percentile £83,750 £80,000 £80,000
90th Percentile £95,000 £96,500 £93,125
London median annual salary £75,000 £72,500 £70,000
% change year-on-year +3.45% +3.57% +3.70%

Data Pipeline is in the Processes and Methodologies category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in Central London with a requirement for process or methodology skills.

All Process and Methodology Skills
Central London
Permanent vacancies with a requirement for process or methodology skills 6,276 6,107 12,236
As % of all permanent IT jobs advertised in Central London 94.88% 94.30% 93.51%
Number of salaries quoted 5,618 5,377 10,951
10th Percentile £42,500 £42,500 £37,500
25th Percentile £53,750 £52,500 £49,977
Median annual salary (50th Percentile) £70,000 £70,000 £65,000
Median % change year-on-year - +7.69% +4.00%
75th Percentile £90,000 £85,000 £82,500
90th Percentile £105,000 £100,000 £96,250
London median annual salary £70,000 £68,346 £65,000
% change year-on-year +2.42% +5.15% +8.33%

Data Pipeline
Job Vacancy Trend in Central London

Job postings citing Data Pipeline as a proportion of all IT jobs advertised in Central London.

Job vacancy trend for Data Pipeline in Central London

Data Pipeline
Salary Trend in Central London

3-month moving average salary quoted in jobs citing Data Pipeline in Central London.

Salary trend for Data Pipeline in Central London

Data Pipeline
Salary Histogram in Central London

Salary distribution for jobs citing Data Pipeline in Central London over the 6 months to 1 August 2021.

Salary histogram for Data Pipeline in Central London

Data Pipeline
Job Locations in Central London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Pipeline within the Central London region over the 6 months to 1 August 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
City of London +25 100 £72,500 - 23
Southwark - 1 £80,000 -
Data Pipeline
London

For the 6 months to 1 August 2021, IT jobs citing Data Pipeline also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the Central London region with a requirement for Data Pipeline.

1 88 (77.88%) Python
2 74 (65.49%) SQL
3 53 (46.90%) AWS
4 49 (43.36%) Data Science
5 48 (42.48%) Analytics
6 42 (37.17%) Apache Spark
7 33 (29.20%) Data Modelling
8 32 (28.32%) Data Engineering
9 31 (27.43%) Big Data
10 29 (25.66%) Data Warehouse
10 29 (25.66%) Java
11 28 (24.78%) Data Lake
12 26 (23.01%) Azure
13 25 (22.12%) ETL
14 24 (21.24%) Degree
14 24 (21.24%) Amazon Redshift
15 23 (20.35%) Data Transformation
15 23 (20.35%) Amazon EC2
15 23 (20.35%) Agile Software Development
16 22 (19.47%) Computer Science
16 22 (19.47%) Hadoop
16 22 (19.47%) Problem-Solving
17 21 (18.58%) R
17 21 (18.58%) Relational Database
17 21 (18.58%) Analytical Skills
17 21 (18.58%) DevOps
18 20 (17.70%) Kafka
18 20 (17.70%) NoSQL
18 20 (17.70%) Apache
18 20 (17.70%) Social Skills

Data Pipeline
Co-occurring IT Skills in Central London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 42 (37.17%) Apache Spark
2 20 (17.70%) Apache
3 19 (16.81%) Apache Airflow
4 5 (4.42%) Elasticsearch
5 2 (1.77%) Apache Pig
6 1 (0.88%) Confluence
Applications
1 4 (3.54%) Microsoft Excel
1 4 (3.54%) Microsoft Office
1 4 (3.54%) Microsoft PowerPoint
Cloud Services
1 53 (46.90%) AWS
2 26 (23.01%) Azure
3 23 (20.35%) Amazon EC2
4 13 (11.50%) GCP
5 10 (8.85%) Power Platform
5 10 (8.85%) Serverless
6 7 (6.19%) Amazon EMR
6 7 (6.19%) Amazon S3
6 7 (6.19%) AWS Lambda
7 6 (5.31%) AWS CloudFormation
7 6 (5.31%) AWS Data Pipeline
7 6 (5.31%) Azure Data Factory
7 6 (5.31%) SaaS
8 4 (3.54%) Amazon ECS
8 4 (3.54%) Amazon EKS
8 4 (3.54%) Amazon ELB
8 4 (3.54%) AWS Fargate
8 4 (3.54%) Azure Logic Apps
8 4 (3.54%) Virtual Private Cloud
9 3 (2.65%) Amazon SQS
Communications & Networking
1 4 (3.54%) DNS
1 4 (3.54%) TCP/IP
2 3 (2.65%) 5G
2 3 (2.65%) Broadband
3 1 (0.88%) Network Security
Database & Business Intelligence
1 31 (27.43%) Big Data
2 29 (25.66%) Data Warehouse
3 28 (24.78%) Data Lake
4 24 (21.24%) Amazon Redshift
5 22 (19.47%) Hadoop
6 21 (18.58%) Relational Database
7 20 (17.70%) NoSQL
8 19 (16.81%) Amazon RDS
9 11 (9.73%) Looker
10 10 (8.85%) Power BI
10 10 (8.85%) Tableau
11 9 (7.96%) PostgreSQL
12 8 (7.08%) MongoDB
13 7 (6.19%) Delta Lake
13 7 (6.19%) Spark SQL
14 6 (5.31%) Data Mining
14 6 (5.31%) Neo4j
15 5 (4.42%) Metadata
16 4 (3.54%) Amazon Aurora
16 4 (3.54%) Electronic Patient Records
Development Applications
1 14 (12.39%) Git (software)
1 14 (12.39%) JIRA
2 13 (11.50%) Jenkins
3 5 (4.42%) Artifactory
3 5 (4.42%) Sonatype Nexus
4 4 (3.54%) Arsenale Dataplane
4 4 (3.54%) Gradle
4 4 (3.54%) Terragrunt
5 2 (1.77%) ER Studio
5 2 (1.77%) ERwin
5 2 (1.77%) git-flow
5 2 (1.77%) PowerDesigner
6 1 (0.88%) Atlassian Bamboo
6 1 (0.88%) Maven
6 1 (0.88%) Octopus Deploy
6 1 (0.88%) TeamCity
6 1 (0.88%) Vagrant
General
1 21 (18.58%) Analytical Skills
2 20 (17.70%) Social Skills
3 19 (16.81%) Finance
4 15 (13.27%) Games
5 14 (12.39%) Marketing
6 6 (5.31%) Advertising
7 5 (4.42%) Telecoms
8 4 (3.54%) Legal
9 2 (1.77%) Banking
9 2 (1.77%) Publishing
9 2 (1.77%) Retail
10 1 (0.88%) Back Office
10 1 (0.88%) Front Office
10 1 (0.88%) Manufacturing
Job Titles
1 71 (62.83%) Data Engineer
2 27 (23.89%) Senior Data Engineer
3 16 (14.16%) AWS Engineer
4 15 (13.27%) Big Data Engineer
4 15 (13.27%) Data Scientist
5 8 (7.08%) Developer
5 8 (7.08%) Java Engineer
6 7 (6.19%) Hadoop Engineer
7 6 (5.31%) Architect
7 6 (5.31%) Azure Engineer
7 6 (5.31%) Data Architect
7 6 (5.31%) Junior
7 6 (5.31%) Junior Data Scientist
7 6 (5.31%) Python Engineer
8 5 (4.42%) .NET Developer
8 5 (4.42%) Analyst
8 5 (4.42%) Data Analyst
8 5 (4.42%) Senior Data Scientist
9 4 (3.54%) Big Data Architect
9 4 (3.54%) DevOps Engineer
Libraries, Frameworks & Software Standards
1 20 (17.70%) Kafka
2 9 (7.96%) PySpark
3 5 (4.42%) .NET
3 5 (4.42%) AngularJS
4 4 (3.54%) .NET Core
4 4 (3.54%) REST
5 3 (2.65%) JSON
6 2 (1.77%) Middleware
6 2 (1.77%) RabbitMQ
6 2 (1.77%) Spark Streaming
7 1 (0.88%) Apache Avro
7 1 (0.88%) GraphQL
7 1 (0.88%) gRPC
7 1 (0.88%) LAPP Stack
7 1 (0.88%) Protocol Buffers
7 1 (0.88%) React
7 1 (0.88%) React Native
7 1 (0.88%) RESTful
7 1 (0.88%) TensorFlow
7 1 (0.88%) YAML
Miscellaneous
1 13 (11.50%) Mobile Games
2 12 (10.62%) Data Structures
3 10 (8.85%) Algorithms
4 9 (7.96%) Data Centre
5 5 (4.42%) Greenfield Project
5 5 (4.42%) Social Media
6 4 (3.54%) Client/Server
6 4 (3.54%) Legacy Code
7 3 (2.65%) Derivative
8 2 (1.77%) Analytical Mindset
8 2 (1.77%) Foreign Exchange (FX)
9 1 (0.88%) Acoustics
9 1 (0.88%) Blog
9 1 (0.88%) e-Learning
9 1 (0.88%) Equities
9 1 (0.88%) Equity Derivative
9 1 (0.88%) Fixed Income
9 1 (0.88%) Management Information System
9 1 (0.88%) Public Cloud
9 1 (0.88%) User Experience
Operating Systems
1 15 (13.27%) Linux
2 4 (3.54%) Unix
Processes & Methodologies
1 49 (43.36%) Data Science
2 48 (42.48%) Analytics
3 33 (29.20%) Data Modelling
4 32 (28.32%) Data Engineering
5 25 (22.12%) ETL
6 23 (20.35%) Agile Software Development
6 23 (20.35%) Data Transformation
7 22 (19.47%) Computer Science
7 22 (19.47%) Problem-Solving
8 21 (18.58%) DevOps
9 18 (15.93%) Artificial Intelligence
10 15 (13.27%) Business Intelligence
10 15 (13.27%) Machine Learning
10 15 (13.27%) Model Validation
11 14 (12.39%) Mentoring
11 14 (12.39%) Statistics
12 13 (11.50%) Data Validation
12 13 (11.50%) Statistical Analysis
12 13 (11.50%) Validation
12 13 (11.50%) Workflow
Programming Languages
1 88 (77.88%) Python
2 74 (65.49%) SQL
3 29 (25.66%) Java
4 21 (18.58%) R
5 17 (15.04%) Scala
6 7 (6.19%) Bash Shell
7 5 (4.42%) C#
7 5 (4.42%) Clojure
7 5 (4.42%) Cypher
8 2 (1.77%) Go
9 1 (0.88%) C
9 1 (0.88%) C++
9 1 (0.88%) Haskell
9 1 (0.88%) JavaScript
9 1 (0.88%) PowerShell
Qualifications
1 24 (21.24%) Degree
2 6 (5.31%) Master's Degree
3 4 (3.54%) AWS Certification
4 3 (2.65%) Computer Science Degree
Quality Assurance & Compliance
1 14 (12.39%) Data Quality
2 11 (9.73%) QA
System Software
1 5 (4.42%) Docker
2 2 (1.77%) Apache Flume
3 1 (0.88%) VirtualBox
Systems Management
1 13 (11.50%) Terraform
2 9 (7.96%) Kubernetes
3 6 (5.31%) Ansible
4 4 (3.54%) Ambari
4 4 (3.54%) Hadoop YARN
5 2 (1.77%) Kubeflow
6 1 (0.88%) Docker Swarm
6 1 (0.88%) Puppet
Vendors
1 13 (11.50%) Talend
2 10 (8.85%) Databricks
3 9 (7.96%) Microsoft
4 8 (7.08%) Facebook
4 8 (7.08%) Oracle
5 5 (4.42%) Cloudera
5 5 (4.42%) Google
5 5 (4.42%) Hortonworks
6 1 (0.88%) Alteryx
6 1 (0.88%) Confluent
6 1 (0.88%) IBM
6 1 (0.88%) Informatica
6 1 (0.88%) Sage
6 1 (0.88%) StreamSets
6 1 (0.88%) VMware