Period
to 18 January 2022

The following table provides summary statistics for contract job vacancies advertised in the City of London with a requirement for Data Pipeline skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Data Pipeline over the 6 months to 18 January 2022 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Data Pipelines
Central London > City of London
6 months to
18 Jan 2022
Same period 2021 Same period 2020
Rank 145 105 167
Rank change year-on-year -40 +62 +65
Contract jobs citing Data Pipeline 42 21 32
As % of all contract jobs advertised in the City of London 1.05% 1.03% 0.65%
As % of the Processes & Methodologies category 1.16% 1.14% 0.73%
Number of daily rates quoted 29 14 25
10th Percentile £373 £436 £455
25th Percentile £500 £512 £485
Median daily rate (50th Percentile) £575 £600 £540
Median % change year-on-year -4.17% +11.11% -1.82%
75th Percentile £700 £631 £588
90th Percentile £815 £646 £675
Central London median daily rate £600 £569 £550
% change year-on-year +5.49% +3.41% -8.33%
Number of hourly rates quoted 1 0 0
10th Percentile - - -
25th Percentile £59.25 - -
Median hourly rate £60.50 - -
75th Percentile £61.75 - -
90th Percentile - - -
Central London median hourly rate £60.50 - -

Data Pipeline is in the Processes and Methodologies category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in the City of London with a requirement for process or methodology skills.

All Process and Methodology Skills
City of London
Contract vacancies with a requirement for process or methodology skills 3,629 1,840 4,405
As % of all contract IT jobs advertised in the City of London 90.73% 90.15% 89.33%
Number of daily rates quoted 2,818 1,376 3,460
10th Percentile £370 £313 £320
25th Percentile £469 £413 £413
Median daily rate (50th Percentile) £550 £525 £513
Median % change year-on-year +4.76% +2.44% -
75th Percentile £660 £600 £605
90th Percentile £750 £688 £675
Central London median daily rate £550 £515 £510
% change year-on-year +6.80% +0.98% -0.49%
Number of hourly rates quoted 36 23 49
10th Percentile £19.13 £13.63 £12.03
25th Percentile £30.00 £19.50 £14.25
Median hourly rate £43.25 £26.50 £20.00
Median % change year-on-year +63.21% +32.50% +5.26%
75th Percentile £61.75 £36.38 £25.76
90th Percentile £64.25 £46.00 £52.25
Central London median hourly rate £43.25 £27.00 £20.42
% change year-on-year +60.19% +32.26% +6.05%

Data Pipeline
Job Vacancy Trend in the City of London

Job postings citing Data Pipeline as a proportion of all IT jobs advertised in the City of London.

Job vacancy trend for Data Pipeline in the City of London

Data Pipeline
Contractor Daily Rate Trend in the City of London

3-month moving average daily rate quoted in jobs citing Data Pipeline in the City of London.

Daily rate trend for Data Pipeline in the City of London

Data Pipeline
Daily Rate Histogram in the City of London

Daily rate distribution for jobs citing Data Pipeline in the City of London over the 6 months to 18 January 2022.

Daily rate histogram for Data Pipeline in the City of London

Data Pipeline
Contractor Hourly Rate Trend in the City of London

3-month moving average hourly rates quoted in jobs citing Data Pipeline in the City of London.

Hourly rate trend for Data Pipeline in the City of London

For the 6 months to 18 January 2022, IT contractor jobs citing Data Pipeline also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the City of London region with a requirement for Data Pipeline.

1 31 (73.81%) Python
2 25 (59.52%) SQL
3 22 (52.38%) AWS
4 14 (33.33%) Analytics
4 14 (33.33%) DevOps
5 12 (28.57%) Azure
6 11 (26.19%) ETL
6 11 (26.19%) Data Science
6 11 (26.19%) Apache Spark
7 10 (23.81%) Agile Software Development
7 10 (23.81%) Kafka
8 9 (21.43%) Big Data
8 9 (21.43%) Amazon Redshift
8 9 (21.43%) Finance
8 9 (21.43%) GCP
8 9 (21.43%) Serverless
9 8 (19.05%) Data Engineering
9 8 (19.05%) Terraform
9 8 (19.05%) Business Intelligence
9 8 (19.05%) Data Modelling
10 7 (16.67%) Software Engineering
10 7 (16.67%) Data Lake
11 6 (14.29%) AWS Lambda
11 6 (14.29%) MongoDB
11 6 (14.29%) NoSQL
11 6 (14.29%) Scala
12 5 (11.90%) Visualisation
12 5 (11.90%) Amazon S3
12 5 (11.90%) Jenkins
12 5 (11.90%) Google

Data Pipeline
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 11 (26.19%) Apache Spark
2 4 (9.52%) Apache Airflow
3 3 (7.14%) Apache
4 2 (4.76%) CMS
Applications
1 2 (4.76%) Microsoft PowerPoint
2 1 (2.38%) SPSS
Cloud Services
1 22 (52.38%) AWS
2 12 (28.57%) Azure
3 9 (21.43%) GCP
3 9 (21.43%) Serverless
4 6 (14.29%) AWS Lambda
5 5 (11.90%) Amazon S3
5 5 (11.90%) dbt
5 5 (11.90%) Snowflake
6 4 (9.52%) Power Platform
7 3 (7.14%) AWS Step Functions
7 3 (7.14%) Azure Cosmos DB
7 3 (7.14%) Azure Data Factory
7 3 (7.14%) GitHub
8 2 (4.76%) Amazon EMR
8 2 (4.76%) Amazon Kinesis
8 2 (4.76%) npm
9 1 (2.38%) Amazon CloudWatch
9 1 (2.38%) AWS CloudFormation
9 1 (2.38%) AWS Data Pipeline
9 1 (2.38%) Virtual Private Cloud
Communications & Networking
1 1 (2.38%) Kerberos
Database & Business Intelligence
1 9 (21.43%) Amazon Redshift
1 9 (21.43%) Big Data
2 7 (16.67%) Data Lake
3 6 (14.29%) MongoDB
3 6 (14.29%) NoSQL
4 5 (11.90%) Data Warehouse
5 4 (9.52%) BigQuery
5 4 (9.52%) Couchbase
5 4 (9.52%) Power BI
5 4 (9.52%) SQL Server
5 4 (9.52%) Tableau
6 3 (7.14%) Amazon DynamoDB
6 3 (7.14%) Hadoop
6 3 (7.14%) Relational Database
7 2 (4.76%) Apache Hive
7 2 (4.76%) Data Mart
7 2 (4.76%) Presto
7 2 (4.76%) SQL Server Integration Services
7 2 (4.76%) SQL Server Reporting Services
8 1 (2.38%) Qlik Sense
Development Applications
1 5 (11.90%) Jenkins
2 3 (7.14%) GitLab
3 2 (4.76%) Travis CI
4 1 (2.38%) CircleCI
4 1 (2.38%) Git (software)
4 1 (2.38%) GoCD
4 1 (2.38%) IBM UrbanCode
4 1 (2.38%) JIRA
4 1 (2.38%) TeamCity
General
1 9 (21.43%) Finance
2 4 (9.52%) Public Sector
3 3 (7.14%) Social Skills
4 2 (4.76%) Marketing
5 1 (2.38%) Advertising
5 1 (2.38%) Analytical Skills
5 1 (2.38%) Banking
5 1 (2.38%) Financial Institution
5 1 (2.38%) Music Industry
5 1 (2.38%) Retail
Job Titles
1 23 (54.76%) Data Engineer
2 5 (11.90%) Developer
3 4 (9.52%) Analyst
3 4 (9.52%) Data Analyst
4 3 (7.14%) DevOps Engineer
4 3 (7.14%) Engineering Manager
4 3 (7.14%) Python Developer
4 3 (7.14%) Senior Data Engineer
4 3 (7.14%) Software Engineering Manager
4 3 (7.14%) Software Manager
5 2 (4.76%) Architect
5 2 (4.76%) AWS Engineer
6 1 (2.38%) Business Intelligence Developer
6 1 (2.38%) Consultant
6 1 (2.38%) Data Architect
6 1 (2.38%) Data Consultant
6 1 (2.38%) Marketing Analyst
6 1 (2.38%) Senior Developer
6 1 (2.38%) Senior Python Developer
6 1 (2.38%) Solutions Architect
Libraries, Frameworks & Software Standards
1 10 (23.81%) Kafka
2 3 (7.14%) Pandas
3 2 (4.76%) Apache Beam
3 2 (4.76%) Django
3 2 (4.76%) Node.js
4 1 (2.38%) PySpark
4 1 (2.38%) pytest
4 1 (2.38%) Spark Streaming
4 1 (2.38%) Spring
4 1 (2.38%) Web Services
Miscellaneous
1 4 (9.52%) Algorithms
2 3 (7.14%) Legacy Applications
3 2 (4.76%) Data Structures
3 2 (4.76%) Greenfield Project
3 2 (4.76%) Private Cloud
4 1 (2.38%) Analytical Mindset
4 1 (2.38%) EPoS
4 1 (2.38%) Social Media
Processes & Methodologies
1 14 (33.33%) Analytics
1 14 (33.33%) DevOps
2 11 (26.19%) Data Science
2 11 (26.19%) ETL
3 10 (23.81%) Agile Software Development
4 8 (19.05%) Business Intelligence
4 8 (19.05%) Data Engineering
4 8 (19.05%) Data Modelling
5 7 (16.67%) Software Engineering
6 5 (11.90%) Infrastructure as Code
6 5 (11.90%) Visualisation
7 4 (9.52%) Clean Code
7 4 (9.52%) Fintech
7 4 (9.52%) Log Aggregation
8 3 (7.14%) Community of Practice
8 3 (7.14%) Continuous Integration
8 3 (7.14%) Data Acquisition
8 3 (7.14%) Full Stack Development
8 3 (7.14%) Software Delivery
8 3 (7.14%) Use Case
Programming Languages
1 31 (73.81%) Python
2 25 (59.52%) SQL
3 6 (14.29%) Scala
4 5 (11.90%) Java
5 2 (4.76%) R
5 2 (4.76%) Ruby
5 2 (4.76%) Rust
6 1 (2.38%) Bash
6 1 (2.38%) JavaScript
6 1 (2.38%) T-SQL
6 1 (2.38%) TypeScript
Qualifications
1 2 (4.76%) AWS Certification
1 2 (4.76%) Computer Science Degree
1 2 (4.76%) Degree
Quality Assurance & Compliance
1 2 (4.76%) Data Quality
2 1 (2.38%) QA
System Software
1 4 (9.52%) Docker
Systems Management
1 8 (19.05%) Terraform
2 5 (11.90%) Kubernetes
3 2 (4.76%) Ansible
4 1 (2.38%) Hadoop YARN
Vendors
1 5 (11.90%) Google
2 4 (9.52%) Oracle
3 3 (7.14%) Databricks
4 2 (4.76%) Microsoft
4 2 (4.76%) Virgin Media
5 1 (2.38%) Alteryx
5 1 (2.38%) Cloudera
5 1 (2.38%) IBM
5 1 (2.38%) ServiceNow