Data Pipeline Contracts

Data Pipelines
UK

The table below provides summary statistics for contract job vacancies requiring Data Pipeline skills. It includes a benchmarking guide to the contractor rates offered in vacancies that cited Data Pipeline over the 6 months leading up to 18 June 2025, comparing them to the same period in the previous two years.

6 months to
18 Jun 2025
Same period 2024 Same period 2023
Rank 139 180 301
Rank change year-on-year +41 +121 -34
Contract jobs citing Data Pipeline 607 579 475
As % of all contract jobs advertised in the UK 2.04% 1.32% 0.86%
As % of the Processes & Methodologies category 2.24% 1.55% 0.95%
Number of daily rates quoted 448 428 320
10th Percentile £400 £363 £405
25th Percentile £428 £449 £479
Median daily rate (50th Percentile) £500 £525 £550
Median % change year-on-year -4.76% -4.55% -8.33%
75th Percentile £625 £638 £675
90th Percentile £738 £738 £738
UK excluding London median daily rate £475 £500 £525
% change year-on-year -5.00% -4.76% -4.55%
Number of hourly rates quoted 9 6 5
10th Percentile £36.98 £34.24 £47.75
25th Percentile £51.25 £49.75 £65.00
Median hourly rate £70.00 £62.50 £75.50
Median % change year-on-year +12.00% -17.22% +123.70%
75th Percentile £80.00 £66.25 £82.50
90th Percentile £92.50 £67.50 £87.00
UK excluding London median hourly rate £74.35 £62.50 £55.88
% change year-on-year +18.96% +11.86% -

All Process and Methodology Skills
UK

Data Pipeline falls under the Processes and Methodologies category. For comparison with the information above, the following table provides summary statistics for all contract job vacancies requiring process or methodology skills.

Contract vacancies with a requirement for process or methodology skills 27,135 37,295 49,745
As % of all contract IT jobs advertised in the UK 91.00% 85.02% 89.61%
Number of daily rates quoted 17,195 23,752 34,382
10th Percentile £300 £300 £325
25th Percentile £406 £413 £438
Median daily rate (50th Percentile) £505 £525 £550
Median % change year-on-year -3.76% -4.55% +1.85%
75th Percentile £625 £638 £650
90th Percentile £738 £750 £750
UK excluding London median daily rate £475 £500 £500
% change year-on-year -5.00% - +1.01%
Number of hourly rates quoted 1,252 2,606 1,681
10th Percentile £14.20 £12.75 £11.00
25th Percentile £17.50 £16.01 £15.75
Median hourly rate £25.50 £37.18 £35.00
Median % change year-on-year -31.41% +6.23% +34.62%
75th Percentile £55.25 £61.30 £64.63
90th Percentile £73.75 £72.50 £75.00
UK excluding London median hourly rate £25.08 £38.12 £32.50
% change year-on-year -34.22% +17.29% +54.76%

Data Pipeline
Job Vacancy Trend

Job postings citing Data Pipeline as a proportion of all IT jobs advertised.

Job vacancy trend for Data Pipeline in the UK

Data Pipeline
Contractor Daily Rate Trend

3-month moving average daily rate quoted in jobs citing Data Pipeline.

Daily rate trend for Data Pipeline in the UK

Data Pipeline
Daily Rate Histogram

Daily rate distribution for jobs citing Data Pipeline over the 6 months to 18 June 2025.

Daily rate histogram for Data Pipeline in the UK

Data Pipeline
Contractor Hourly Rate Trend

3-month moving average hourly rates quoted in jobs citing Data Pipeline.

Hourly rate trend for Data Pipeline in the UK

Data Pipeline
Hourly Rate Histogram

Hourly rate distribution of jobs citing Data Pipeline over the 6 months to 18 June 2025.

Hourly rate histogram for Data Pipeline in the UK

Data Pipeline
Top 16 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Data Pipeline within the UK over the 6 months to 18 June 2025. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Jobs
England +46 542 £500 -4.76% 6,103
London +31 350 £538 -2.27% 2,317
Work from Home +28 187 £500 -2.95% 2,655
UK excluding London +58 186 £475 -5.00% 4,574
North of England +41 89 £475 -5.00% 971
North West +37 60 £475 -2.06% 451
Midlands -5 32 £461 -16.23% 698
Yorkshire +14 26 £575 +11.65% 410
South East +49 24 £402 +7.20% 1,338
Scotland +16 19 £475 +4.86% 264
West Midlands -23 16 £500 -9.09% 389
East Midlands +19 15 £455 +13.69% 309
South West +34 13 £531 -3.41% 484
East of England +30 11 £367 -31.73% 609
North East - 3 £450 - 111
Northern Ireland -4 2 £560 +31.76% 95

Data Pipeline
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 108 (17.79%) Apache Spark
2 84 (13.84%) Apache Airflow
3 82 (13.51%) Apache
4 10 (1.65%) Confluence
4 10 (1.65%) Jupyter
5 4 (0.66%) SAS
6 3 (0.49%) SharePoint
7 1 (0.16%) Luigi
7 1 (0.16%) nginx
Applications
1 15 (2.47%) Microsoft Excel
2 2 (0.33%) Chatbot
2 2 (0.33%) MS Visio
3 1 (0.16%) CATIA
3 1 (0.16%) Microsoft Office
3 1 (0.16%) Oracle Smart View
Business Applications
1 6 (0.99%) Aladdin
1 6 (0.99%) Payment Gateway
2 2 (0.33%) Endur
2 2 (0.33%) NetSuite
3 1 (0.16%) Aligne
3 1 (0.16%) Dynamics AX
3 1 (0.16%) Dynamics CRM
3 1 (0.16%) Oracle Assets
3 1 (0.16%) Oracle ERP
3 1 (0.16%) Oracle General Ledger
3 1 (0.16%) Oracle Payables
3 1 (0.16%) Oracle Receivables
3 1 (0.16%) Sparx Enterprise Architect
Cloud Services
1 260 (42.83%) AWS
2 237 (39.04%) Azure
3 123 (20.26%) GCP
4 101 (16.64%) Power Platform
5 90 (14.83%) Snowflake
6 81 (13.34%) Amazon S3
7 73 (12.03%) Azure Data Factory
8 68 (11.20%) dbt
9 61 (10.05%) Serverless
10 57 (9.39%) AWS Lambda
11 50 (8.24%) Azure Synapse Analytics
12 48 (7.91%) GitHub
13 27 (4.45%) GitHub Actions
14 22 (3.62%) Azure DevOps
15 21 (3.46%) Amazon EC2
16 20 (3.29%) Cloud Computing
17 18 (2.97%) Amazon CloudWatch
17 18 (2.97%) Amazon EKS
18 17 (2.80%) Amazon Kinesis
19 15 (2.47%) AWS CloudFormation
Communications & Networking
1 5 (0.82%) DNS
2 4 (0.66%) SSH
3 1 (0.16%) Firewall
3 1 (0.16%) TCP/IP
3 1 (0.16%) VPN
Database & Business Intelligence
1 164 (27.02%) Data Warehouse
2 101 (16.64%) Power BI
3 93 (15.32%) Data Lake
4 74 (12.19%) Big Data
5 55 (9.06%) BigQuery
6 53 (8.73%) NoSQL
7 49 (8.07%) Amazon Redshift
8 46 (7.58%) Hadoop
9 43 (7.08%) SQL Server
10 42 (6.92%) Relational Database
10 42 (6.92%) Tableau
11 36 (5.93%) PostgreSQL
12 28 (4.61%) Flink
13 27 (4.45%) Amazon RDS
13 27 (4.45%) Metadata
14 24 (3.95%) Amazon Athena
14 24 (3.95%) Snowflake Schema
15 22 (3.62%) Delta Lake
16 21 (3.46%) Apache Hive
17 18 (2.97%) Azure SQL Database
Development Applications
1 57 (9.39%) Git
2 27 (4.45%) Jenkins
3 25 (4.12%) GitLab
4 21 (3.46%) JIRA
5 8 (1.32%) ERwin
5 8 (1.32%) MLflow
6 6 (0.99%) ER Studio
7 5 (0.82%) Terragrunt
8 4 (0.66%) Postman
8 4 (0.66%) RStudio
8 4 (0.66%) Selenium
9 3 (0.49%) JMeter
9 3 (0.49%) Liquibase
10 2 (0.33%) Cypress.io
10 2 (0.33%) Gradle
10 2 (0.33%) MagicDraw
10 2 (0.33%) SoapUI
11 1 (0.16%) Jasmine
11 1 (0.16%) Mocha
11 1 (0.16%) Mockito
General
1 167 (27.51%) Social Skills
2 129 (21.25%) Finance
3 89 (14.66%) Analytical Skills
4 44 (7.25%) Public Sector
5 39 (6.43%) Banking
6 23 (3.79%) Investment Banking
7 22 (3.62%) Retail
8 16 (2.64%) Presentation Skills
9 13 (2.14%) Inclusion and Diversity
10 9 (1.48%) Retail Banking
11 8 (1.32%) Pharmaceutical
12 7 (1.15%) Marketing
12 7 (1.15%) Telecoms
13 6 (0.99%) Local Government
13 6 (0.99%) Organisational Skills
14 5 (0.82%) Corporate Banking
14 5 (0.82%) Documentation Skills
14 5 (0.82%) Front Office
14 5 (0.82%) Influencing Skills
14 5 (0.82%) Manufacturing
Job Titles
1 279 (45.96%) Data Engineer
2 88 (14.50%) Senior
3 80 (13.18%) Developer
4 74 (12.19%) Architect
5 52 (8.57%) Senior Data Engineer
6 43 (7.08%) Data Architect
7 41 (6.75%) Lead
8 38 (6.26%) Analyst
9 34 (5.60%) Azure Engineer
10 33 (5.44%) AWS Engineer
10 33 (5.44%) Azure Data Engineer
11 30 (4.94%) Python Engineer
12 28 (4.61%) AWS Data Engineer
13 24 (3.95%) Python Data Engineer
14 21 (3.46%) Data Analyst
14 21 (3.46%) Python Developer
15 20 (3.29%) Cloud Engineer
16 19 (3.13%) Solutions Architect
17 18 (2.97%) Snowflake Data Engineer
17 18 (2.97%) Snowflake Engineer
Libraries, Frameworks & Software Standards
1 87 (14.33%) PySpark
2 85 (14.00%) Kafka
3 30 (4.94%) Pandas
4 20 (3.29%) Spark Streaming
5 17 (2.80%) pytest
6 16 (2.64%) NumPy
7 14 (2.31%) React
7 14 (2.31%) RESTful
8 12 (1.98%) Apache Beam
8 12 (1.98%) Apache NiFi
9 11 (1.81%) AngularJS
9 11 (1.81%) TensorFlow
10 10 (1.65%) FastAPI
10 10 (1.65%) REST
11 9 (1.48%) JDBC
11 9 (1.48%) JSON
11 9 (1.48%) pydantic
11 9 (1.48%) YAML
12 8 (1.32%) HTML
12 8 (1.32%) PyTorch
Miscellaneous
1 33 (5.44%) Cloud Native
2 27 (4.45%) Distributed Systems
3 25 (4.12%) Legacy Systems
4 23 (3.79%) Data Structures
5 21 (3.46%) Distributed Computing
6 18 (2.97%) Product Ownership
7 14 (2.31%) Derivative
8 12 (1.98%) Analytical Mindset
9 11 (1.81%) Algorithms
9 11 (1.81%) Virtual Team
10 10 (1.65%) Equities
10 10 (1.65%) Market Risk
10 10 (1.65%) Reference Data
11 9 (1.48%) Onboarding
12 8 (1.32%) Data Protection Act
12 8 (1.32%) Equity Derivative
13 7 (1.15%) Public Cloud
13 7 (1.15%) Self-Motivation
14 6 (0.99%) Clustering
14 6 (0.99%) Greenfield Project
Operating Systems
1 35 (5.77%) Linux
2 15 (2.47%) Unix
3 1 (0.16%) Red Hat Enterprise Linux
Processes & Methodologies
1 286 (47.12%) ETL
2 231 (38.06%) Analytics
3 218 (35.91%) Data Modelling
4 193 (31.80%) Data Engineering
5 172 (28.34%) Problem-Solving
6 170 (28.01%) CI/CD
7 125 (20.59%) Data Governance
8 117 (19.28%) Agile
9 114 (18.78%) Business Intelligence
10 111 (18.29%) Data Science
11 109 (17.96%) DevOps
12 107 (17.63%) Visualisation
13 101 (16.64%) Data Integration
14 95 (15.65%) Decision-Making
15 89 (14.66%) Machine Learning
16 75 (12.36%) Artificial Intelligence
17 73 (12.03%) Containerisation
18 68 (11.20%) Data Ingestion
19 66 (10.87%) Migration
20 64 (10.54%) Data Transformation
Programming Languages
1 394 (64.91%) Python
2 342 (56.34%) SQL
3 65 (10.71%) Java
4 39 (6.43%) Scala
5 25 (4.12%) R
6 22 (3.62%) C++
7 19 (3.13%) JavaScript
8 17 (2.80%) Data Analysis Expressions
9 13 (2.14%) Bash
10 10 (1.65%) C#
10 10 (1.65%) Shell Script
11 9 (1.48%) TypeScript
12 6 (0.99%) PL/SQL
12 6 (0.99%) SPARQL
13 5 (0.82%) Perl
14 4 (0.66%) C
14 4 (0.66%) PowerShell
14 4 (0.66%) T-SQL
15 3 (0.49%) Go
16 2 (0.33%) SAS Macro
Qualifications
1 101 (16.64%) Security Cleared
2 58 (9.56%) SC Cleared
3 52 (8.57%) Degree
4 25 (4.12%) Computer Science Degree
5 18 (2.97%) AWS Certification
6 14 (2.31%) DV Cleared
7 11 (1.81%) BPSS Clearance
8 10 (1.65%) Numerate Degree
9 9 (1.48%) AWS Certified Solutions Architect
10 7 (1.15%) Azure Certification
10 7 (1.15%) Master's Degree
11 6 (0.99%) PMI Certification
11 6 (0.99%) PMP
12 3 (0.49%) SnowPro
13 2 (0.33%) Microsoft Certification
13 2 (0.33%) Tableau Certification
14 1 (0.16%) ISTQB
14 1 (0.16%) ITIL Certification
14 1 (0.16%) Oracle Certification
14 1 (0.16%) PhD
Quality Assurance & Compliance
1 140 (23.06%) Data Quality
2 37 (6.10%) QA
3 30 (4.94%) GDPR
4 20 (3.29%) Accessibility
5 3 (0.49%) Actionable Recommendations
5 3 (0.49%) Disclosure Scotland
5 3 (0.49%) HIPAA
6 2 (0.33%) Sarbanes-Oxley
6 2 (0.33%) Solvency II
7 1 (0.16%) California Consumer Privacy Act
7 1 (0.16%) GxP
7 1 (0.16%) HL7
7 1 (0.16%) ISO 20022
7 1 (0.16%) ISO/IEC 27001
7 1 (0.16%) WCAG
System Software
1 72 (11.86%) Docker
2 2 (0.33%) HDFS
3 1 (0.16%) Tivoli Workload Scheduler
Systems Management
1 88 (14.50%) Kubernetes
2 74 (12.19%) Terraform
3 9 (1.48%) Kibana
4 7 (1.15%) Grafana
4 7 (1.15%) logstash
5 6 (0.99%) Ansible
6 5 (0.82%) Prometheus
7 4 (0.66%) Istio
7 4 (0.66%) Kustomize
8 2 (0.33%) Kubeflow
9 1 (0.16%) ITRS Geneos
9 1 (0.16%) Tivoli
Vendors
1 128 (21.09%) Databricks
2 55 (9.06%) Microsoft
3 52 (8.57%) Google
4 28 (4.61%) Oracle
5 18 (2.97%) Informatica
6 14 (2.31%) Palantir
7 10 (1.65%) Foundry
8 7 (1.15%) SAP
9 6 (0.99%) Alteryx
9 6 (0.99%) Talend
9 6 (0.99%) Teradata
10 5 (0.82%) Dataiku
10 5 (0.82%) DataRobot
10 5 (0.82%) MicroStrategy
10 5 (0.82%) Splunk
11 4 (0.66%) MuleSoft
12 3 (0.49%) OpenAI
12 3 (0.49%) ServiceNow
12 3 (0.49%) Sparx
13 2 (0.33%) Duck Creek