Data Pipeline Jobs

Data Pipelines
UK

The table below provides summary statistics for permanent job vacancies requiring Data Pipeline skills. It includes a benchmarking guide to the annual salaries offered in vacancies that cited Data Pipeline over the 6 months leading up to 30 April 2025, comparing them to the same period in the previous two years.

6 months to
30 Apr 2025
Same period 2024 Same period 2023
Rank 254 230 279
Rank change year-on-year -24 +49 -57
Permanent jobs citing Data Pipeline 606 1,267 1,338
As % of all permanent jobs advertised in the UK 1.16% 1.29% 1.29%
As % of the Processes & Methodologies category 1.25% 1.50% 1.35%
Number of salaries quoted 385 700 809
10th Percentile £45,300 £37,000 £46,850
25th Percentile £53,750 £49,750 £56,856
Median annual salary (50th Percentile) £69,500 £67,500 £69,447
Median % change year-on-year +2.96% -2.80% -7.40%
75th Percentile £90,000 £83,750 £87,500
90th Percentile £100,000 £105,000 £100,000
UK excluding London median annual salary £64,000 £60,000 £60,000
% change year-on-year +6.67% - -7.69%

All Process and Methodology Skills
UK

Data Pipeline falls under the Processes and Methodologies category. For comparison with the information above, the following table provides summary statistics for all permanent job vacancies requiring process or methodology skills.

Permanent vacancies with a requirement for process or methodology skills 48,641 84,574 99,457
As % of all permanent jobs advertised in the UK 93.26% 86.00% 95.63%
Number of salaries quoted 26,531 59,798 58,397
10th Percentile £30,580 £29,000 £34,000
25th Percentile £42,500 £40,000 £45,000
Median annual salary (50th Percentile) £57,500 £55,000 £61,500
Median % change year-on-year +4.55% -10.57% +2.50%
75th Percentile £76,250 £72,500 £82,500
90th Percentile £98,750 £92,500 £100,000
UK excluding London median annual salary £52,500 £50,000 £55,000
% change year-on-year +5.00% -9.09% +4.76%

Data Pipeline
Job Vacancy Trend

Job postings citing Data Pipeline as a proportion of all IT jobs advertised.

Job vacancy trend for Data Pipeline in the UK

Data Pipeline
Salary Trend

3-month moving average salary quoted in jobs citing Data Pipeline.

Salary trend for Data Pipeline in the UK

Data Pipeline
Salary Histogram

Salary distribution for jobs citing Data Pipeline over the 6 months to 30 April 2025.

Salary histogram for Data Pipeline in the UK

Data Pipeline
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Pipeline within the UK over the 6 months to 30 April 2025. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
England -3 552 £70,000 +3.70% 676
UK excluding London -40 291 £64,000 +6.67% 258
London +14 279 £85,000 +13.33% 445
Work from Home +28 247 £69,500 +1.66% 401
North of England -47 88 £64,000 +24.27% 76
Midlands +32 72 £65,000 +26.21% 35
South East +49 58 £82,500 +22.22% 59
Yorkshire +69 48 £62,500 +0.81% 24
West Midlands +36 43 £60,000 - 21
North West -73 36 £67,500 +35.00% 43
East Midlands +35 29 £65,000 +36.84% 14
South West +64 24 £57,500 -14.81% 40
Scotland -12 19 £45,000 -32.84% 22
East of England +44 18 £50,000 -20.00% 15
Northern Ireland +41 8 £55,000 -8.33% 7
North East +13 6 £73,750 +43.90% 9
Wales +24 5 £80,000 +52.38% 8

Data Pipeline
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 88 (14.52%) Apache Spark
2 56 (9.24%) Apache Airflow
3 29 (4.79%) Apache
4 10 (1.65%) Luigi
5 9 (1.49%) SAS
6 3 (0.50%) Jupyter
Applications
1 20 (3.30%) Microsoft Excel
2 3 (0.50%) Spreadsheet
3 2 (0.33%) Microsoft Office
3 2 (0.33%) MS Visio
Business Applications
1 3 (0.50%) NetSuite
2 1 (0.17%) Aladdin
2 1 (0.17%) Carpe Diem
2 1 (0.17%) Elite 3E
2 1 (0.17%) Endur
2 1 (0.17%) SAP FI
Cloud Services
1 293 (48.35%) Azure
2 196 (32.34%) AWS
3 131 (21.62%) Power Platform
4 111 (18.32%) Azure Data Factory
4 111 (18.32%) GCP
5 103 (17.00%) Snowflake
6 66 (10.89%) Azure Synapse Analytics
7 57 (9.41%) Serverless
8 52 (8.58%) dbt
9 41 (6.77%) Amazon S3
10 31 (5.12%) AWS Lambda
11 24 (3.96%) GitHub
12 23 (3.80%) SaaS
13 20 (3.30%) Azure DevOps
14 19 (3.14%) Azure Logic Apps
14 19 (3.14%) Power Automate
15 18 (2.97%) Azure Functions
15 18 (2.97%) Cloud Computing
16 17 (2.81%) Amazon Kinesis
17 15 (2.48%) Microsoft Purview
Communications & Networking
1 13 (2.15%) HTTP
2 3 (0.50%) DHCP
2 3 (0.50%) DNS
3 2 (0.33%) Internet
4 1 (0.17%) 5G
Database & Business Intelligence
1 195 (32.18%) Data Warehouse
2 126 (20.79%) Power BI
3 122 (20.13%) Data Lake
4 75 (12.38%) SQL Server
5 73 (12.05%) Tableau
6 72 (11.88%) Big Data
7 65 (10.73%) NoSQL
8 55 (9.08%) Amazon Redshift
9 46 (7.59%) SQL Server Integration Services
10 41 (6.77%) Metadata
11 40 (6.60%) BigQuery
12 37 (6.11%) Relational Database
13 34 (5.61%) Hadoop
14 33 (5.45%) SQL Server Reporting Services
15 26 (4.29%) PostgreSQL
16 20 (3.30%) Amazon Athena
16 20 (3.30%) Azure SQL Database
17 18 (2.97%) Apache Hive
18 17 (2.81%) SQL Server Analysis Services
19 14 (2.31%) MongoDB
Development Applications
1 85 (14.03%) Git
2 23 (3.80%) Jenkins
3 10 (1.65%) CircleCI
4 9 (1.49%) JIRA
5 8 (1.32%) MLflow
5 8 (1.32%) SQL Server Management Studio (SSMS)
5 8 (1.32%) Visual Studio
6 7 (1.16%) GitLab
7 5 (0.83%) ERwin
8 4 (0.66%) JMeter
8 4 (0.66%) Locust
8 4 (0.66%) Postman
8 4 (0.66%) Robot Framework
8 4 (0.66%) Selenium
9 2 (0.33%) Artifactory
9 2 (0.33%) Atlassian Bamboo
9 2 (0.33%) ER Studio
9 2 (0.33%) kappa
9 2 (0.33%) Oracle Forms
9 2 (0.33%) Spinnaker
General
1 201 (33.17%) Social Skills
2 166 (27.39%) Analytical Skills
3 162 (26.73%) Finance
4 74 (12.21%) Retail
5 69 (11.39%) Banking
6 36 (5.94%) Law
7 35 (5.78%) Inclusion and Diversity
8 32 (5.28%) Public Sector
9 28 (4.62%) Legal
10 23 (3.80%) Marketing
11 21 (3.47%) Presentation Skills
12 19 (3.14%) Retail Banking
13 16 (2.64%) Manufacturing
14 13 (2.15%) Front Office
15 12 (1.98%) Electronics
16 9 (1.49%) Advertising
17 8 (1.32%) Investment Banking
18 6 (0.99%) Agriculture
19 5 (0.83%) Billing
19 5 (0.83%) Games
Job Titles
1 251 (41.42%) Data Engineer
2 188 (31.02%) Senior
3 101 (16.67%) Lead
4 96 (15.84%) Senior Data Engineer
5 72 (11.88%) Analyst
6 47 (7.76%) Data Analyst
7 45 (7.43%) Developer
8 38 (6.27%) Architect
9 31 (5.12%) Senior Analyst
10 30 (4.95%) Azure Engineer
11 29 (4.79%) Senior Data Analyst
12 26 (4.29%) Software Engineer
13 25 (4.13%) Azure Data Engineer
14 21 (3.47%) Data Architect
14 21 (3.47%) Lead Data Engineer
15 20 (3.30%) AWS Engineer
16 18 (2.97%) AWS Data Engineer
17 17 (2.81%) Graduate
17 17 (2.81%) Python Engineer
18 16 (2.64%) Data Science Graduate
Libraries, Frameworks & Software Standards
1 64 (10.56%) PySpark
2 63 (10.40%) Kafka
3 32 (5.28%) Pandas
4 23 (3.80%) GraphQL
4 23 (3.80%) JSON
4 23 (3.80%) React
5 21 (3.47%) REST
5 21 (3.47%) RESTful
6 15 (2.48%) scikit-learn
7 14 (2.31%) AngularJS
7 14 (2.31%) PyTorch
7 14 (2.31%) TensorFlow
8 13 (2.15%) Flask
9 11 (1.82%) .NET
9 11 (1.82%) NumPy
9 11 (1.82%) Web Services
10 10 (1.65%) pytest
11 9 (1.49%) Spring
12 8 (1.32%) Spark Streaming
12 8 (1.32%) YAML
Miscellaneous
1 40 (6.60%) Data Structures
2 36 (5.94%) Cloud Native
3 35 (5.78%) Product Ownership
4 28 (4.62%) Algorithms
5 23 (3.80%) Credit Risk
6 22 (3.63%) Management Information System
7 21 (3.47%) Distributed Systems
8 20 (3.30%) Legacy Systems
9 16 (2.64%) Learning Management System
10 11 (1.82%) Distributed Applications
10 11 (1.82%) Greenfield Project
11 10 (1.65%) Low-Code
12 9 (1.49%) Linux Command Line
13 8 (1.32%) Distributed Computing
13 8 (1.32%) Robotics
13 8 (1.32%) SCADA
14 7 (1.16%) Market Risk
15 6 (0.99%) FMCG
15 6 (0.99%) Public Cloud
16 5 (0.83%) Replication
Operating Systems
1 18 (2.97%) Linux
2 4 (0.66%) Windows
3 3 (0.50%) VMS
4 2 (0.33%) Unix
Processes & Methodologies
1 338 (55.78%) Analytics
2 279 (46.04%) ETL
3 267 (44.06%) Data Engineering
4 201 (33.17%) Problem-Solving
5 195 (32.18%) Data Modelling
6 176 (29.04%) Business Intelligence
7 165 (27.23%) Visualisation
8 147 (24.26%) CI/CD
9 125 (20.63%) Machine Learning
10 123 (20.30%) Data Science
11 120 (19.80%) Data Governance
12 110 (18.15%) Agile
13 109 (17.99%) Data Architecture
14 103 (17.00%) Decision-Making
15 102 (16.83%) DevOps
16 97 (16.01%) Artificial Intelligence
16 97 (16.01%) Mentoring
17 92 (15.18%) Data Visualisation
18 89 (14.69%) Data Management
19 79 (13.04%) Data Transformation
Programming Languages
1 422 (69.64%) Python
2 420 (69.31%) SQL
3 76 (12.54%) Java
4 42 (6.93%) Scala
5 40 (6.60%) R
6 35 (5.78%) C#
7 27 (4.46%) Data Analysis Expressions
8 24 (3.96%) Bash
9 20 (3.30%) PowerShell
10 19 (3.14%) JavaScript
11 16 (2.64%) T-SQL
12 10 (1.65%) C++
13 8 (1.32%) C
13 8 (1.32%) SPARQL
14 7 (1.16%) Bicep
14 7 (1.16%) Shell Script
15 6 (0.99%) PL/SQL
15 6 (0.99%) TypeScript
16 5 (0.83%) Go
17 3 (0.50%) Rust
Qualifications
1 153 (25.25%) Degree
2 50 (8.25%) Master's Degree
3 49 (8.09%) DBS Check
4 45 (7.43%) Computer Science Degree
5 44 (7.26%) Security Cleared
6 38 (6.27%) SC Cleared
7 30 (4.95%) PhD
8 15 (2.48%) Azure Certification
9 14 (2.31%) Numerate Degree
10 11 (1.82%) AWS Certification
11 8 (1.32%) DV Cleared
12 4 (0.66%) AWS Certified Solutions Architect
12 4 (0.66%) ISTQB
12 4 (0.66%) SnowPro
12 4 (0.66%) Tableau Certification
13 3 (0.50%) Microsoft Certification
14 2 (0.33%) PMI Certification
14 2 (0.33%) PMP
15 1 (0.17%) HND
15 1 (0.17%) ITIL Certification
Quality Assurance & Compliance
1 132 (21.78%) Data Quality
2 42 (6.93%) Accessibility
3 28 (4.62%) QA
4 27 (4.46%) GDPR
5 9 (1.49%) PCI DSS
6 8 (1.32%) SLA
7 7 (1.16%) Basel II
7 7 (1.16%) Basel III
7 7 (1.16%) Dodd-Frank
7 7 (1.16%) IFRS
7 7 (1.16%) IFRS 9
8 6 (0.99%) ISO/IEC 27001
9 4 (0.66%) SOC 2
10 3 (0.50%) Actionable Recommendations
11 2 (0.33%) Cyber Essentials
12 1 (0.17%) California Consumer Privacy Act
12 1 (0.17%) HIPAA
12 1 (0.17%) NCSC
System Software
1 57 (9.41%) Docker
2 5 (0.83%) Apache Flume
3 3 (0.50%) HDFS
3 3 (0.50%) Virtual Machines
4 2 (0.33%) Active Directory
Systems Management
1 52 (8.58%) Kubernetes
2 49 (8.09%) Terraform
3 17 (2.81%) Grafana
3 17 (2.81%) Prometheus
4 9 (1.49%) Oozie
5 5 (0.83%) Ansible
6 4 (0.66%) Kubeflow
7 3 (0.50%) Argo
7 3 (0.50%) Control-M
7 3 (0.50%) Packer
8 2 (0.33%) ITRS Geneos
8 2 (0.33%) Kibana
8 2 (0.33%) Nagios
Vendors
1 177 (29.21%) Databricks
2 91 (15.02%) Microsoft
3 43 (7.10%) Google
4 18 (2.97%) Oracle
5 14 (2.31%) SAP
6 12 (1.98%) Palantir
7 6 (0.99%) Cloudera
7 6 (0.99%) Foundry
7 6 (0.99%) Salesforce
8 5 (0.83%) Splunk
9 4 (0.66%) Adobe
9 4 (0.66%) Informatica
9 4 (0.66%) Talend
9 4 (0.66%) TikTok
10 3 (0.50%) Alteryx
10 3 (0.50%) Nintex
10 3 (0.50%) Snowplow
10 3 (0.50%) StreamSets
11 2 (0.33%) Amplitude
11 2 (0.33%) Dynatrace