Data Pipeline Jobs

Data Pipelines
UK

The table below provides summary statistics for permanent job vacancies requiring Data Pipeline skills. It includes a benchmarking guide to the annual salaries offered in vacancies that cited Data Pipeline over the 6 months leading up to 22 May 2025, comparing them to the same period in the previous two years.

6 months to
22 May 2025
Same period 2024 Same period 2023
Rank 285 230 269
Rank change year-on-year -55 +39 -55
Permanent jobs citing Data Pipeline 601 1,296 1,306
As % of all permanent jobs advertised in the UK 1.09% 1.30% 1.32%
As % of the Processes & Methodologies category 1.17% 1.52% 1.38%
Number of salaries quoted 375 687 723
10th Percentile £45,850 £36,900 £46,250
25th Percentile £55,000 £49,875 £56,250
Median annual salary (50th Percentile) £70,000 £67,500 £69,447
Median % change year-on-year +3.70% -2.80% -7.40%
75th Percentile £90,000 £85,000 £87,500
90th Percentile £105,000 £105,000 £100,000
UK excluding London median annual salary £64,000 £60,000 £60,000
% change year-on-year +6.67% - -7.69%

All Process and Methodology Skills
UK

Data Pipeline falls under the Processes and Methodologies category. For comparison with the information above, the following table provides summary statistics for all permanent job vacancies requiring process or methodology skills.

Permanent vacancies with a requirement for process or methodology skills 51,521 85,132 94,507
As % of all permanent jobs advertised in the UK 93.87% 85.14% 95.59%
Number of salaries quoted 25,367 59,760 55,710
10th Percentile £29,625 £29,078 £34,000
25th Percentile £41,250 £40,000 £45,000
Median annual salary (50th Percentile) £57,500 £55,000 £61,000
Median % change year-on-year +4.55% -9.84% +1.67%
75th Percentile £75,000 £72,500 £81,250
90th Percentile £97,500 £92,500 £100,000
UK excluding London median annual salary £51,645 £50,000 £55,000
% change year-on-year +3.29% -9.09% +4.76%

Data Pipeline
Job Vacancy Trend

Job postings citing Data Pipeline as a proportion of all IT jobs advertised.

Job vacancy trend for Data Pipeline in the UK

Data Pipeline
Salary Trend

3-month moving average salary quoted in jobs citing Data Pipeline.

Salary trend for Data Pipeline in the UK

Data Pipeline
Salary Histogram

Salary distribution for jobs citing Data Pipeline over the 6 months to 22 May 2025.

Salary histogram for Data Pipeline in the UK

Data Pipeline
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Pipeline within the UK over the 6 months to 22 May 2025. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
England -32 553 £70,000 +3.70% 1,467
UK excluding London -58 286 £64,000 +6.67% 841
London +1 281 £85,000 +13.33% 878
Work from Home -59 218 £70,000 +3.70% 718
North of England -66 83 £64,000 +24.27% 213
Midlands +14 68 £65,000 +25.00% 99
South East +47 61 £70,750 +4.81% 312
Yorkshire +67 44 £62,500 +4.60% 72
West Midlands +48 38 £62,500 +4.17% 59
North West -97 36 £67,500 +42.11% 112
East Midlands +7 30 £65,000 +62.50% 34
South West +51 23 £57,471 -14.86% 79
East of England +34 23 £50,000 -14.89% 60
Scotland -18 17 £52,500 -21.64% 65
Northern Ireland +46 8 £55,000 -8.33% 15
North East +8 5 £62,500 +19.05% 32
Wales +30 4 £81,250 +76.13% 23

Data Pipeline
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 89 (14.81%) Apache Spark
2 62 (10.32%) Apache Airflow
3 32 (5.32%) Apache
4 10 (1.66%) Luigi
5 7 (1.16%) SAS
6 5 (0.83%) Appian
7 2 (0.33%) Jupyter
8 1 (0.17%) SharePoint
Applications
1 21 (3.49%) Microsoft Excel
2 3 (0.50%) Spreadsheet
3 2 (0.33%) Microsoft Office
3 2 (0.33%) MS Visio
Business Applications
1 3 (0.50%) NetSuite
2 2 (0.33%) Carpe Diem
2 2 (0.33%) Elite 3E
3 1 (0.17%) Aladdin
3 1 (0.17%) Endur
3 1 (0.17%) SAP FI
Cloud Services
1 306 (50.92%) Azure
2 215 (35.77%) AWS
3 134 (22.30%) Power Platform
4 119 (19.80%) Snowflake
5 115 (19.13%) GCP
6 109 (18.14%) Azure Data Factory
7 62 (10.32%) Azure Synapse Analytics
8 54 (8.99%) Serverless
9 51 (8.49%) dbt
10 50 (8.32%) Amazon S3
11 32 (5.32%) AWS Lambda
12 27 (4.49%) GitHub
13 25 (4.16%) Azure DevOps
14 23 (3.83%) SaaS
15 21 (3.49%) Power Automate
16 19 (3.16%) Azure Logic Apps
17 18 (3.00%) Amazon Kinesis
17 18 (3.00%) Azure Functions
17 18 (3.00%) Cloud Computing
18 15 (2.50%) Microsoft Purview
Communications & Networking
1 9 (1.50%) HTTP
2 3 (0.50%) DHCP
2 3 (0.50%) DNS
2 3 (0.50%) Internet
3 1 (0.17%) PuTTY
Database & Business Intelligence
1 196 (32.61%) Data Warehouse
2 124 (20.63%) Power BI
3 110 (18.30%) Data Lake
4 74 (12.31%) SQL Server
4 74 (12.31%) Tableau
5 66 (10.98%) Big Data
6 61 (10.15%) NoSQL
7 58 (9.65%) Amazon Redshift
8 49 (8.15%) Metadata
9 46 (7.65%) SQL Server Integration Services
10 41 (6.82%) BigQuery
11 36 (5.99%) Relational Database
12 32 (5.32%) SQL Server Reporting Services
13 31 (5.16%) Hadoop
14 28 (4.66%) PostgreSQL
15 24 (3.99%) Amazon Athena
16 17 (2.83%) Apache Hive
16 17 (2.83%) Azure SQL Database
17 16 (2.66%) MongoDB
17 16 (2.66%) SQL Server Analysis Services
Development Applications
1 87 (14.48%) Git
2 22 (3.66%) Jenkins
3 9 (1.50%) JIRA
3 9 (1.50%) MLflow
3 9 (1.50%) Robot Framework
3 9 (1.50%) Selenium
3 9 (1.50%) Visual Studio
4 8 (1.33%) CircleCI
4 8 (1.33%) SQL Server Management Studio (SSMS)
5 5 (0.83%) ERwin
5 5 (0.83%) Snyk
6 4 (0.67%) JMeter
6 4 (0.67%) Locust
6 4 (0.67%) Postman
7 3 (0.50%) GitLab
7 3 (0.50%) Spinnaker
8 2 (0.33%) Artifactory
8 2 (0.33%) ER Studio
8 2 (0.33%) PowerDesigner
9 1 (0.17%) SpecFlow
General
1 207 (34.44%) Social Skills
2 157 (26.12%) Analytical Skills
3 155 (25.79%) Finance
4 75 (12.48%) Retail
5 58 (9.65%) Banking
6 55 (9.15%) Law
7 35 (5.82%) Inclusion and Diversity
8 32 (5.32%) Public Sector
9 28 (4.66%) Marketing
10 21 (3.49%) Presentation Skills
11 18 (3.00%) Retail Banking
12 15 (2.50%) Legal
13 13 (2.16%) Manufacturing
14 12 (2.00%) Front Office
15 10 (1.66%) Advertising
16 9 (1.50%) Electronics
16 9 (1.50%) Investment Banking
17 6 (1.00%) Agriculture
18 5 (0.83%) Billing
18 5 (0.83%) Tech for Good
Job Titles
1 254 (42.26%) Data Engineer
2 196 (32.61%) Senior
3 97 (16.14%) Senior Data Engineer
4 95 (15.81%) Lead
5 75 (12.48%) Analyst
6 48 (7.99%) Data Analyst
7 45 (7.49%) Developer
8 41 (6.82%) Architect
9 31 (5.16%) Azure Engineer
9 31 (5.16%) Senior Analyst
10 29 (4.83%) Senior Data Analyst
11 23 (3.83%) Azure Data Engineer
12 22 (3.66%) Software Engineer
13 19 (3.16%) Data Architect
14 18 (3.00%) Python Engineer
15 15 (2.50%) AWS Engineer
15 15 (2.50%) Lead Data Engineer
15 15 (2.50%) Python Developer
15 15 (2.50%) SQL Analyst
16 14 (2.33%) Lead Architect
Libraries, Frameworks & Software Standards
1 72 (11.98%) Kafka
2 70 (11.65%) PySpark
3 39 (6.49%) Pandas
4 22 (3.66%) React
5 21 (3.49%) REST
5 21 (3.49%) RESTful
6 20 (3.33%) GraphQL
7 18 (3.00%) JSON
8 15 (2.50%) AngularJS
9 14 (2.33%) pytest
9 14 (2.33%) scikit-learn
10 13 (2.16%) PyTorch
10 13 (2.16%) TensorFlow
11 12 (2.00%) NumPy
12 11 (1.83%) Web Services
13 10 (1.66%) Flask
14 9 (1.50%) .NET
14 9 (1.50%) Spark Streaming
14 9 (1.50%) YAML
15 8 (1.33%) Plotly
Miscellaneous
1 43 (7.15%) Cloud Native
2 37 (6.16%) Data Structures
3 34 (5.66%) Product Ownership
4 20 (3.33%) Legacy Systems
5 17 (2.83%) Algorithms
6 16 (2.66%) Distributed Systems
6 16 (2.66%) Management Information System
7 14 (2.33%) Low-Code
8 11 (1.83%) Credit Risk
8 11 (1.83%) FMCG
9 10 (1.66%) Distributed Applications
9 10 (1.66%) Greenfield Project
9 10 (1.66%) Linux Command Line
10 9 (1.50%) Replication
11 7 (1.16%) Distributed Computing
12 6 (1.00%) Learning Management System
13 5 (0.83%) Market Risk
13 5 (0.83%) Public Cloud
13 5 (0.83%) Reinsurance
14 4 (0.67%) Analytical Mindset
Operating Systems
1 18 (3.00%) Linux
2 4 (0.67%) Windows
3 3 (0.50%) VMS
4 2 (0.33%) Unix
Processes & Methodologies
1 332 (55.24%) Analytics
2 286 (47.59%) ETL
3 267 (44.43%) Data Engineering
4 218 (36.27%) Data Modelling
5 210 (34.94%) Problem-Solving
6 161 (26.79%) Business Intelligence
7 153 (25.46%) CI/CD
8 148 (24.63%) Visualisation
9 136 (22.63%) Data Governance
10 124 (20.63%) Data Science
11 118 (19.63%) Machine Learning
12 114 (18.97%) Data Architecture
13 108 (17.97%) DevOps
14 106 (17.64%) Agile
15 105 (17.47%) Decision-Making
16 101 (16.81%) Mentoring
17 95 (15.81%) Data Management
18 86 (14.31%) Artificial Intelligence
19 85 (14.14%) Data Transformation
20 78 (12.98%) Software Engineering
Programming Languages
1 422 (70.22%) Python
2 413 (68.72%) SQL
3 71 (11.81%) Java
4 40 (6.66%) Scala
5 34 (5.66%) C#
6 28 (4.66%) Data Analysis Expressions
6 28 (4.66%) R
7 26 (4.33%) Bash
8 19 (3.16%) PowerShell
9 18 (3.00%) JavaScript
10 17 (2.83%) T-SQL
11 9 (1.50%) Bicep
12 8 (1.33%) SPARQL
13 7 (1.16%) C++
13 7 (1.16%) Shell Script
14 5 (0.83%) PL/SQL
15 4 (0.67%) C
15 4 (0.67%) Cypher
15 4 (0.67%) TypeScript
16 3 (0.50%) VB
Qualifications
1 143 (23.79%) Degree
2 47 (7.82%) DBS Check
2 47 (7.82%) Security Cleared
3 45 (7.49%) Computer Science Degree
4 39 (6.49%) Master's Degree
5 37 (6.16%) SC Cleared
6 21 (3.49%) PhD
7 17 (2.83%) Azure Certification
8 13 (2.16%) AWS Certification
9 12 (2.00%) DV Cleared
10 9 (1.50%) Numerate Degree
11 5 (0.83%) AWS Certified Solutions Architect
12 4 (0.67%) ISTQB
12 4 (0.67%) Microsoft Certification
12 4 (0.67%) SnowPro
12 4 (0.67%) Tableau Certification
13 2 (0.33%) PMI Certification
13 2 (0.33%) PMP
14 1 (0.17%) HND
14 1 (0.17%) ITIL Certification
Quality Assurance & Compliance
1 146 (24.29%) Data Quality
2 43 (7.15%) QA
3 42 (6.99%) Accessibility
4 25 (4.16%) GDPR
5 7 (1.16%) ISO/IEC 27001
5 7 (1.16%) PCI DSS
6 5 (0.83%) Basel II
6 5 (0.83%) Basel III
6 5 (0.83%) Dodd-Frank
6 5 (0.83%) IFRS
6 5 (0.83%) IFRS 9
7 4 (0.67%) SLA
7 4 (0.67%) SOC 2
8 3 (0.50%) Actionable Recommendations
8 3 (0.50%) Cyber Essentials
9 1 (0.17%) California Consumer Privacy Act
9 1 (0.17%) Cyber Essentials PLUS
9 1 (0.17%) EU AI Act
9 1 (0.17%) NCSC
9 1 (0.17%) NIST
System Software
1 59 (9.82%) Docker
2 6 (1.00%) Apache Flume
3 4 (0.67%) HDFS
3 4 (0.67%) VMware Infrastructure
4 3 (0.50%) Virtual Machines
5 2 (0.33%) Active Directory
Systems Management
1 61 (10.15%) Terraform
2 51 (8.49%) Kubernetes
3 17 (2.83%) Grafana
3 17 (2.83%) Prometheus
4 10 (1.66%) Ansible
5 9 (1.50%) Oozie
6 3 (0.50%) Control-M
6 3 (0.50%) Kibana
6 3 (0.50%) Kubeflow
6 3 (0.50%) Packer
7 2 (0.33%) ITRS Geneos
7 2 (0.33%) Nagios
8 1 (0.17%) Argo
8 1 (0.17%) logstash
Vendors
1 178 (29.62%) Databricks
2 95 (15.81%) Microsoft
3 38 (6.32%) Google
4 18 (3.00%) Oracle
5 14 (2.33%) SAP
6 12 (2.00%) Palantir
7 7 (1.16%) Salesforce
7 7 (1.16%) Splunk
8 6 (1.00%) Cloudera
9 5 (0.83%) BiZZdesign
9 5 (0.83%) Black Duck
9 5 (0.83%) Informatica
9 5 (0.83%) UiPath
9 5 (0.83%) VMware
10 4 (0.67%) Adobe
10 4 (0.67%) Foundry
10 4 (0.67%) StreamSets
10 4 (0.67%) Talend
10 4 (0.67%) TikTok
11 3 (0.50%) Alteryx