Data Pipeline Jobs

Data Pipelines
UK

The table below provides summary statistics for permanent job vacancies requiring Data Pipeline skills. It includes a benchmarking guide to the annual salaries offered in vacancies that cited Data Pipeline over the 6 months leading up to 25 June 2024, comparing them to the same period in the previous two years.

6 months to
25 Jun 2024
Same period 2023 Same period 2022
Rank 231 281 211
Rank change year-on-year +50 -70 +82
Permanent jobs citing Data Pipeline 1,294 1,182 3,337
As % of all permanent jobs advertised in the UK 1.18% 1.28% 1.93%
As % of the Processes & Methodologies category 1.42% 1.34% 2.01%
Number of salaries quoted 704 691 1,064
10th Percentile £37,000 £46,250 £46,250
25th Percentile £50,000 £56,250 £57,500
Median annual salary (50th Percentile) £67,500 £70,000 £75,000
Median % change year-on-year -3.57% -6.67% +11.11%
75th Percentile £85,000 £87,419 £87,500
90th Percentile £103,925 £100,000 £100,000
UK excluding London median annual salary £60,000 £61,000 £65,000
% change year-on-year -1.64% -6.15% +13.04%

All Process and Methodology Skills
UK

Data Pipeline falls under the Processes and Methodologies category. For comparison with the information above, the following table provides summary statistics for all permanent job vacancies requiring process or methodology skills.

Permanent vacancies with a requirement for process or methodology skills 90,810 88,242 165,721
As % of all permanent jobs advertised in the UK 82.65% 95.42% 95.92%
Number of salaries quoted 62,635 53,650 83,244
10th Percentile £29,750 £33,818 £33,500
25th Percentile £40,000 £45,000 £43,896
Median annual salary (50th Percentile) £55,000 £60,899 £60,000
Median % change year-on-year -9.69% +1.50% +9.09%
75th Percentile £72,500 £81,250 £80,000
90th Percentile £92,500 £100,000 £96,250
UK excluding London median annual salary £50,000 £55,000 £52,500
% change year-on-year -9.09% +4.76% +8.81%

Data Pipeline
Job Vacancy Trend

Job postings citing Data Pipeline as a proportion of all IT jobs advertised.

Job vacancy trend for Data Pipeline in the UK

Data Pipeline
Salary Trend

3-month moving average salary quoted in jobs citing Data Pipeline.

Salary trend for Data Pipeline in the UK

Data Pipeline
Salary Histogram

Salary distribution for jobs citing Data Pipeline over the 6 months to 25 June 2024.

Salary histogram for Data Pipeline in the UK

Data Pipeline
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Pipeline within the UK over the 6 months to 25 June 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
England +44 1,082 £67,500 -3.57% 173
UK excluding London +29 741 £60,000 -1.64% 77
Work from Home +14 542 £67,500 -1.92% 80
London +78 503 £75,000 -3.23% 113
North of England +10 236 £52,500 -19.23% 23
Scotland +23 163 £66,500 +2.31% 7
North West +43 149 £50,000 -10.80% 16
South East +9 112 £67,500 +12.50% 8
Midlands -1 111 £52,000 -9.57% 18
South West -70 74 £67,500 -1.46% 12
West Midlands -4 59 £60,000 - 12
East Midlands +2 52 £40,000 -27.27% 6
Yorkshire -30 47 £70,000 +0.80% 6
North East +20 40 £52,500 +34.62% 1
East of England +6 30 £57,500 -8.00% 6
Wales +14 16 £46,131 +6.05% 3
Northern Ireland +9 3 £60,000 -20.00% 3

Data Pipeline
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 148 (11.44%) Apache Spark
2 90 (6.96%) Apache
3 72 (5.56%) Apache Airflow
4 12 (0.93%) SAS
5 7 (0.54%) Confluence
6 5 (0.39%) Luigi
6 5 (0.39%) SharePoint
7 3 (0.23%) Microsoft Exchange
8 1 (0.077%) Apache Pulsar
Applications
1 24 (1.85%) Microsoft Excel
2 9 (0.70%) Microsoft Office
3 4 (0.31%) Spreadsheet
4 3 (0.23%) Microsoft PowerPoint
4 3 (0.23%) MS Visio
5 1 (0.077%) Revit
Business Applications
1 4 (0.31%) Primavera
2 1 (0.077%) Salesforce CRM
2 1 (0.077%) SAP S/4HANA
2 1 (0.077%) Shopify
Cloud Services
1 504 (38.95%) AWS
2 468 (36.17%) Azure
3 355 (27.43%) GCP
4 203 (15.69%) Power Platform
5 166 (12.83%) Azure Data Factory
6 153 (11.82%) Snowflake
7 84 (6.49%) Serverless
8 76 (5.87%) Azure Synapse Analytics
9 64 (4.95%) Amazon S3
10 63 (4.87%) dbt
11 56 (4.33%) Matillion
12 54 (4.17%) Cloud Computing
13 44 (3.40%) Azure DevOps
14 40 (3.09%) AWS Lambda
15 37 (2.86%) GitHub
16 36 (2.78%) Azure Cosmos DB
17 28 (2.16%) Amazon EMR
18 21 (1.62%) SaaS
19 20 (1.55%) AWS Step Functions
19 20 (1.55%) Fivetran
Communications & Networking
1 65 (5.02%) Kerberos
2 10 (0.77%) DSL
3 4 (0.31%) Firewall
4 2 (0.15%) Intrusion Detection
4 2 (0.15%) Network Security
4 2 (0.15%) VPN
5 1 (0.077%) 3GPP
5 1 (0.077%) 5G
5 1 (0.077%) PPP
Database & Business Intelligence
1 358 (27.67%) Data Warehouse
2 241 (18.62%) Data Lake
3 227 (17.54%) BigQuery
4 212 (16.38%) Tableau
5 199 (15.38%) Power BI
6 188 (14.53%) Looker
7 144 (11.13%) SQL Server
8 130 (10.05%) Big Data
9 124 (9.58%) Metadata
10 120 (9.27%) NoSQL
11 76 (5.87%) SQL Server Integration Services
12 67 (5.18%) Amazon Redshift
13 56 (4.33%) PostgreSQL
14 52 (4.02%) Elasticsearch
15 51 (3.94%) Azure SQL Database
16 49 (3.79%) Hadoop
17 43 (3.32%) Azure SQL Data Warehouse
18 40 (3.09%) Star Schema
19 35 (2.70%) Relational Database
20 33 (2.55%) Data Mining
Development Applications
1 286 (22.10%) Git
2 103 (7.96%) Jenkins
3 70 (5.41%) Bitbucket
4 65 (5.02%) Artifactory
5 49 (3.79%) GitLab
6 14 (1.08%) Spinnaker
7 12 (0.93%) JIRA
8 7 (0.54%) CircleCI
8 7 (0.54%) Oracle Forms
9 6 (0.46%) RStudio
10 5 (0.39%) GDB
11 4 (0.31%) Octopus Deploy
11 4 (0.31%) Visual Studio
12 3 (0.23%) MLflow
13 2 (0.15%) webpack
14 1 (0.077%) ERwin
14 1 (0.077%) Mockito
14 1 (0.077%) SpecFlow
14 1 (0.077%) TeamCity
General
1 445 (34.39%) Analytical Skills
2 365 (28.21%) Social Skills
3 282 (21.79%) Finance
4 178 (13.76%) Inclusion and Diversity
5 167 (12.91%) Retail
6 165 (12.75%) Presentation Skills
7 147 (11.36%) Law
8 105 (8.11%) Marketing
9 62 (4.79%) Banking
10 43 (3.32%) Legal
11 41 (3.17%) Public Sector
12 21 (1.62%) Games
13 16 (1.24%) Billing
14 14 (1.08%) Front Office
15 13 (1.00%) Investment Banking
16 12 (0.93%) Back Office
16 12 (0.93%) Financial Institution
17 11 (0.85%) Influencing Skills
17 11 (0.85%) Telecoms
18 7 (0.54%) Health Technology
Job Titles
1 670 (51.78%) Data Engineer
2 372 (28.75%) Senior
3 182 (14.06%) Principal Engineer
4 180 (13.91%) Principal Data Engineer
5 177 (13.68%) Lead
5 177 (13.68%) Platform Engineer
6 159 (12.29%) Analyst
7 153 (11.82%) Data Analyst
8 134 (10.36%) Senior Data Engineer
9 128 (9.89%) Software Engineer
10 103 (7.96%) Senior Analyst
10 103 (7.96%) Senior Data Analyst
11 86 (6.65%) Developer
12 71 (5.49%) Architect
13 69 (5.33%) Lead Software Engineer
14 65 (5.02%) Data Architect
15 60 (4.64%) Senior Software Engineer
16 54 (4.17%) AWS Engineer
17 53 (4.10%) Lead Data Engineer
18 48 (3.71%) Azure Engineer
Libraries, Frameworks & Software Standards
1 86 (6.65%) PySpark
2 83 (6.41%) Kafka
3 77 (5.95%) XML
4 75 (5.80%) JSON
5 72 (5.56%) RESTful
6 71 (5.49%) Web Services
7 65 (5.02%) JSON-LD
7 65 (5.02%) LDAP
7 65 (5.02%) OpenID
7 65 (5.02%) SAML
8 50 (3.86%) Pandas
9 32 (2.47%) React
10 31 (2.40%) Apache Beam
11 21 (1.62%) PyTorch
11 21 (1.62%) TensorFlow
12 19 (1.47%) .NET
12 19 (1.47%) XGBoost
13 17 (1.31%) REST
14 15 (1.16%) .NET Core
15 13 (1.00%) Apache NiFi
Miscellaneous
1 215 (16.62%) Self-Motivation
2 107 (8.27%) Data Structures
3 94 (7.26%) Management Information System
4 80 (6.18%) Algorithms
5 65 (5.02%) Enterprise Cloud
5 65 (5.02%) Reference Data
6 59 (4.56%) Onboarding
7 56 (4.33%) Cloud Native
8 36 (2.78%) Product Ownership
9 20 (1.55%) Hedge funds
9 20 (1.55%) IoT
10 18 (1.39%) Public Cloud
11 14 (1.08%) Greenfield Project
11 14 (1.08%) Reinsurance
11 14 (1.08%) Taxonomies
12 11 (0.85%) Blog
12 11 (0.85%) Enterprise Software
12 11 (0.85%) Private Cloud
13 10 (0.77%) Distributed Systems
14 9 (0.70%) Client/Server
Operating Systems
1 40 (3.09%) Linux
2 20 (1.55%) Unix
3 2 (0.15%) Windows
4 1 (0.077%) Android
4 1 (0.077%) Windows Server
Processes & Methodologies
1 532 (41.11%) Agile
2 531 (41.04%) Data Engineering
3 515 (39.80%) Analytics
4 498 (38.49%) Problem-Solving
5 448 (34.62%) ETL
6 424 (32.77%) Data Science
7 393 (30.37%) Data Modelling
8 359 (27.74%) Visualisation
9 316 (24.42%) CI/CD
10 287 (22.18%) Data Analysis
11 279 (21.56%) DevOps
12 263 (20.32%) Mentoring
13 250 (19.32%) Machine Learning
14 247 (19.09%) Business Intelligence
15 219 (16.92%) Platform Engineering
16 216 (16.69%) Software Engineering
17 188 (14.53%) DataOps
18 180 (13.91%) Artificial Intelligence
18 180 (13.91%) Mathematics
19 168 (12.98%) Statistics
Programming Languages
1 811 (62.67%) SQL
2 734 (56.72%) Python
3 163 (12.60%) Java
4 95 (7.34%) Scala
5 83 (6.41%) Go
6 65 (5.02%) Prolog
6 65 (5.02%) SPARQL
7 55 (4.25%) T-SQL
8 36 (2.78%) C#
9 31 (2.40%) R
10 30 (2.32%) TypeScript
11 29 (2.24%) JavaScript
12 21 (1.62%) Kotlin
13 16 (1.24%) Data Analysis Expressions
14 12 (0.93%) PL/SQL
15 10 (0.77%) Clojure
16 9 (0.70%) C
17 7 (0.54%) MATLAB
18 6 (0.46%) PowerShell
19 4 (0.31%) C++
Qualifications
1 296 (22.87%) Degree
2 176 (13.60%) DBS Check
3 79 (6.11%) Computer Science Degree
4 49 (3.79%) Master's Degree
5 40 (3.09%) Security Cleared
6 32 (2.47%) SC Cleared
7 21 (1.62%) DV Cleared
8 15 (1.16%) Microsoft Certification
9 12 (0.93%) Azure Certification
10 10 (0.77%) PhD
11 5 (0.39%) Numerate Degree
12 4 (0.31%) CIMA
12 4 (0.31%) PMI Certification
13 3 (0.23%) AWS Certification
13 3 (0.23%) AWS Certified Solutions Architect
13 3 (0.23%) BPSS Clearance
14 2 (0.15%) SnowPro
15 1 (0.077%) Postgraduate
Quality Assurance & Compliance
1 193 (14.91%) Data Quality
2 183 (14.14%) GDPR
3 23 (1.78%) QA
4 12 (0.93%) Accessibility
5 10 (0.77%) SLA
6 4 (0.31%) Solvency II
7 3 (0.23%) ISO/IEC 27001
8 1 (0.077%) California Consumer Privacy Act
8 1 (0.077%) HIPAA
8 1 (0.077%) NCSC
System Software
1 69 (5.33%) Active Directory
2 48 (3.71%) Docker
3 7 (0.54%) HDFS
4 4 (0.31%) Virtual Machines
Systems Management
1 92 (7.11%) Kubernetes
2 53 (4.10%) Terraform
3 14 (1.08%) Ansible
4 7 (0.54%) Argo
5 4 (0.31%) Kubeflow
6 1 (0.077%) Grafana
6 1 (0.077%) Prometheus
Vendors
1 372 (28.75%) Databricks
2 233 (18.01%) Google
3 147 (11.36%) Microsoft
4 42 (3.25%) Oracle
5 22 (1.70%) Alteryx
5 22 (1.70%) Informatica
6 12 (0.93%) OpenAI
7 10 (0.77%) Cloudera
7 10 (0.77%) Cognos
7 10 (0.77%) SAP
7 10 (0.77%) Sybase
7 10 (0.77%) YouTube
8 9 (0.70%) Teradata
9 7 (0.54%) Bloomberg
10 6 (0.46%) Dataiku
10 6 (0.46%) F5
10 6 (0.46%) Talend
11 4 (0.31%) Hortonworks
12 3 (0.23%) SnapLogic
12 3 (0.23%) Splunk