Period
to

The following table provides summary statistics for permanent job vacancies with a requirement for Data Pipeline skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Pipeline over the 6 months to 1 July 2022 with a comparison to the same period in the previous 2 years.

Data Pipelines
UK
6 months to
1 Jul 2022
Same period 2021 Same period 2020
Rank 205 269 347
Rank change year-on-year +64 +78 +217
Permanent jobs citing Data Pipeline 3,361 1,357 675
As % of all permanent jobs advertised in the UK 1.87% 1.27% 0.86%
As % of the Processes & Methodologies category 1.96% 1.35% 0.92%
Number of salaries quoted 1,108 886 518
10th Percentile £46,775 £42,500 £45,000
25th Percentile £57,500 £50,750 £55,688
Median annual salary (50th Percentile) £75,000 £70,000 £70,000
Median % change year-on-year +7.14% - +3.70%
75th Percentile £87,500 £85,000 £82,188
90th Percentile £99,125 £95,000 £91,250
UK excluding London median annual salary £68,500 £57,500 £62,250
% change year-on-year +19.13% -7.63% +8.26%

Data Pipeline is in the Processes and Methodologies category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for process or methodology skills.

All Process and Methodology Skills
UK
Permanent vacancies with a requirement for process or methodology skills 171,218 100,289 73,335
As % of all permanent IT jobs advertised in the UK 95.33% 93.83% 92.91%
Number of salaries quoted 84,584 69,434 59,179
10th Percentile £33,750 £32,500 £31,750
25th Percentile £45,000 £41,744 £41,250
Median annual salary (50th Percentile) £60,000 £55,000 £55,000
Median % change year-on-year +9.09% - +4.76%
75th Percentile £80,000 £75,000 £73,750
90th Percentile £96,250 £91,250 £90,000
UK excluding London median annual salary £52,500 £49,500 £47,500
% change year-on-year +6.06% +4.21% +5.56%

Data Pipeline
Job Vacancy Trend

Job postings citing Data Pipeline as a proportion of all IT jobs advertised.

Job vacancy trend for Data Pipeline in the UK

Data Pipeline
Salary Trend

3-month moving average salary quoted in jobs citing Data Pipeline.

Salary trend for Data Pipeline in the UK

Data Pipeline
Salary Histogram

Salary distribution for jobs citing Data Pipeline over the 6 months to 1 July 2022.

Salary histogram for Data Pipeline in the UK

Data Pipeline
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Pipeline within the UK over the 6 months to 1 July 2022. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
England +58 2,811 £77,000 +10.00% 178
London +26 2,008 £80,000 +6.67% 97
Work from Home +93 1,701 £80,000 +14.29% 72
UK excluding London +70 866 £68,500 +19.13% 94
North of England +51 252 £80,000 +33.33% 38
North West +32 161 £77,500 +29.17% 12
South East -45 154 £85,000 +41.67% 14
Midlands +25 153 £67,500 +35.00% 9
West Midlands +13 118 £68,750 +44.74% 8
South West +35 112 £50,000 +11.11% 6
East of England -57 83 £65,000 -11.86% 13
Yorkshire -17 77 £74,500 +29.57% 20
Scotland -13 75 £52,500 +0.96% 9
Wales +20 58 £50,000 -2.44% 2
East Midlands -23 35 £67,500 +25.58% 1
North East - 15 £85,000 - 6
Northern Ireland - 2 £69,500 - 3

For the 6 months to 1 July 2022, IT jobs citing Data Pipeline also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Data Pipeline.

1 2,427 (72.21%) Python
2 1,955 (58.17%) SQL
3 1,793 (53.35%) AWS
4 1,578 (46.95%) Analytics
5 1,297 (38.59%) Data Engineering
6 1,265 (37.64%) ETL
7 1,227 (36.51%) Data Science
8 1,185 (35.26%) Agile
9 1,114 (33.14%) Data Warehouse
10 1,056 (31.42%) Azure
11 958 (28.50%) GCP
12 909 (27.05%) Big Data
13 883 (26.27%) CI/CD
14 841 (25.02%) Finance
15 810 (24.10%) Machine Learning
16 806 (23.98%) Java
17 776 (23.09%) Data Modelling
18 772 (22.97%) Apache Spark
19 765 (22.76%) Business Intelligence
19 765 (22.76%) Software Engineering
20 757 (22.52%) Data Lake
21 747 (22.23%) Social Skills
22 743 (22.11%) Kafka
23 677 (20.14%) Apache Airflow
24 674 (20.05%) Kubernetes
25 644 (19.16%) Analytical Skills
26 597 (17.76%) NoSQL
27 561 (16.69%) Visualisation
28 542 (16.13%) DevOps
29 537 (15.98%) Problem-Solving

Data Pipeline
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 772 (22.97%) Apache Spark
2 677 (20.14%) Apache Airflow
3 335 (9.97%) Apache
4 120 (3.57%) Apache Solr
5 109 (3.24%) Luigi
6 42 (1.25%) Jupyter
7 41 (1.22%) Oracle Workflow
8 39 (1.16%) Confluence
9 27 (0.80%) Apache Pig
10 19 (0.57%) SAS
11 12 (0.36%) SharePoint
11 12 (0.36%) Tornado
12 7 (0.21%) Ethereum
13 6 (0.18%) Apache Pulsar
14 5 (0.15%) nginx
15 2 (0.060%) Solana
16 1 (0.030%) CICS
16 1 (0.030%) SharePoint Server
Applications
1 69 (2.05%) Microsoft Excel
2 32 (0.95%) Microsoft Office
3 28 (0.83%) Microsoft PowerPoint
4 8 (0.24%) Spreadsheet
Business Applications
1 19 (0.57%) Oracle Payables
2 16 (0.48%) Salesforce.com CRM
3 11 (0.33%) Shopify
4 8 (0.24%) NetSuite
4 8 (0.24%) SAP S/4HANA
5 6 (0.18%) JDE OneWorld
5 6 (0.18%) SAP Oil and Gas
6 5 (0.15%) Dynamics CRM
7 1 (0.030%) Cognos Business Intelligence
7 1 (0.030%) Dynamics AX
Cloud Services
1 1,793 (53.35%) AWS
2 1,056 (31.42%) Azure
3 958 (28.50%) GCP
4 507 (15.08%) Snowflake
5 458 (13.63%) Power Platform
6 390 (11.60%) Amazon S3
7 306 (9.10%) Serverless
8 261 (7.77%) Azure Data Factory
9 249 (7.41%) Amazon EMR
10 219 (6.52%) dbt
11 205 (6.10%) GitHub
12 170 (5.06%) AWS Lambda
13 164 (4.88%) Amazon Kinesis
14 148 (4.40%) Azure Synapse Analytics
15 131 (3.90%) Azure Cosmos DB
16 130 (3.87%) Azure DevOps
17 111 (3.30%) Matillion
18 106 (3.15%) SaaS
19 103 (3.06%) AWS CloudFormation
20 92 (2.74%) Google Data Studio
Communications & Networking
1 34 (1.01%) Internet
2 17 (0.51%) Broadband
3 15 (0.45%) Wireless
4 6 (0.18%) HTTP
4 6 (0.18%) Network Security
5 5 (0.15%) Skype
6 3 (0.089%) Wi-Fi
7 2 (0.060%) Kerberos
8 1 (0.030%) AMQP
8 1 (0.030%) Intrusion Detection
Database & Business Intelligence
1 1,114 (33.14%) Data Warehouse
2 909 (27.05%) Big Data
3 757 (22.52%) Data Lake
4 597 (17.76%) NoSQL
5 456 (13.57%) Power BI
6 451 (13.42%) Hadoop
7 444 (13.21%) Amazon Redshift
8 433 (12.88%) PostgreSQL
9 429 (12.76%) BigQuery
10 417 (12.41%) Tableau
11 362 (10.77%) Relational Database
12 326 (9.70%) SQL Server
13 275 (8.18%) Looker
14 248 (7.38%) MySQL
15 214 (6.37%) MongoDB
16 210 (6.25%) Redis
17 191 (5.68%) Apache Hive
18 183 (5.44%) Amazon Athena
19 162 (4.82%) Elasticsearch
20 161 (4.79%) SQL Server Integration Services
Development Applications
1 310 (9.22%) Git (software)
2 200 (5.95%) Jenkins
3 119 (3.54%) JIRA
4 117 (3.48%) GitLab
5 115 (3.42%) CircleCI
6 35 (1.04%) Visual Studio
7 24 (0.71%) Atlassian Bamboo
7 24 (0.71%) Selenium
8 22 (0.65%) ERwin
9 20 (0.60%) Bitbucket
9 20 (0.60%) ER Studio
10 15 (0.45%) Cucumber
11 13 (0.39%) RStudio
12 12 (0.36%) LoadRunner
13 9 (0.27%) MLflow
13 9 (0.27%) Postman
13 9 (0.27%) PowerDesigner
14 8 (0.24%) Cypress.io
14 8 (0.24%) Spinnaker
15 7 (0.21%) NUnit
General
1 841 (25.02%) Finance
2 747 (22.23%) Social Skills
3 644 (19.16%) Analytical Skills
4 311 (9.25%) Marketing
5 220 (6.55%) Inclusion and Diversity
6 198 (5.89%) Games
7 192 (5.71%) Retail
8 182 (5.42%) Banking
9 145 (4.31%) Advertising
10 100 (2.98%) Law
11 81 (2.41%) Public Sector
12 76 (2.26%) Telecoms
13 74 (2.20%) Financial Institution
14 66 (1.96%) Automotive
15 56 (1.67%) Influencing Skills
16 53 (1.58%) Manufacturing
17 45 (1.34%) Organisational Skills
18 44 (1.31%) Documentation Skills
19 42 (1.25%) Legal
20 38 (1.13%) Presentation Skills
Job Titles
1 1,805 (53.70%) Data Engineer
2 501 (14.91%) Senior Data Engineer
3 490 (14.58%) Developer
4 223 (6.63%) Software Engineer
5 193 (5.74%) Senior Developer
6 163 (4.85%) Java Developer
7 155 (4.61%) Analyst
8 127 (3.78%) Architect
9 124 (3.69%) Data Analyst
10 123 (3.66%) Senior Java Developer
11 111 (3.30%) Data Scientist
12 103 (3.06%) Python Engineer
13 96 (2.86%) Big Data Engineer
14 93 (2.77%) Senior Analyst
15 90 (2.68%) Azure Engineer
16 87 (2.59%) Azure Data Engineer
16 87 (2.59%) Python Developer
17 81 (2.41%) Senior Data Analyst
18 79 (2.35%) Data Manager
19 77 (2.29%) Senior Software Engineer
Libraries, Frameworks & Software Standards
1 743 (22.11%) Kafka
2 299 (8.90%) PySpark
3 192 (5.71%) Spring
4 190 (5.65%) React
5 151 (4.49%) Pandas
6 128 (3.81%) REST
7 123 (3.66%) Apache Beam
8 120 (3.57%) Django
9 117 (3.48%) Node.js
10 108 (3.21%) .NET
11 104 (3.09%) NumPy
12 82 (2.44%) Spark Streaming
13 80 (2.38%) AngularJS
14 71 (2.11%) Flask
15 68 (2.02%) Apache NiFi
16 58 (1.73%) CSS
16 58 (1.73%) scikit-learn
17 55 (1.64%) Spring Boot
18 51 (1.52%) RESTful
19 50 (1.49%) Web Services
Miscellaneous
1 302 (8.99%) Algorithms
2 242 (7.20%) Data Structures
3 175 (5.21%) Cloud Native
4 144 (4.28%) Public Cloud
5 138 (4.11%) Social Media
6 136 (4.05%) Self-Motivation
7 104 (3.09%) User Experience
8 101 (3.01%) Greenfield Project
9 84 (2.50%) Management Information System
10 82 (2.44%) Hackathon
11 62 (1.84%) IoT
12 59 (1.76%) Virtual Team
13 44 (1.31%) Mobile App
14 42 (1.25%) Blog
15 30 (0.89%) Linux Command Line
16 29 (0.86%) Legacy Systems
17 27 (0.80%) Blockchain
18 22 (0.65%) Educational Technology
19 20 (0.60%) Life Science
19 20 (0.60%) Sports Betting
Operating Systems
1 229 (6.81%) Linux
2 74 (2.20%) Unix
3 53 (1.58%) Windows
4 35 (1.04%) Android
5 28 (0.83%) Apple iOS
6 15 (0.45%) Mac OS
7 12 (0.36%) CentOS
8 8 (0.24%) Red Hat Enterprise Linux
9 7 (0.21%) Ubuntu
10 3 (0.089%) Windows Server
11 1 (0.030%) zOS
Processes & Methodologies
1 1,578 (46.95%) Analytics
2 1,297 (38.59%) Data Engineering
3 1,265 (37.64%) ETL
4 1,227 (36.51%) Data Science
5 1,185 (35.26%) Agile
6 883 (26.27%) CI/CD
7 810 (24.10%) Machine Learning
8 776 (23.09%) Data Modelling
9 765 (22.76%) Business Intelligence
9 765 (22.76%) Software Engineering
10 561 (16.69%) Visualisation
11 542 (16.13%) DevOps
12 537 (15.98%) Problem-Solving
13 498 (14.82%) Computer Science
14 496 (14.76%) Data Analytics
15 494 (14.70%) Data Architecture
16 450 (13.39%) Data Management
17 430 (12.79%) Artificial Intelligence
18 360 (10.71%) Data Analysis
18 360 (10.71%) Decision-Making
Programming Languages
1 2,427 (72.21%) Python
2 1,955 (58.17%) SQL
3 806 (23.98%) Java
4 404 (12.02%) Scala
5 310 (9.22%) R
6 262 (7.80%) JavaScript
7 197 (5.86%) C#
8 131 (3.90%) Bash
9 127 (3.78%) TypeScript
10 126 (3.75%) C++
11 109 (3.24%) Go
12 102 (3.03%) PowerShell
13 93 (2.77%) C
14 82 (2.44%) Ruby
15 70 (2.08%) T-SQL
16 61 (1.81%) Shell Script
17 25 (0.74%) VBA
18 22 (0.65%) Kotlin
18 22 (0.65%) Rust
19 18 (0.54%) PHP
Qualifications
1 496 (14.76%) Degree
2 192 (5.71%) Computer Science Degree
3 90 (2.68%) Master's Degree
4 88 (2.62%) PhD
5 54 (1.61%) Security Cleared
6 43 (1.28%) AWS Certification
7 38 (1.13%) SC Cleared
8 15 (0.45%) Numerate Degree
9 14 (0.42%) PMI Certification
10 13 (0.39%) Sun Certification
11 11 (0.33%) BPSS Clearance
12 9 (0.27%) Microsoft Certification
13 8 (0.24%) IBM Certification
14 7 (0.21%) DBS Check
15 2 (0.060%) CBAP
15 2 (0.060%) FFIEC
15 2 (0.060%) ISTQB
15 2 (0.060%) ISTQB Foundation Certification
15 2 (0.060%) PMP
16 1 (0.030%) CTC Cleared
Quality Assurance & Compliance
1 448 (13.33%) Data Quality
2 123 (3.66%) QA
3 47 (1.40%) GDPR
4 23 (0.68%) Accessibility
5 18 (0.54%) SLA
6 16 (0.48%) HIPAA
7 15 (0.45%) Actionable Recommendations
8 8 (0.24%) ISO/IEC 27001
9 6 (0.18%) SOC 2
10 4 (0.12%) GxP
11 2 (0.060%) GLBA
11 2 (0.060%) NIST
11 2 (0.060%) PCI DSS
12 1 (0.030%) HL7
System Software
1 485 (14.43%) Docker
2 50 (1.49%) Apache Flume
3 20 (0.60%) Virtual Machines
4 13 (0.39%) ISA Server
5 12 (0.36%) HDFS
6 9 (0.27%) Apache ZooKeeper
7 6 (0.18%) LXC
8 4 (0.12%) Active Directory
8 4 (0.12%) VirtualBox
8 4 (0.12%) zsh
Systems Management
1 674 (20.05%) Kubernetes
2 464 (13.81%) Terraform
3 90 (2.68%) Ansible
4 47 (1.40%) Oozie
5 36 (1.07%) Argo
6 26 (0.77%) Grafana
7 21 (0.62%) Prometheus
8 13 (0.39%) Consul
9 12 (0.36%) Opscode Chef
10 8 (0.24%) Marathon
11 7 (0.21%) Puppet
12 6 (0.18%) Panaseer
12 6 (0.18%) Proxmox
13 4 (0.12%) Control-M
13 4 (0.12%) Docker Swarm
13 4 (0.12%) Packer
13 4 (0.12%) SCCM
14 3 (0.089%) Kubeflow
15 2 (0.060%) Hadoop YARN
15 2 (0.060%) Istio
Vendors
1 496 (14.76%) Databricks
2 495 (14.73%) Google
3 285 (8.48%) Microsoft
4 208 (6.19%) Oracle
5 145 (4.31%) Twitter
6 107 (3.18%) Salesforce.com
7 74 (2.20%) Teradata
8 68 (2.02%) Cloudera
9 56 (1.67%) Informatica
10 55 (1.64%) Palantir
11 49 (1.46%) Splunk
12 46 (1.37%) Snapchat
13 43 (1.28%) Funding Circle
14 37 (1.10%) IBM
15 33 (0.98%) Facebook
16 31 (0.92%) Revolut
17 28 (0.83%) Alteryx
18 27 (0.80%) Talend
19 26 (0.77%) CA
20 22 (0.65%) SAP