Period
to 17 October 2019

The median contractor rate for a Data Engineer was £550 per day in advertised job vacancies during the 6 months to 17 October 2019.

The first table below provides contractor rate benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Data Engineer
UK
6 months to
17 Oct 2019
Same period 2018 Same period 2017
Rank 192 330 401
Rank change year-on-year +138 +71 +190
Contract jobs requiring a Data Engineer 1,226 820 593
As % of all contract IT jobs advertised in the UK 1.44% 0.79% 0.60%
As % of the Job Titles category 1.53% 0.84% 0.63%
Number of daily rates quoted 944 624 451
UK median daily rate £550 £543 £500
Median daily rate % change year-on-year +1.38% +8.50% -4.76%
10th Percentile £413 £385 £263
90th Percentile £688 £686 £638
UK excluding London median daily rate £500 £500 £450
% change year-on-year - +11.11% +28.57%
Number of hourly rates quoted 4 10 6
UK median hourly rate £44.25 £25.50 £50.63
Median hourly rate % change year-on-year +73.53% -49.63% +289.42%
10th Percentile £32.03 £15.00 £17.91
90th Percentile £53.75 £85.18 £106.25
UK excluding London median hourly rate £44.25 £18.50 £50.63
% change year-on-year +139.19% -63.46% +305.00%

The following table is for comparison with the above and includes summary statistics for all contract IT job vacancies. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of contract jobs in our overall sample.

All Contract IT Job Vacancies
UK
Contract vacancies in the UK with a recognized job title 80,094 97,771 94,259
% of contract IT jobs with a recognized job title 94.13% 94.76% 94.93%
Number of daily rates quoted 52,569 64,251 61,834
UK median daily rate £463 £450 £440
Median daily rate % change year-on-year +2.78% +2.27% +3.53%
10th Percentile £263 £268 £263
90th Percentile £645 £638 £625
UK excluding London median daily rate £425 £415 £400
% change year-on-year +2.41% +3.75% +2.56%
Number of hourly rates quoted 2,843 3,144 2,822
UK median hourly rate £20.00 £18.00 £18.70
Median hourly rate % change year-on-year +11.11% -3.74% +1.08%
10th Percentile £11.17 £11.00 £10.75
90th Percentile £52.50 £48.75 £48.75
UK excluding London median hourly rate £18.50 £17.50 £18.00
% change year-on-year +5.71% -2.78% +1.41%

Data Engineer
Job Vacancy Trend

Job postings that featured Data Engineer in the job title as a percentage of all IT jobs advertised.

Job vacancy trend for Data Engineer in the UK

Data Engineer
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Data Engineer.

Contractor daily rate trend for Data Engineer in the UK

Data Engineer
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing Data Engineer over the 6 months to 17 October 2019.

Contractor daily rate histogram for Data Engineer in the UK

Data Engineer
Contractor Hourly Rate Trend

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing Data Engineer.

Contractor hourly rate trend for Data Engineer in the UK

Data Engineer
Contractor Hourly Rate Histogram

The hourly rate distribution of IT jobs citing Data Engineer over the 6 months to 17 October 2019.

Contractor hourly rate histogram for Data Engineer in the UK

Data Engineer
Top 15 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Data Engineer within the UK over the 6 months to 17 October 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +124 1,163 £550 - 299
London +102 840 £550 - 196
UK excluding London +164 372 £500 - 134
South East +72 108 £500 - 39
North of England +58 80 £475 - 26
East of England +62 63 £550 +10.00% 7
Scotland +60 53 £475 +15.15% 26
North West +48 50 £450 -12.20% 15
South West +5 37 £500 -4.76% 14
Midlands +55 28 £450 -8.35% 17
Yorkshire +31 21 £475 - 8
West Midlands +45 15 £448 -9.50% 11
East Midlands +33 13 £450 -6.49% 6
North East +25 10 £500 +11.11% 3
Northern Ireland +8 3 £450 - 1

Data Engineer Skill Set
Top 30 Co-occurring IT Skills

For the 6 months to 17 October 2019, Data Engineer contract jobs required the following IT skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads featuring Data Engineer in the job title.

1 774 (63.13%) Python
2 570 (46.49%) SQL
3 560 (45.68%) Big Data
4 535 (43.64%) Apache Spark
5 470 (38.34%) AWS
6 420 (34.26%) Hadoop
7 366 (29.85%) ETL
8 334 (27.24%) Agile Software Development
9 328 (26.75%) Analytics
10 300 (24.47%) Azure
11 299 (24.39%) Data Engineering
12 298 (24.31%) Data Science
13 293 (23.90%) Scala
14 280 (22.84%) Kafka
15 268 (21.86%) Data Modelling
16 261 (21.29%) Java
17 260 (21.21%) Data Warehouse
18 257 (20.96%) GCP
19 237 (19.33%) Apache Hive
20 211 (17.21%) Finance
21 209 (17.05%) Google
22 207 (16.88%) DevOps
23 196 (15.99%) Docker
24 183 (14.93%) Machine Learning
25 168 (13.70%) Apache
26 167 (13.62%) Data Lake
27 165 (13.46%) Business Intelligence
28 148 (12.07%) NoSQL
29 138 (11.26%) Amazon Redshift
30 130 (10.60%) Kubernetes

Data Engineer Skill Set
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 535 (43.64%) Apache Spark
2 168 (13.70%) Apache
3 108 (8.81%) Apache Airflow
4 44 (3.59%) Elasticsearch
5 24 (1.96%) Jupyter
6 20 (1.63%) SAS
7 13 (1.06%) Luigi
8 11 (0.90%) Confluence
8 11 (0.90%) SharePoint
9 10 (0.82%) Apache Solr
9 10 (0.82%) Pachyderm
10 7 (0.57%) Apache Pig
11 5 (0.41%) BizTalk Server
12 2 (0.16%) Mule
13 1 (0.082%) ArcGIS
13 1 (0.082%) IIS
13 1 (0.082%) JBoss
13 1 (0.082%) Oracle Coherence
13 1 (0.082%) Oracle Workflow
Applications
1 40 (3.26%) Microsoft Excel
2 2 (0.16%) Microsoft Office
2 2 (0.16%) Microsoft PowerPoint
3 1 (0.082%) InDesign
3 1 (0.082%) MS Visio
3 1 (0.082%) Spreadsheet
Business Applications
1 7 (0.57%) SAP Oil and Gas
2 3 (0.24%) FINCAD Analytics
3 2 (0.16%) Dynamics CRM
4 1 (0.082%) assyst
4 1 (0.082%) Cognos Business Intelligence
4 1 (0.082%) Distributed Ledger
4 1 (0.082%) Dynamics AX
4 1 (0.082%) Markit EDM
4 1 (0.082%) PolarLake
4 1 (0.082%) SAP ERP
Cloud Services
1 470 (38.34%) AWS
2 300 (24.47%) Azure
3 257 (20.96%) GCP
4 118 (9.62%) Amazon S3
5 104 (8.48%) Amazon EMR
6 89 (7.26%) Serverless
7 79 (6.44%) AWS Lambda
7 79 (6.44%) Power Platform
8 75 (6.12%) Azure Data Factory
9 55 (4.49%) Amazon Kinesis
10 45 (3.67%) Amazon SQS
11 35 (2.85%) Amazon EC2
12 34 (2.77%) GitHub
13 29 (2.37%) Cloud Computing
14 27 (2.20%) AWS CloudFormation
15 17 (1.39%) Amazon SageMaker
16 15 (1.22%) Google Analytics
17 12 (0.98%) Amazon ECS
18 11 (0.90%) AWS Fargate
19 10 (0.82%) Amazon CloudWatch
Communications & Networking
1 17 (1.39%) Kerberos
2 12 (0.98%) SSL
3 11 (0.90%) Internet
4 10 (0.82%) DNS
5 7 (0.57%) Firewall
6 5 (0.41%) Multicast
7 4 (0.33%) F5 BIG-IP GTM
7 4 (0.33%) F5 BIG-IP LTM
7 4 (0.33%) LAN
7 4 (0.33%) WAN
8 2 (0.16%) Cisco Nexus
8 2 (0.16%) DHCP
8 2 (0.16%) HTTP
8 2 (0.16%) Network Security
8 2 (0.16%) VPN
9 1 (0.082%) Cisco ASA
9 1 (0.082%) Cisco IOS
9 1 (0.082%) DMZ
9 1 (0.082%) TACACS
9 1 (0.082%) TCP/IP
Database & Business Intelligence
1 560 (45.68%) Big Data
2 420 (34.26%) Hadoop
3 260 (21.21%) Data Warehouse
4 237 (19.33%) Apache Hive
5 167 (13.62%) Data Lake
6 148 (12.07%) NoSQL
7 138 (11.26%) Amazon Redshift
8 105 (8.56%) SQL Server
9 101 (8.24%) PostgreSQL
10 91 (7.42%) BigQuery
11 82 (6.69%) Relational Database
12 79 (6.44%) Power BI
13 71 (5.79%) Impala
14 63 (5.14%) MongoDB
15 62 (5.06%) Amazon Athena
15 62 (5.06%) Apache Cassandra
16 59 (4.81%) HBase
17 54 (4.40%) SQL Server Integration Services
18 49 (4.00%) MySQL
19 41 (3.34%) Azure SQL Database
Development Applications
1 128 (10.44%) Jenkins
2 123 (10.03%) Git (software)
3 39 (3.18%) Artifactory
4 38 (3.10%) JIRA
5 24 (1.96%) GitLab
6 20 (1.63%) Sonatype Nexus
7 19 (1.55%) Visual Studio Team System
8 15 (1.22%) CircleCI
8 15 (1.22%) Maven
9 12 (0.98%) Atlassian Bamboo
10 9 (0.73%) IntelliJ
10 9 (0.73%) ProC
10 9 (0.73%) SonarQube
11 8 (0.65%) Visual Studio
12 7 (0.57%) Bitbucket
12 7 (0.57%) Cucumber
12 7 (0.57%) Selenium
13 6 (0.49%) TeamCity
13 6 (0.49%) tSQLt
14 5 (0.41%) Octopus Deploy
General
1 211 (17.21%) Finance
2 81 (6.61%) Banking
3 42 (3.43%) Retail
4 41 (3.34%) Legal
5 31 (2.53%) Front Office
6 25 (2.04%) Public Sector
7 23 (1.88%) Investment Banking
8 13 (1.06%) Marketing
8 13 (1.06%) Telecoms
9 12 (0.98%) Games
10 8 (0.65%) Publishing
11 6 (0.49%) Manufacturing
12 5 (0.41%) Pharmaceutical
13 4 (0.33%) Advertising
13 4 (0.33%) Electronics
14 3 (0.24%) Financial Institution
14 3 (0.24%) Retail Banking
15 2 (0.16%) Corporate Banking
15 2 (0.16%) International Banking
15 2 (0.16%) Local Government
Libraries, Frameworks & Software Standards
1 280 (22.84%) Kafka
2 124 (10.11%) PySpark
3 61 (4.98%) Pandas
4 46 (3.75%) Spark Streaming
5 43 (3.51%) Apache Beam
6 38 (3.10%) Web Services
7 35 (2.85%) Apache NiFi
8 31 (2.53%) REST
9 30 (2.45%) scikit-learn
10 29 (2.37%) NumPy
11 24 (1.96%) RESTful
12 23 (1.88%) JSON
13 21 (1.71%) Flask
14 19 (1.55%) MLlib
15 16 (1.31%) Apache Avro
15 16 (1.31%) J2EE
15 16 (1.31%) TensorFlow
15 16 (1.31%) Velocity
16 15 (1.22%) Play Framework
16 15 (1.22%) XML
Miscellaneous
1 89 (7.26%) Analytical Skills
2 81 (6.61%) Data Structures
3 76 (6.20%) Algorithms
4 60 (4.89%) Greenfield Project
5 18 (1.47%) Public Cloud
6 16 (1.31%) Self-Motivation
7 15 (1.22%) User Experience
8 13 (1.06%) Reference Data
9 9 (0.73%) Management Information System
10 8 (0.65%) PKI
11 7 (0.57%) CMDB
11 7 (0.57%) Equities
11 7 (0.57%) Life Science
12 6 (0.49%) Cloud Native
12 6 (0.49%) Data Centre
12 6 (0.49%) Linux Command Line
12 6 (0.49%) Robotics
13 5 (0.41%) Derivative
13 5 (0.41%) Fixed Income
13 5 (0.41%) Market Risk
Operating Systems
1 117 (9.54%) Linux
2 51 (4.16%) Unix
3 22 (1.79%) Windows
4 10 (0.82%) Red Hat Enterprise Linux
5 9 (0.73%) CentOS
6 6 (0.49%) VMS
7 3 (0.24%) Windows Server
Processes & Methodologies
1 366 (29.85%) ETL
2 334 (27.24%) Agile Software Development
3 328 (26.75%) Analytics
4 299 (24.39%) Data Engineering
5 298 (24.31%) Data Science
6 268 (21.86%) Data Modelling
7 207 (16.88%) DevOps
8 183 (14.93%) Machine Learning
9 165 (13.46%) Business Intelligence
10 110 (8.97%) Migration
11 103 (8.40%) Visualisation
12 101 (8.24%) Scrum
13 99 (8.08%) Data Analysis
14 85 (6.93%) Artificial Intelligence
15 84 (6.85%) Continuous Integration
16 78 (6.36%) Computer Science
17 75 (6.12%) Data Analytics
18 68 (5.55%) Software Engineering
19 67 (5.46%) Data Integration
20 63 (5.14%) Data Ingestion
Programming Languages
1 774 (63.13%) Python
2 570 (46.49%) SQL
3 293 (23.90%) Scala
4 261 (21.29%) Java
5 66 (5.38%) R
6 46 (3.75%) C#
7 41 (3.34%) Bash Shell
8 33 (2.69%) JavaScript
9 25 (2.04%) PL/SQL
10 21 (1.71%) Go
11 20 (1.63%) Perl
11 20 (1.63%) T-SQL
12 17 (1.39%) C++
13 13 (1.06%) C
14 11 (0.90%) PowerShell
15 10 (0.82%) Julia
15 10 (0.82%) Shell Script
16 9 (0.73%) Groovy
17 8 (0.65%) Clojure
17 8 (0.65%) SAS Macro
Qualifications
1 91 (7.42%) Degree
2 75 (6.12%) Security Cleared
3 61 (4.98%) SC Cleared
4 38 (3.10%) Computer Science Degree
5 32 (2.61%) Master's Degree
6 18 (1.47%) PMI Certification
6 18 (1.47%) PMP
7 15 (1.22%) AWS Certification
8 9 (0.73%) DV Cleared
8 9 (0.73%) PhD
9 8 (0.65%) Cisco Certification
10 7 (0.57%) DBS Check
11 6 (0.49%) CCNP
12 4 (0.33%) BPSS Clearance
12 4 (0.33%) CCIE
12 4 (0.33%) CCNA
13 2 (0.16%) CCENT
13 2 (0.16%) ITIL Certification
13 2 (0.16%) Microsoft Certification
13 2 (0.16%) Postgraduate
Quality Assurance & Compliance
1 54 (4.40%) Data Quality
2 10 (0.82%) GDPR
2 10 (0.82%) QA
3 5 (0.41%) GxP
4 4 (0.33%) Sarbanes-Oxley
5 3 (0.24%) Disclosure Scotland
6 1 (0.082%) PCI DSS
System Software
1 196 (15.99%) Docker
2 45 (3.67%) HDFS
3 32 (2.61%) Apache Flume
4 13 (1.06%) Active Directory
5 6 (0.49%) Apache ZooKeeper
6 4 (0.33%) Virtual Servers
7 2 (0.16%) VirtualBox
Systems Management
1 130 (10.60%) Kubernetes
2 98 (7.99%) Terraform
3 54 (4.40%) Ansible
4 51 (4.16%) Hadoop YARN
5 36 (2.94%) Oozie
6 17 (1.39%) Puppet
7 14 (1.14%) Kubeflow
8 9 (0.73%) Kibana
9 7 (0.57%) Opscode Chef
10 6 (0.49%) Istio
10 6 (0.49%) linkerd
11 5 (0.41%) Prometheus
12 4 (0.33%) Avamar
12 4 (0.33%) Consul
12 4 (0.33%) FortiGate
12 4 (0.33%) logstash
12 4 (0.33%) Mesos
13 2 (0.16%) Ambari
13 2 (0.16%) Docker Swarm
13 2 (0.16%) Packer
Vendors
1 209 (17.05%) Google
2 109 (8.89%) Cloudera
3 107 (8.73%) Microsoft
4 106 (8.65%) Databricks
5 73 (5.95%) Oracle
6 62 (5.06%) Informatica
6 62 (5.06%) Tableau
7 57 (4.65%) Talend
8 45 (3.67%) Hortonworks
9 37 (3.02%) SAP
10 27 (2.20%) Alteryx
11 26 (2.12%) Teradata
12 16 (1.31%) Red Hat
13 15 (1.22%) Splunk
14 12 (0.98%) MapR
14 12 (0.98%) StreamSets
15 11 (0.90%) Salesforce.com
16 10 (0.82%) Cisco
16 10 (0.82%) Cognos
16 10 (0.82%) Palantir