Period
to 27 June 2019

The following table provides summary statistics for contract job vacancies with a requirement for Big Data skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Big Data over the 6 months to 27 June 2019 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Big Data
UK
6 months to
27 Jun 2019
Same period 2018 Same period 2017
Rank 69 79 94
Rank change year-on-year +10 +15 +73
Contract jobs citing Big Data 2,913 2,986 2,366
As % of all contract IT jobs advertised in the UK 3.20% 2.95% 2.57%
As % of the Database & Business Intelligence category 14.40% 13.10% 11.38%
Number of daily rates quoted 2,065 2,078 1,650
UK median daily rate £550 £550 £525
Median daily rate % change year-on-year +0.00% +4.76% -
10th Percentile £413 £396 £358
90th Percentile £713 £688 £675
UK excluding London median daily rate £525 £475 £475
% change year-on-year +10.53% - +1.88%
Number of hourly rates quoted 16 13 7
UK median hourly rate £46.85 £44.50 £47.00
Median hourly rate % change year-on-year +5.28% -5.32% +74.56%
10th Percentile £25.13 £18.50 £22.25
90th Percentile £57.50 £72.00 £50.00
UK excluding London median hourly rate £36.00 £44.50 £47.00
% change year-on-year -19.10% -5.32% +10.20%

Big Data is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Contract vacancies with a requirement for database or business intelligence skills 20,223 22,797 20,782
As % of all contract IT jobs advertised in the UK 22.19% 22.55% 22.57%
Number of daily rates quoted 13,856 15,872 13,826
UK median daily rate £475 £463 £445
Median daily rate % change year-on-year +2.70% +3.93% +4.71%
10th Percentile £325 £313 £300
90th Percentile £650 £634 £613
UK excluding London median daily rate £430 £413 £400
% change year-on-year +4.24% +3.13% +6.67%
Number of hourly rates quoted 246 232 272
UK median hourly rate £35.00 £33.50 £30.50
Median hourly rate % change year-on-year +4.48% +9.84% +7.02%
10th Percentile £14.27 £14.00 £10.50
90th Percentile £59.56 £54.88 £52.17
UK excluding London median hourly rate £35.00 £33.50 £31.00
% change year-on-year +4.48% +8.06% +3.33%

Big Data
Job Vacancy Trend

Job postings citing Big Data as a percentage of all IT jobs advertised.

Job vacancy trend for Big Data in the UK

Big Data
Contractor Daily Rate Trend

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Big Data.

Contractor daily rate trend for Big Data in the UK

Big Data
Contractor Daily Rate Histogram

The daily rate distribution of IT jobs citing Big Data over the 6 months to 27 June 2019.

Contractor daily rate histogram for Big Data in the UK

Big Data
Contractor Hourly Rate Trend

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing Big Data.

Contractor hourly rate trend for Big Data in the UK

Big Data
Contractor Hourly Rate Histogram

The hourly rate distribution of IT jobs citing Big Data over the 6 months to 27 June 2019.

Contractor hourly rate histogram for Big Data in the UK

Big Data
Top 17 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Big Data within the UK over the 6 months to 27 June 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +6 2,719 £550 - 322
London +8 1,806 £575 +4.55% 181
UK excluding London +9 1,060 £525 +10.53% 176
South East -32 339 £500 - 39
North of England +31 282 £550 +10.00% 40
Scotland +86 152 £475 +11.76% 21
Yorkshire +8 137 £475 - 8
North West +28 127 £550 - 26
Midlands +26 94 £525 +11.11% 15
South West -4 92 £550 +16.09% 25
East of England -4 79 £550 +15.79% 22
West Midlands +33 57 £612 +48.24% 10
East Midlands -3 37 £481 +1.32% 5
North East +25 18 £575 -9.80% 6
Northern Ireland +3 13 £400 -7.30% 8
Wales +1 8 £525 +7.69% 6
Channel Islands - 1 - -

For the 6 months to 27 June 2019, IT contractor jobs citing Big Data also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for Big Data.

1 1,178 (40.44%) Hadoop
2 1,153 (39.58%) Apache Spark
3 1,088 (37.35%) Python
4 1,002 (34.40%) Agile Software Development
5 962 (33.02%) Amazon AWS
6 926 (31.79%) SQL
7 853 (29.28%) Analytics
8 827 (28.39%) Java
9 710 (24.37%) Finance
10 694 (23.82%) Kafka
11 592 (20.32%) Apache Hive
12 589 (20.22%) DevOps
13 574 (19.70%) Microsoft Azure
14 523 (17.95%) Data Warehouse
15 517 (17.75%) Data Science
16 478 (16.41%) Scala
17 436 (14.97%) Machine Learning
18 430 (14.76%) Jenkins
18 430 (14.76%) Google Cloud Platform
19 416 (14.28%) Data Modelling
20 415 (14.25%) Docker
21 414 (14.21%) ETL
22 410 (14.07%) Linux
23 405 (13.90%) Business Intelligence
24 381 (13.08%) Kubernetes
25 364 (12.50%) Google
26 358 (12.29%) NoSQL
27 335 (11.50%) Scrum
28 331 (11.36%) Git (software)
29 297 (10.20%) Continuous Integration

Big Data
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 1,153 (39.58%) Apache Spark
2 289 (9.92%) Apache
3 146 (5.01%) Elasticsearch
4 138 (4.74%) Apache Airflow
5 75 (2.57%) SAS
6 48 (1.65%) Apache Pig
7 47 (1.61%) Confluence
8 35 (1.20%) Apache Solr
9 31 (1.06%) Luigi
10 30 (1.03%) Jupyter
11 26 (0.89%) nginx
12 20 (0.69%) Cloud Foundry
13 18 (0.62%) IIS
14 17 (0.58%) Tomcat
15 14 (0.48%) Skype for Business
16 12 (0.41%) WebSphere
17 11 (0.38%) Exchange Server 2013
18 10 (0.34%) Oracle Coherence
19 9 (0.31%) Apache Qpid
19 9 (0.31%) OpenStack
Applications
1 59 (2.03%) Microsoft Excel
2 27 (0.93%) Microsoft Office
3 20 (0.69%) Microsoft PowerPoint
4 17 (0.58%) SPSS
5 16 (0.55%) Stata
6 10 (0.34%) MS Visio
7 3 (0.10%) Chatbot
7 3 (0.10%) InVision
7 3 (0.10%) Spreadsheet
8 2 (0.069%) Microsoft Project
8 2 (0.069%) Sketch
9 1 (0.034%) LaTeX
Business Applications
1 14 (0.48%) SAP Oil and Gas
2 11 (0.38%) Dynamics CRM
3 9 (0.31%) SAP S/4HANA
4 8 (0.27%) Distributed Ledger
5 7 (0.24%) Oracle BPM
6 6 (0.21%) SAP ERP
7 4 (0.14%) Dynamics AX
7 4 (0.14%) Dynamics NAV
8 2 (0.069%) BPCS
8 2 (0.069%) SAP R/3
9 1 (0.034%) Actimize
9 1 (0.034%) Markit EDM
9 1 (0.034%) NetSuite
9 1 (0.034%) Oracle Procure-to-Pay
9 1 (0.034%) Oracle Retail
9 1 (0.034%) PeopleSoft Financials
9 1 (0.034%) Salesforce.com CRM
9 1 (0.034%) SAP APO
Cloud Services
1 962 (33.02%) Amazon AWS
2 574 (19.70%) Microsoft Azure
3 430 (14.76%) Google Cloud Platform
4 166 (5.70%) GitHub
5 155 (5.32%) Amazon EMR
6 154 (5.29%) Amazon S3
7 153 (5.25%) Power Platform
8 140 (4.81%) Serverless
9 93 (3.19%) Amazon Kinesis
10 84 (2.88%) Amazon EC2
11 80 (2.75%) AWS Lambda
11 80 (2.75%) Azure Data Factory
12 73 (2.51%) PaaS
13 65 (2.23%) AWS CloudFormation
14 57 (1.96%) Amazon CloudWatch
15 44 (1.51%) Amazon SQS
16 43 (1.48%) Cloud Computing
17 31 (1.06%) IaaS
17 31 (1.06%) Office 365
18 30 (1.03%) Virtual Private Cloud
Communications & Networking
1 46 (1.58%) Firewall
1 46 (1.58%) Kerberos
2 23 (0.79%) DNS
3 20 (0.69%) HTTP
4 19 (0.65%) TCP/IP
5 17 (0.58%) SSL
6 16 (0.55%) Internet
6 16 (0.55%) SAN
7 14 (0.48%) MQTT
8 13 (0.45%) AMQP
8 13 (0.45%) Cisco Nexus
8 13 (0.45%) Fibre Channel
8 13 (0.45%) iSCSI
9 11 (0.38%) 3G
9 11 (0.38%) 4G
9 11 (0.38%) Cisco ASA
9 11 (0.38%) Cisco Firepower
10 9 (0.31%) 2G
11 7 (0.24%) 5G
11 7 (0.24%) VPN
Database & Business Intelligence
1 1,178 (40.44%) Hadoop
2 592 (20.32%) Apache Hive
3 523 (17.95%) Data Warehouse
4 358 (12.29%) NoSQL
5 267 (9.17%) HBase
6 251 (8.62%) Data Lake
7 204 (7.00%) SQL Server
8 179 (6.14%) Impala
8 179 (6.14%) Relational Database
9 178 (6.11%) Amazon Redshift
10 153 (5.25%) Power BI
11 151 (5.18%) PostgreSQL
12 146 (5.01%) Apache Cassandra
12 146 (5.01%) MongoDB
13 109 (3.74%) MySQL
14 108 (3.71%) BigQuery
15 89 (3.06%) SQL Server Integration Services
16 84 (2.88%) RDBMS
17 77 (2.64%) Star Schema
18 66 (2.27%) OLAP
Development Applications
1 430 (14.76%) Jenkins
2 331 (11.36%) Git (software)
3 165 (5.66%) JIRA
4 92 (3.16%) Maven
5 70 (2.40%) Gradle
6 63 (2.16%) GitLab
7 57 (1.96%) Sonatype Nexus
8 51 (1.75%) TeamCity
9 50 (1.72%) Visual Studio Team System
10 46 (1.58%) Selenium
11 44 (1.51%) Atlassian Bamboo
11 44 (1.51%) Subversion
12 41 (1.41%) Visual Studio
13 38 (1.30%) JUnit
14 32 (1.10%) Cucumber
15 31 (1.06%) CircleCI
16 27 (0.93%) Eclipse
17 26 (0.89%) IntelliJ
18 25 (0.86%) Ant
19 21 (0.72%) Jasmine
General
1 710 (24.37%) Finance
2 271 (9.30%) Banking
3 127 (4.36%) Retail
4 89 (3.06%) Telecoms
5 79 (2.71%) Investment Banking
6 77 (2.64%) Marketing
7 50 (1.72%) Financial Institution
7 50 (1.72%) Front Office
8 45 (1.54%) Manufacturing
9 41 (1.41%) Legal
10 33 (1.13%) Advertising
11 26 (0.89%) Pharmaceutical
12 25 (0.86%) Games
13 14 (0.48%) Electronics
14 13 (0.45%) Publishing
15 10 (0.34%) Law
15 10 (0.34%) Retail Banking
16 8 (0.27%) Automotive
17 4 (0.14%) Back Office
17 4 (0.14%) General Ledger
Job Titles
1 705 (24.20%) Developer
2 698 (23.96%) Data Engineer
3 419 (14.38%) Architect
4 398 (13.66%) Big Data Engineer
5 218 (7.48%) Java Developer
6 213 (7.31%) Data Architect
6 213 (7.31%) DevOps Engineer
7 191 (6.56%) Big Data Developer
8 185 (6.35%) Python Engineer
9 130 (4.46%) Cloud Engineer
10 124 (4.26%) Hadoop Engineer
10 124 (4.26%) Python Developer
11 116 (3.98%) Analyst
12 109 (3.74%) Data Scientist
13 106 (3.64%) AWS Engineer
14 94 (3.23%) Big Data Architect
15 85 (2.92%) Senior Developer
16 84 (2.88%) Consultant
17 80 (2.75%) Software Engineer
18 75 (2.57%) Java Engineer
Libraries, Frameworks & Software Standards
1 694 (23.82%) Kafka
2 151 (5.18%) Web Services
3 131 (4.50%) PySpark
3 131 (4.50%) Spring
4 115 (3.95%) .NET
5 114 (3.91%) REST
5 114 (3.91%) RESTful
6 90 (3.09%) JSON
7 76 (2.61%) HTML
8 74 (2.54%) XML
9 72 (2.47%) Apache NiFi
10 70 (2.40%) AngularJS
10 70 (2.40%) Node.js
11 69 (2.37%) React
11 69 (2.37%) Spark Streaming
12 68 (2.33%) Natural Language Processing
13 67 (2.30%) Spring Boot
14 66 (2.27%) CSS
15 61 (2.09%) SOAP
16 58 (1.99%) TensorFlow
Miscellaneous
1 297 (10.20%) Analytical Skills
2 155 (5.32%) Greenfield Project
3 131 (4.50%) Algorithms
4 99 (3.40%) Data Structures
5 66 (2.27%) Self-Motivation
6 57 (1.96%) Public Cloud
7 55 (1.89%) Management Information System
7 55 (1.89%) User Experience
8 44 (1.51%) Cloud Native
9 38 (1.30%) Virtual Team
10 32 (1.10%) Data Protection Act
10 32 (1.10%) Enterprise Software
11 30 (1.03%) Clustering
12 29 (1.00%) Hedge funds
12 29 (1.00%) Linux Command Line
13 25 (0.86%) Data Centre
14 24 (0.82%) Mobile App
15 23 (0.79%) Market Risk
15 23 (0.79%) Robotics
16 19 (0.65%) Hybrid Cloud
Operating Systems
1 410 (14.07%) Linux
2 173 (5.94%) Unix
3 107 (3.67%) Windows
4 42 (1.44%) Red Hat Enterprise Linux
5 22 (0.76%) Windows Server
6 16 (0.55%) Ubuntu
7 13 (0.45%) Android
8 12 (0.41%) Apple iOS
9 11 (0.38%) CentOS
9 11 (0.38%) Solaris
10 3 (0.10%) Debian
10 3 (0.10%) Windows 10
10 3 (0.10%) Windows Server 2003
10 3 (0.10%) Windows Vista
11 2 (0.069%) AIX
11 2 (0.069%) SUSE Linux Enterprise Server
12 1 (0.034%) Embedded Linux
12 1 (0.034%) Mac OS
12 1 (0.034%) Mac OS X
12 1 (0.034%) VMS
Processes & Methodologies
1 1,002 (34.40%) Agile Software Development
2 853 (29.28%) Analytics
3 589 (20.22%) DevOps
4 517 (17.75%) Data Science
5 436 (14.97%) Machine Learning
6 416 (14.28%) Data Modelling
7 414 (14.21%) ETL
8 405 (13.90%) Business Intelligence
9 335 (11.50%) Scrum
10 297 (10.20%) Continuous Integration
10 297 (10.20%) Migration
11 282 (9.68%) Software Engineering
12 268 (9.20%) Data Engineering
13 244 (8.38%) TDD
14 232 (7.96%) Microservices
15 228 (7.83%) Problem-Solving
16 222 (7.62%) Data Analytics
17 205 (7.04%) Visualisation
18 195 (6.69%) Mentoring
19 191 (6.56%) Artificial Intelligence
Programming Languages
1 1,088 (37.35%) Python
2 926 (31.79%) SQL
3 827 (28.39%) Java
4 478 (16.41%) Scala
5 174 (5.97%) C#
6 160 (5.49%) JavaScript
7 158 (5.42%) R
8 126 (4.33%) Bash Shell
9 105 (3.60%) C++
10 103 (3.54%) Java 8
11 99 (3.40%) Shell Script
12 86 (2.95%) Ruby
13 66 (2.27%) PowerShell
13 66 (2.27%) T-SQL
14 49 (1.68%) C
15 41 (1.41%) Go
16 38 (1.30%) PL/SQL
17 33 (1.13%) MATLAB
18 31 (1.06%) Groovy
19 29 (1.00%) Perl
Qualifications
1 282 (9.68%) Security Cleared
2 222 (7.62%) SC Cleared
3 214 (7.35%) Degree
4 53 (1.82%) DV Cleared
5 49 (1.68%) PhD
6 44 (1.51%) Computer Science Degree
7 34 (1.17%) Master's Degree
8 27 (0.93%) CISSP
9 23 (0.79%) CISM
10 21 (0.72%) Numerate Degree
10 21 (0.72%) PMI Certification
10 21 (0.72%) PMP
11 18 (0.62%) CCA
11 18 (0.62%) TOGAF Certification
12 16 (0.55%) AWS Certification
13 13 (0.45%) Microsoft Certification
14 12 (0.41%) ITIL Certification
15 11 (0.38%) BPSS Clearance
15 11 (0.38%) CISA
15 11 (0.38%) MCSE
Quality Assurance & Compliance
1 158 (5.42%) Data Quality
2 41 (1.41%) QA
3 32 (1.10%) GDPR
4 13 (0.45%) Sarbanes-Oxley
5 10 (0.34%) PCI DSS
6 9 (0.31%) ISO/IEC 27001
6 9 (0.31%) PMO
7 6 (0.21%) IFRS
7 6 (0.21%) SLA
8 3 (0.10%) GxP
8 3 (0.10%) MiFID
8 3 (0.10%) NIST
9 2 (0.069%) IFRS 9
9 2 (0.069%) PSD2
10 1 (0.034%) Actionable Recommendations
10 1 (0.034%) Disclosure Scotland
10 1 (0.034%) EMIR
10 1 (0.034%) NCSC
System Software
1 415 (14.25%) Docker
2 196 (6.73%) HDFS
3 102 (3.50%) Apache Flume
4 44 (1.51%) Active Directory
5 33 (1.13%) Virtual Machines
6 31 (1.06%) Apache ZooKeeper
7 23 (0.79%) VMware Infrastructure
8 20 (0.69%) Hyper-V
9 17 (0.58%) Oracle RAC
10 16 (0.55%) XenApp
11 11 (0.38%) vCloud
12 5 (0.17%) vSphere
13 4 (0.14%) VMware ESXi
14 2 (0.069%) NFS
14 2 (0.069%) Samba
14 2 (0.069%) VirtualBox
15 1 (0.034%) KVM
15 1 (0.034%) Virtual Servers
Systems Management
1 381 (13.08%) Kubernetes
2 271 (9.30%) Ansible
3 248 (8.51%) Terraform
4 140 (4.81%) Puppet
5 98 (3.36%) Oozie
6 93 (3.19%) Opscode Chef
7 91 (3.12%) Hadoop YARN
8 27 (0.93%) Salt
9 19 (0.65%) Kibana
9 19 (0.65%) Microsoft Clustering
10 18 (0.62%) logstash
11 16 (0.55%) SCOM
12 14 (0.48%) Docker Swarm
13 13 (0.45%) Microsoft Data Protection Manager
13 13 (0.45%) Packer
13 13 (0.45%) SCOrch
13 13 (0.45%) Symantec Enterprise Vault
14 11 (0.38%) QRadar
14 11 (0.38%) ZABBIX
15 9 (0.31%) Kubeflow
Vendors
1 364 (12.50%) Google
2 236 (8.10%) Cloudera
3 228 (7.83%) Microsoft
4 209 (7.17%) Oracle
5 143 (4.91%) Hortonworks
6 128 (4.39%) Tableau
7 104 (3.57%) Databricks
8 80 (2.75%) Talend
9 79 (2.71%) Informatica
10 61 (2.09%) SAP
11 51 (1.75%) Greenplum
12 37 (1.27%) Red Hat
13 36 (1.24%) MapR
14 35 (1.20%) Teradata
15 30 (1.03%) Splunk
16 29 (1.00%) MicroStrategy
17 26 (0.89%) VMware
18 24 (0.82%) Facebook
19 21 (0.72%) Alteryx
20 20 (0.69%) Cisco