Job Vacancy Trend Daily Rate Trend Daily Rate Histogram Hourly Rate Trend Hourly Rate Histogram Job Locations Skill Sets
Period
to 16 October 2021

The following table provides summary statistics for contract job vacancies with a requirement for Big Data skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Big Data over the 6 months to 16 October 2021 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Big Data
UK
6 months to
16 Oct 2021		 Same period 2020 Same period 2019
Rank 118 99 87
Rank change year-on-year -19 -12 +11
Contract jobs citing Big Data 1,843 743 2,367
As % of all contract jobs advertised in the UK 2.22% 2.41% 2.78%
As % of the Database & Business Intelligence category 10.34% 10.53% 12.63%
Number of daily rates quoted 1,275 514 1,695
10th Percentile £425 £367 £413
25th Percentile £500 £450 £481
Median daily rate (50th Percentile) £600 £539 £550
Median % change year-on-year +11.37% -2.05% +1.85%
75th Percentile £688 £606 £638
90th Percentile £763 £688 £725
UK excluding London median daily rate £550 £525 £525
% change year-on-year +4.76% - +5.00%
Number of hourly rates quoted 13 11 12
10th Percentile £27.15 £9.75 £36.18
25th Percentile £36.25 £16.63 £46.14
Median hourly rate £44.25 £20.00 £49.93
Median % change year-on-year +121.25% -59.94% +12.20%
75th Percentile £53.38 £25.02 £52.66
90th Percentile £63.50 £61.25 £53.75
UK excluding London median hourly rate £26.12 £20.00 £52.50
% change year-on-year +30.58% -61.90% +31.25%

Big Data is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for database or business intelligence skills.

All Database and Business Intelligence Skills
UK
Contract vacancies with a requirement for database or business intelligence skills 17,820 7,056 18,741
As % of all contract IT jobs advertised in the UK 21.43% 22.85% 22.00%
Number of daily rates quoted 12,481 4,824 12,951
10th Percentile £350 £313 £320
25th Percentile £425 £400 £400
Median daily rate (50th Percentile) £525 £498 £475
Median % change year-on-year +5.47% +4.80% -
75th Percentile £625 £588 £575
90th Percentile £713 £675 £656
UK excluding London median daily rate £463 £450 £438
% change year-on-year +2.78% +2.86% +2.94%
Number of hourly rates quoted 220 118 248
10th Percentile £15.66 £12.63 £11.22
25th Percentile £23.19 £17.47 £17.00
Median hourly rate £35.00 £25.00 £30.00
Median % change year-on-year +40.00% -16.67% +1.69%
75th Percentile £58.05 £42.00 £48.85
90th Percentile £65.50 £55.00 £60.00
UK excluding London median hourly rate £32.50 £23.22 £27.94
% change year-on-year +40.00% -16.91% +7.46%

Big Data
Job Vacancy Trend

Job postings citing Big Data as a proportion of all IT jobs advertised.

Job vacancy trend for Big Data in the UK

Big Data
Contractor Daily Rate Trend

3-month moving average daily rate quoted in jobs citing Big Data.

Daily rate trend for Big Data in the UK

Big Data
Daily Rate Histogram

Daily rate distribution for jobs citing Big Data over the 6 months to 16 October 2021.

Daily rate histogram for Big Data in the UK

Big Data
Contractor Hourly Rate Trend

3-month moving average hourly rates quoted in jobs citing Big Data.

Hourly rate trend for Big Data in the UK

Big Data
Hourly Rate Histogram

Hourly rate distribution of jobs citing Big Data over the 6 months to 16 October 2021.

Hourly rate histogram for Big Data in the UK

Big Data
Top 18 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Big Data within the UK over the 6 months to 16 October 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year		 Matching
Contract
IT Job Ads		 Median
Daily Rate
Past 6 Months		 Median Daily Rate
% Change
on Same Period
Last Year		 Live
Job
Vacancies
England -9 1,538 £600 +9.09% 422
London +6 1,065 £600 +9.09% 274
Work from Home -34 498 £600 +14.29% 78
UK excluding London -48 442 £550 +4.76% 165
South East -25 141 £550 +8.91% 44
North of England -77 84 £600 +9.09% 46
Midlands -43 62 £565 +47.52% 25
Scotland -39 53 £475 +35.52% 18
South West -6 51 £550 +18.92% 13
West Midlands -26 50 £550 +23.94% 14
Yorkshire -13 40 £650 - 10
North West -90 32 £600 +9.09% 30
East of England -48 26 £551 +10.25% 12
Wales -28 15 £450 -10.00% 6
North East - 13 £565 - 6
East Midlands -41 12 £775 +104.22% 11
Northern Ireland +1 11 £475 +21.79% 1
Isle of Man - 1 £750 -

For the 6 months to 16 October 2021, IT contractor jobs citing Big Data also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads with a requirement for Big Data.

1 661 (35.87%) Agile Software Development
2 637 (34.56%) Python
3 624 (33.86%) SQL
4 613 (33.26%) AWS
5 606 (32.88%) Apache Spark
6 559 (30.33%) Azure
7 536 (29.08%) Hadoop
8 532 (28.87%) Analytics
9 523 (28.38%) DevOps
10 469 (25.45%) Finance
11 429 (23.28%) Social Skills
12 425 (23.06%) Java
13 398 (21.60%) Data Warehouse
14 391 (21.22%) Kafka
15 361 (19.59%) Data Modelling
16 347 (18.83%) ETL
17 307 (16.66%) Scala
18 299 (16.22%) Business Intelligence
19 295 (16.01%) Apache Hive
20 291 (15.79%) GCP
21 272 (14.76%) Kubernetes
22 255 (13.84%) Linux
23 254 (13.78%) Docker
24 252 (13.67%) Terraform
25 248 (13.46%) Data Lake
26 245 (13.29%) Data Engineering
27 239 (12.97%) NoSQL
28 237 (12.86%) Banking
29 234 (12.70%) Analytical Skills
30 224 (12.15%) Scrum

Big Data
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 606 (32.88%) Apache Spark
2 147 (7.98%) Apache
3 82 (4.45%) Apache Airflow
4 74 (4.02%) Confluence
5 48 (2.60%) Elasticsearch
6 33 (1.79%) Luigi
7 32 (1.74%) SAS
8 15 (0.81%) SharePoint
9 14 (0.76%) Tomcat
10 12 (0.65%) Apache Pig
11 9 (0.49%) Oracle Coherence
12 7 (0.38%) nginx
13 6 (0.33%) Jupyter
13 6 (0.33%) OpenStack
14 5 (0.27%) Oracle SOA Suite
15 4 (0.22%) Apache Solr
15 4 (0.22%) Oracle Workflow
16 3 (0.16%) Apache Pulsar
16 3 (0.16%) CMS
16 3 (0.16%) Moodle
Applications
1 26 (1.41%) Microsoft Excel
2 10 (0.54%) Spreadsheet
3 5 (0.27%) Microsoft Office
3 5 (0.27%) Microsoft PowerPoint
4 3 (0.16%) Microsoft Project
5 2 (0.11%) Chatbot
5 2 (0.11%) MS Visio
6 1 (0.054%) Figma
6 1 (0.054%) Sketch
6 1 (0.054%) SPSS
Business Applications
1 7 (0.38%) Dynamics CRM
2 6 (0.33%) Dynamics AX
3 4 (0.22%) SAP Oil and Gas
3 4 (0.22%) SAP S/4HANA
3 4 (0.22%) Sparx Enterprise Architect
4 3 (0.16%) Oracle Financials
4 3 (0.16%) Workday HCM
5 2 (0.11%) Oracle EBS
5 2 (0.11%) SAP R/3
6 1 (0.054%) Oracle BPM
6 1 (0.054%) Oracle Procure-to-Pay
6 1 (0.054%) Salesforce.com CRM
6 1 (0.054%) SAP ERP
6 1 (0.054%) SAP PLM
6 1 (0.054%) Sentinel
Cloud Services
1 613 (33.26%) AWS
2 559 (30.33%) Azure
3 291 (15.79%) GCP
4 193 (10.47%) Serverless
5 142 (7.70%) Power Platform
6 117 (6.35%) PaaS
7 98 (5.32%) IaaS
8 91 (4.94%) AWS Lambda
8 91 (4.94%) Azure Data Factory
9 84 (4.56%) Amazon S3
10 83 (4.50%) Snowflake
11 78 (4.23%) Amazon EMR
12 77 (4.18%) GitHub
13 58 (3.15%) Azure Synapse Analytics
14 57 (3.09%) Azure Functions
15 53 (2.88%) Azure Cosmos DB
16 39 (2.12%) Cloud Computing
17 37 (2.01%) Amazon Kinesis
18 32 (1.74%) OpenShift
19 27 (1.47%) Amazon EC2
Communications & Networking
1 22 (1.19%) Firewall
2 18 (0.98%) FTP
2 18 (0.98%) HTTP
3 9 (0.49%) SAN
4 7 (0.38%) Intrusion Detection
4 7 (0.38%) Kerberos
4 7 (0.38%) SSH
5 6 (0.33%) Internet
5 6 (0.33%) TCP/IP
6 4 (0.22%) Network Security
6 4 (0.22%) Remote Desktop
7 3 (0.16%) Intranet
7 3 (0.16%) Wireless
8 2 (0.11%) Broadband
8 2 (0.11%) LAN
8 2 (0.11%) Multicast
8 2 (0.11%) Skype
8 2 (0.11%) SSL
8 2 (0.11%) Unified Communications
8 2 (0.11%) WAN
Database & Business Intelligence
1 536 (29.08%) Hadoop
2 398 (21.60%) Data Warehouse
3 295 (16.01%) Apache Hive
4 248 (13.46%) Data Lake
5 239 (12.97%) NoSQL
6 211 (11.45%) Relational Database
7 153 (8.30%) PostgreSQL
8 132 (7.16%) Power BI
9 125 (6.78%) Metadata
10 121 (6.57%) HBase
11 119 (6.46%) MongoDB
12 117 (6.35%) Tableau
13 115 (6.24%) SQL Server
14 114 (6.19%) BigQuery
15 92 (4.99%) Amazon Redshift
16 82 (4.45%) Apache Cassandra
17 72 (3.91%) RDBMS
18 60 (3.26%) Impala
19 53 (2.88%) Amazon Athena
20 40 (2.17%) Spark SQL
Development Applications
1 202 (10.96%) Jenkins
2 190 (10.31%) Git (software)
3 150 (8.14%) JIRA
4 88 (4.77%) GitLab
5 56 (3.04%) Maven
6 43 (2.33%) PowerDesigner
7 34 (1.84%) TeamCity
8 25 (1.36%) Artifactory
8 25 (1.36%) IntelliJ
9 23 (1.25%) Bitbucket
10 22 (1.19%) Subversion
11 19 (1.03%) Hudson
11 19 (1.03%) Sonatype Nexus
12 17 (0.92%) Visual Studio Team System
13 16 (0.87%) ERwin
14 15 (0.81%) JUnit
15 14 (0.76%) kappa
16 13 (0.71%) Gradle
17 12 (0.65%) Cucumber
18 11 (0.60%) CircleCI
General
1 469 (25.45%) Finance
2 429 (23.28%) Social Skills
3 237 (12.86%) Banking
4 234 (12.70%) Analytical Skills
5 90 (4.88%) Presentation Skills
6 85 (4.61%) Investment Banking
7 72 (3.91%) Public Sector
8 46 (2.50%) Retail
9 38 (2.06%) Influencing Skills
10 33 (1.79%) Marketing
11 29 (1.57%) Documentation Skills
12 26 (1.41%) Telecoms
13 22 (1.19%) Games
14 21 (1.14%) Front Office
15 20 (1.09%) Financial Institution
15 20 (1.09%) Legal
16 15 (0.81%) Manufacturing
17 14 (0.76%) Publishing
18 9 (0.49%) Aerospace
18 9 (0.49%) Law
Job Titles
1 454 (24.63%) Data Engineer
2 382 (20.73%) Architect
3 320 (17.36%) Developer
4 210 (11.39%) Data Architect
5 150 (8.14%) DevOps Engineer
6 141 (7.65%) Solutions Architect
7 139 (7.54%) Big Data Engineer
8 104 (5.64%) AWS Engineer
9 82 (4.45%) Java Developer
10 77 (4.18%) Azure Engineer
11 76 (4.12%) Azure Architect
12 75 (4.07%) Analyst
13 56 (3.04%) AWS DevOps
14 52 (2.82%) Consultant
15 50 (2.71%) Big Data Developer
16 49 (2.66%) AWS DevOps Engineer
16 49 (2.66%) Cloud Engineer
17 46 (2.50%) Cloud Architect
18 45 (2.44%) Software Engineer
19 43 (2.33%) Senior Developer
Libraries, Frameworks & Software Standards
1 391 (21.22%) Kafka
2 123 (6.67%) PySpark
3 84 (4.56%) Web Services
4 72 (3.91%) JSON
5 67 (3.64%) Spring
6 62 (3.36%) Apache NiFi
7 58 (3.15%) RESTful
8 53 (2.88%) .NET
9 52 (2.82%) Spark Streaming
10 49 (2.66%) Node.js
11 48 (2.60%) REST
12 41 (2.22%) HTML
12 41 (2.22%) XML
13 40 (2.17%) React
14 38 (2.06%) Apache Avro
15 34 (1.84%) YAML
16 33 (1.79%) CSS
17 28 (1.52%) ARM Templates
17 28 (1.52%) Spring Boot
18 27 (1.47%) AngularJS
Miscellaneous
1 95 (5.15%) Data Structures
2 75 (4.07%) Data Protection Act
3 61 (3.31%) Public Cloud
4 59 (3.20%) Algorithms
5 49 (2.66%) Cloud Native
6 46 (2.50%) Data Centre
7 42 (2.28%) Virtual Team
8 38 (2.06%) Hybrid Cloud
9 35 (1.90%) Management Information System
10 31 (1.68%) Enterprise Software
11 22 (1.19%) Self-Motivation
11 22 (1.19%) User Experience
12 21 (1.14%) Legacy Systems
12 21 (1.14%) Market Risk
12 21 (1.14%) Reference Data
13 20 (1.09%) Greenfield Project
13 20 (1.09%) iPad
14 18 (0.98%) Linux Command Line
15 17 (0.92%) Legacy Applications
16 12 (0.65%) Hedge funds
Operating Systems
1 255 (13.84%) Linux
2 120 (6.51%) Unix
3 54 (2.93%) Windows
4 22 (1.19%) VMS
5 18 (0.98%) Red Hat Enterprise Linux
6 16 (0.87%) CoreOS
7 5 (0.27%) AIX
7 5 (0.27%) Apple iOS
8 3 (0.16%) Android
9 2 (0.11%) Ubuntu
Processes & Methodologies
1 661 (35.87%) Agile Software Development
2 532 (28.87%) Analytics
3 523 (28.38%) DevOps
4 361 (19.59%) Data Modelling
5 347 (18.83%) ETL
6 299 (16.22%) Business Intelligence
7 245 (13.29%) Data Engineering
8 224 (12.15%) Scrum
9 223 (12.10%) Data Pipeline
10 210 (11.39%) Data Architecture
10 210 (11.39%) Data Science
11 193 (10.47%) Data Management
12 186 (10.09%) Migration
13 170 (9.22%) Machine Learning
13 170 (9.22%) Microservices
14 166 (9.01%) Software Engineering
15 161 (8.74%) Visualisation
16 158 (8.57%) Data Analytics
17 155 (8.41%) Artificial Intelligence
17 155 (8.41%) Data Integration
Programming Languages
1 637 (34.56%) Python
2 624 (33.86%) SQL
3 425 (23.06%) Java
4 307 (16.66%) Scala
5 88 (4.77%) JavaScript
6 83 (4.50%) R
7 67 (3.64%) C#
8 48 (2.60%) Bash Shell
8 48 (2.60%) Shell Script
9 46 (2.50%) Go
10 44 (2.39%) PL/SQL
11 38 (2.06%) PowerShell
12 34 (1.84%) Groovy
13 29 (1.57%) Java 8
14 25 (1.36%) C++
15 24 (1.30%) C
16 21 (1.14%) PHP
16 21 (1.14%) Ruby
17 17 (0.92%) Kotlin
17 17 (0.92%) TypeScript
Qualifications
1 184 (9.98%) Security Cleared
2 137 (7.43%) SC Cleared
3 127 (6.89%) Degree
4 51 (2.77%) Computer Science Degree
5 48 (2.60%) DV Cleared
6 21 (1.14%) BPSS Clearance
7 19 (1.03%) AWS Certification
8 11 (0.60%) Master's Degree
9 9 (0.49%) PMP
10 8 (0.43%) ISEB
10 8 (0.43%) PMI Certification
10 8 (0.43%) TOGAF Certification
11 7 (0.38%) ISTQB
11 7 (0.38%) PhD
12 6 (0.33%) BC Cleared
12 6 (0.33%) Microsoft Certification
12 6 (0.33%) OCP
12 6 (0.33%) Oracle Certification
12 6 (0.33%) SAS Certification
13 5 (0.27%) CISSP
Quality Assurance & Compliance
1 154 (8.36%) Data Quality
2 59 (3.20%) GDPR
3 44 (2.39%) QA
4 15 (0.81%) PMO
4 15 (0.81%) Sarbanes-Oxley
5 12 (0.65%) NCSC
6 10 (0.54%) IFRS
7 7 (0.38%) SLA
8 5 (0.27%) PCI DSS
9 4 (0.22%) Accessibility
9 4 (0.22%) MiFID
10 3 (0.16%) ISO 20022
10 3 (0.16%) ISO/IEC 27001
11 2 (0.11%) COBIT
11 2 (0.11%) IFRS 9
11 2 (0.11%) NIST
11 2 (0.11%) SOC 1
11 2 (0.11%) SOC 2
12 1 (0.054%) Actionable Recommendations
12 1 (0.054%) Disclosure Scotland
System Software
1 254 (13.78%) Docker
2 83 (4.50%) HDFS
3 46 (2.50%) Active Directory
4 20 (1.09%) Apache ZooKeeper
5 14 (0.76%) Apache Flume
6 13 (0.71%) VMware Infrastructure
7 7 (0.38%) Oracle RAC
7 7 (0.38%) Virtual Machines
8 2 (0.11%) VMware NSX
9 1 (0.054%) Active Directory Lightweight Directory Services
9 1 (0.054%) Platform Symphony
9 1 (0.054%) SRDF
Systems Management
1 272 (14.76%) Kubernetes
2 252 (13.67%) Terraform
3 193 (10.47%) Ansible
4 27 (1.47%) Hadoop YARN
4 27 (1.47%) Oozie
5 23 (1.25%) Kibana
5 23 (1.25%) Puppet
6 21 (1.14%) Grafana
7 16 (0.87%) Prometheus
8 14 (0.76%) Single Sign-On
9 12 (0.65%) Control-M
9 12 (0.65%) Opscode Chef
10 10 (0.54%) logstash
11 9 (0.49%) Docker Swarm
11 9 (0.49%) Istio
11 9 (0.49%) Rancher
12 8 (0.43%) Ambari
13 6 (0.33%) Data Guard
13 6 (0.33%) Oracle RMAN
14 5 (0.27%) WebInspect
Vendors
1 198 (10.74%) Google
2 193 (10.47%) Databricks
3 185 (10.04%) Oracle
4 162 (8.79%) Microsoft
5 160 (8.68%) Cloudera
6 88 (4.77%) Informatica
7 74 (4.02%) Hortonworks
8 59 (3.20%) Salesforce.com
9 53 (2.88%) SAP
10 50 (2.71%) Teradata
11 40 (2.17%) Ab Initio
12 37 (2.01%) BiZZdesign
13 31 (1.68%) Red Hat
14 29 (1.57%) IBM
15 28 (1.52%) StreamSets
16 26 (1.41%) Talend
17 25 (1.36%) VMware
18 22 (1.19%) Splunk
19 21 (1.14%) MuleSoft
20 20 (1.09%) Confluent
499 Big Data job vacancies Nationwide