Period
to 8 December 2021

The median contractor rate for a Big Data Engineer was £625 per day in advertised job vacancies in London during the 6 months to 8 December 2021.

The first table below provides contractor rate benchmarking and summary statistics including a comparison to same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Big Data Engineer
England > London
6 months to
8 Dec 2021
Same period 2020 Same period 2019
Rank 486 327 437
Rank change year-on-year -159 +110 -39
Contract jobs requiring a Big Data Engineer 102 60 114
As % of all contract jobs advertised in London 0.27% 0.37% 0.33%
As % of the Job Titles category 0.29% 0.40% 0.36%
Number of daily rates quoted 73 49 101
10th Percentile £450 £445 £438
25th Percentile £538 £512 £513
Median daily rate (50th Percentile) £625 £550 £560
Median % change year-on-year +13.64% -1.79% -2.61%
75th Percentile £671 £588 £650
90th Percentile £710 £613 £695
England median daily rate £600 £550 £550
% change year-on-year +9.09% - -

The following table is for comparison with the above and includes summary statistics for all contract IT job vacancies advertised in London. Most job vacancies include a discernible job title that can be normalized. As such, the figures in the second row provide an indication of the number of contract jobs in our overall sample.

All Contract IT Job Vacancies
London
Contract vacancies in London with a recognized job title 35,495 14,975 32,021
% of contract IT jobs with a recognized job title 93.07% 91.75% 93.62%
Number of daily rates quoted 25,780 10,858 23,413
10th Percentile £363 £313 £300
25th Percentile £460 £406 £408
Median daily rate (50th Percentile) £550 £500 £500
Median % change year-on-year +10.00% - -
75th Percentile £655 £610 £600
90th Percentile £750 £700 £687
England median daily rate £517 £475 £463
% change year-on-year +8.75% +2.70% +1.87%
Number of hourly rates quoted 386 125 429
10th Percentile £14.69 £18.00 £12.40
25th Percentile £20.00 £22.50 £15.48
Median hourly rate £30.00 £32.50 £20.50
Median % change year-on-year -7.69% +58.54% +8.75%
75th Percentile £51.69 £46.00 £38.75
90th Percentile £67.25 £61.45 £52.60
England median hourly rate £20.00 £23.00 £18.17
% change year-on-year -13.04% +26.58% +3.83%

Big Data Engineer
Job Vacancy Trend in London

Job postings that featured Big Data Engineer in the job title as a proportion of all IT jobs advertised in London.

Job vacancy trend for Big Data Engineer in London

Big Data Engineer
Contractor Daily Rate Trend in London

3-month moving average daily rate quoted in jobs citing Big Data Engineer in London.

Daily rate trend for Big Data Engineer in London

Big Data Engineer
Daily Rate Histogram in London

Daily rate distribution for jobs citing Big Data Engineer in London over the 6 months to 8 December 2021.

Daily rate histogram for Big Data Engineer in London

Big Data Engineer
Contract Job Locations in London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Big Data Engineer within the London region over the 6 months to 8 December 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
South London +5 5 £540 +8.00% 1
West London -18 5 £613 +42.77% 3
Central London -56 4 £660 +20.00% 4
Big Data Engineer
England

Big Data Engineer Skill Set
Top 30 Co-occurring IT Skills in London

For the 6 months to 8 December 2021, Big Data Engineer contract jobs required the following IT skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the London region featuring Big Data Engineer in the job title.

1 102 (100.00%) Big Data
2 71 (69.61%) Apache Spark
3 53 (51.96%) Python
4 51 (50.00%) Scala
5 50 (49.02%) Finance
6 46 (45.10%) SQL
6 46 (45.10%) Apache Hive
7 43 (42.16%) Agile Software Development
8 39 (38.24%) AWS
9 38 (37.25%) Hadoop
10 35 (34.31%) Azure
11 32 (31.37%) Banking
11 32 (31.37%) Java
12 31 (30.39%) Kafka
13 30 (29.41%) Jenkins
14 29 (28.43%) Data Modelling
15 27 (26.47%) Scrum
16 26 (25.49%) Git (software)
17 24 (23.53%) Ansible
18 23 (22.55%) Analytics
18 23 (22.55%) Hudson
19 22 (21.57%) Kubernetes
19 22 (21.57%) Deployment Automation
20 21 (20.59%) DevOps
20 21 (20.59%) PostgreSQL
21 20 (19.61%) Data Warehouse
21 20 (19.61%) Docker
21 20 (19.61%) Relational Database
21 20 (19.61%) Cloudera
21 20 (19.61%) PySpark

Big Data Engineer Skill Set
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 71 (69.61%) Apache Spark
2 16 (15.69%) Elasticsearch
3 14 (13.73%) Apache
4 12 (11.76%) Tomcat
5 11 (10.78%) Confluence
6 4 (3.92%) Apache Airflow
7 3 (2.94%) Luigi
7 3 (2.94%) SAS
8 2 (1.96%) Apache Pig
9 1 (0.98%) Oracle Coherence
Applications
1 3 (2.94%) Microsoft Excel
Cloud Services
1 39 (38.24%) AWS
2 35 (34.31%) Azure
3 13 (12.75%) GCP
4 6 (5.88%) Azure Cosmos DB
4 6 (5.88%) Azure Functions
4 6 (5.88%) GitHub
4 6 (5.88%) Serverless
5 4 (3.92%) Akamai
5 4 (3.92%) Amazon S3
5 4 (3.92%) Azure Data Factory
6 3 (2.94%) Amazon SQS
6 3 (2.94%) AWS Lambda
6 3 (2.94%) Azure Service Bus
7 2 (1.96%) Amazon Kinesis
7 2 (1.96%) Azure Service Fabric
7 2 (1.96%) IaaS
7 2 (1.96%) PaaS
7 2 (1.96%) Power Platform
7 2 (1.96%) SaaS
7 2 (1.96%) Snowflake
Communications & Networking
1 3 (2.94%) Intranet
Database & Business Intelligence
1 102 (100.00%) Big Data
2 46 (45.10%) Apache Hive
3 38 (37.25%) Hadoop
4 21 (20.59%) PostgreSQL
5 20 (19.61%) Data Warehouse
5 20 (19.61%) Relational Database
6 17 (16.67%) Tableau
7 13 (12.75%) Data Lake
7 13 (12.75%) SQL Server
8 12 (11.76%) Azure SQL Data Warehouse
8 12 (11.76%) Parquet
9 10 (9.80%) NoSQL
10 8 (7.84%) HBase
11 7 (6.86%) Amazon Athena
11 7 (6.86%) Amazon Redshift
11 7 (6.86%) Apache Cassandra
12 6 (5.88%) MongoDB
12 6 (5.88%) Spark SQL
13 4 (3.92%) Flink
14 3 (2.94%) Metadata
Development Applications
1 30 (29.41%) Jenkins
2 26 (25.49%) Git (software)
3 23 (22.55%) Hudson
4 18 (17.65%) GitLab
5 17 (16.67%) JIRA
6 14 (13.73%) Maven
7 12 (11.76%) IntelliJ
7 12 (11.76%) Subversion
8 7 (6.86%) TeamCity
9 4 (3.92%) GoCD
10 2 (1.96%) Atlassian Bamboo
10 2 (1.96%) IBM UrbanCode
10 2 (1.96%) Octopus Deploy
10 2 (1.96%) RStudio
11 1 (0.98%) Team Foundation Server
11 1 (0.98%) tSQLt
General
1 50 (49.02%) Finance
2 32 (31.37%) Banking
3 17 (16.67%) Social Skills
4 12 (11.76%) Investment Banking
5 10 (9.80%) Analytical Skills
6 6 (5.88%) Financial Institution
7 3 (2.94%) Public Sector
7 3 (2.94%) Publishing
8 2 (1.96%) Presentation Skills
9 1 (0.98%) Law
9 1 (0.98%) Telecoms
Libraries, Frameworks & Software Standards
1 31 (30.39%) Kafka
2 20 (19.61%) PySpark
3 15 (14.71%) Apache Avro
4 14 (13.73%) Elastic Stack
5 13 (12.75%) Spring
6 12 (11.76%) OAuth
6 12 (11.76%) OAuth2
7 8 (7.84%) Java Concurrency
7 8 (7.84%) RabbitMQ
7 8 (7.84%) Spring Boot
8 3 (2.94%) JSON
8 3 (2.94%) Lambda Expressions
8 3 (2.94%) Protocol Buffers
8 3 (2.94%) RESTful
8 3 (2.94%) Spark Streaming
8 3 (2.94%) XML
8 3 (2.94%) XSLT
8 3 (2.94%) YAML
9 2 (1.96%) Keras
9 2 (1.96%) Theano
Miscellaneous
1 8 (7.84%) Legacy Applications
2 6 (5.88%) Data Protection Act
3 5 (4.90%) Cloud Native
3 5 (4.90%) Data Centre
4 4 (3.92%) Algorithms
5 3 (2.94%) Data Structures
5 3 (2.94%) Management Information System
6 1 (0.98%) Greenfield Project
6 1 (0.98%) Legacy Systems
6 1 (0.98%) Public Cloud
6 1 (0.98%) Taxonomies
Operating Systems
1 19 (18.63%) Unix
2 11 (10.78%) Linux
3 2 (1.96%) Red Hat Enterprise Linux
3 2 (1.96%) VMS
Processes & Methodologies
1 43 (42.16%) Agile Software Development
2 29 (28.43%) Data Modelling
3 27 (26.47%) Scrum
4 23 (22.55%) Analytics
5 22 (21.57%) Deployment Automation
6 21 (20.59%) DevOps
7 18 (17.65%) Data Pipeline
8 16 (15.69%) ETL
9 14 (13.73%) Data Engineering
10 12 (11.76%) Mentoring
10 12 (11.76%) Software Engineering
11 11 (10.78%) Data Science
12 10 (9.80%) SDLC
13 9 (8.82%) Microservices
13 9 (8.82%) Visualisation
14 8 (7.84%) Front End Development
14 8 (7.84%) Infrastructure as Code
14 8 (7.84%) Machine Learning
14 8 (7.84%) Multithreaded Programming
14 8 (7.84%) Multithreading
Programming Languages
1 53 (51.96%) Python
2 51 (50.00%) Scala
3 46 (45.10%) SQL
4 32 (31.37%) Java
5 15 (14.71%) PL/SQL
6 12 (11.76%) Java 8
6 12 (11.76%) Kotlin
7 6 (5.88%) Bash Shell
7 6 (5.88%) Go
8 5 (4.90%) R
9 4 (3.92%) C#
9 4 (3.92%) HiveQL
9 4 (3.92%) T-SQL
10 2 (1.96%) Groovy
10 2 (1.96%) PowerShell
11 1 (0.98%) Cypher
11 1 (0.98%) U-SQL
Qualifications
1 5 (4.90%) Security Cleared
2 3 (2.94%) Computer Science Degree
2 3 (2.94%) Degree
2 3 (2.94%) DV Cleared
3 1 (0.98%) AWS Certification
Quality Assurance & Compliance
1 6 (5.88%) Data Quality
2 3 (2.94%) IFRS
3 1 (0.98%) GDPR
3 1 (0.98%) QA
System Software
1 20 (19.61%) Docker
2 5 (4.90%) HDFS
3 2 (1.96%) Apache Flume
Systems Management
1 24 (23.53%) Ansible
2 22 (21.57%) Kubernetes
3 12 (11.76%) Oozie
3 12 (11.76%) Single Sign-On
4 11 (10.78%) Kibana
5 10 (9.80%) logstash
6 9 (8.82%) Terraform
7 8 (7.84%) Docker Swarm
8 4 (3.92%) Grafana
8 4 (3.92%) Hadoop YARN
8 4 (3.92%) Istio
8 4 (3.92%) Prometheus
8 4 (3.92%) Puppet
9 2 (1.96%) Ambari
10 1 (0.98%) Active Directory Federation Services
Vendors
1 20 (19.61%) Cloudera
2 18 (17.65%) Oracle
3 11 (10.78%) Hortonworks
4 8 (7.84%) Databricks
5 7 (6.86%) Teradata
6 5 (4.90%) Google
7 4 (3.92%) Netezza
8 3 (2.94%) Confluent
8 3 (2.94%) Informatica
8 3 (2.94%) Microsoft
8 3 (2.94%) Salesforce.com
9 2 (1.96%) Alteryx
9 2 (1.96%) IBM
9 2 (1.96%) Red Hat