Period
to 16 January 2022

The following table provides summary statistics for contract job vacancies advertised in London with a requirement for Spark Streaming skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Spark Streaming over the 6 months to 16 January 2022 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark Streaming
England > London
6 months to
16 Jan 2022
Same period 2021 Same period 2020
Rank 522 389 470
Rank change year-on-year -133 +81 +99
Contract jobs citing Spark Streaming 55 17 51
As % of all contract jobs advertised in London 0.15% 0.099% 0.17%
As % of the Libraries, Frameworks & Software Standards category 0.46% 0.28% 0.47%
Number of daily rates quoted 47 8 30
10th Percentile £430 £439 £456
25th Percentile £484 £488 £531
Median daily rate (50th Percentile) £635 £556 £604
Median % change year-on-year +14.16% -7.91% -8.83%
75th Percentile £675 £622 £650
90th Percentile £680 £665 £689
England median daily rate £600 £475 £600
% change year-on-year +26.32% -20.83% -

Spark Streaming is in the Software Libraries and Frameworks category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in London with a requirement for technical specification, industry standards, software libraries and framework skills.

All Software Libraries and Frameworks
London
Contract vacancies with a requirement for technical specification, industry standards, software libraries and framework skills 11,911 6,159 10,785
As % of all contract IT jobs advertised in London 32.44% 36.00% 35.50%
Number of daily rates quoted 9,173 4,599 8,082
10th Percentile £413 £355 £375
25th Percentile £500 £425 £440
Median daily rate (50th Percentile) £590 £525 £525
Median % change year-on-year +12.38% - -
75th Percentile £688 £613 £613
90th Percentile £763 £700 £700
England median daily rate £550 £488 £500
% change year-on-year +12.82% -2.50% +2.56%
Number of hourly rates quoted 59 25 59
10th Percentile £20.71 £22.75 £20.85
25th Percentile £35.00 £30.68 £36.60
Median hourly rate £55.50 £45.00 £47.83
Median % change year-on-year +23.33% -5.92% +25.87%
75th Percentile £74.09 £54.00 £53.28
90th Percentile £80.50 £63.40 £78.75
England median hourly rate £50.88 £46.00 £46.00
% change year-on-year +10.60% - +9.93%

Spark Streaming
Job Vacancy Trend in London

Job postings citing Spark Streaming as a proportion of all IT jobs advertised in London.

Job vacancy trend for Spark Streaming in London

Spark Streaming
Contractor Daily Rate Trend in London

3-month moving average daily rate quoted in jobs citing Spark Streaming in London.

Daily rate trend for Spark Streaming in London

Spark Streaming
Daily Rate Histogram in London

Daily rate distribution for jobs citing Spark Streaming in London over the 6 months to 16 January 2022.

Daily rate histogram for Spark Streaming in London

Spark Streaming
Contract Job Locations in London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Spark Streaming within the London region over the 6 months to 16 January 2022. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London - 5 £500 - 5
South London - 1 £625 -
Spark Streaming
England

For the 6 months to 16 January 2022, IT contractor jobs citing Spark Streaming also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the London region with a requirement for Spark Streaming.

1 54 (98.18%) Apache Spark
2 47 (85.45%) Kafka
3 34 (61.82%) Analytics
3 34 (61.82%) Hadoop
4 32 (58.18%) Big Data
5 29 (52.73%) Apache Cassandra
6 27 (49.09%) NoSQL
6 27 (49.09%) HBase
7 26 (47.27%) Data Warehouse
8 25 (45.45%) Python
9 23 (41.82%) Apache Hive
10 22 (40.00%) SQL
10 22 (40.00%) Data Pipeline
11 21 (38.18%) ETL
12 20 (36.36%) MongoDB
12 20 (36.36%) Data Analytics
13 19 (34.55%) MapR
13 19 (34.55%) Linux
13 19 (34.55%) Ab Initio
14 18 (32.73%) HDFS
15 17 (30.91%) Use Case
15 17 (30.91%) Banking
15 17 (30.91%) Data Management
15 17 (30.91%) Business Intelligence
16 16 (29.09%) Java
16 16 (29.09%) Performance Tuning
16 16 (29.09%) AWS
16 16 (29.09%) Data Engineering
17 15 (27.27%) Apache Airflow
17 15 (27.27%) Social Skills

Spark Streaming
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 54 (98.18%) Apache Spark
2 15 (27.27%) Apache Airflow
3 8 (14.55%) Apache
4 3 (5.45%) Elasticsearch
5 1 (1.82%) Confluence
5 1 (1.82%) Luigi
Cloud Services
1 16 (29.09%) AWS
2 14 (25.45%) Azure
3 10 (18.18%) Azure Data Factory
4 9 (16.36%) GCP
5 8 (14.55%) Amazon EMR
5 8 (14.55%) Snowflake
6 7 (12.73%) Amazon Kinesis
6 7 (12.73%) Amazon S3
6 7 (12.73%) Serverless
7 5 (9.09%) Azure Functions
7 5 (9.09%) Azure Synapse Analytics
7 5 (9.09%) Power Platform
8 4 (7.27%) Azure Logic Apps
8 4 (7.27%) Azure Monitor
9 3 (5.45%) AWS Lambda
9 3 (5.45%) Azure Cosmos DB
10 1 (1.82%) Amazon EC2
10 1 (1.82%) Azure Machine Learning
Communications & Networking
1 1 (1.82%) FTP
Database & Business Intelligence
1 34 (61.82%) Hadoop
2 32 (58.18%) Big Data
3 29 (52.73%) Apache Cassandra
4 27 (49.09%) HBase
4 27 (49.09%) NoSQL
5 26 (47.27%) Data Warehouse
6 23 (41.82%) Apache Hive
7 20 (36.36%) MongoDB
8 12 (21.82%) Flink
9 11 (20.00%) Data Lake
10 8 (14.55%) Amazon Redshift
11 7 (12.73%) BigQuery
11 7 (12.73%) MySQL
12 6 (10.91%) Relational Database
12 6 (10.91%) Sqoop
13 5 (9.09%) Power BI
14 3 (5.45%) Amazon Athena
14 3 (5.45%) RDBMS
15 2 (3.64%) PostgreSQL
16 1 (1.82%) Microsoft Analysis Services
Development Applications
1 12 (21.82%) Git (software)
2 9 (16.36%) Jenkins
3 3 (5.45%) CircleCI
3 3 (5.45%) JIRA
4 1 (1.82%) kappa
4 1 (1.82%) Maven
4 1 (1.82%) MLflow
4 1 (1.82%) Visual Studio
General
1 17 (30.91%) Banking
2 15 (27.27%) Social Skills
3 13 (23.64%) Investment Banking
4 12 (21.82%) Finance
5 3 (5.45%) Games
Job Titles
1 31 (56.36%) Data Engineer
2 18 (32.73%) Big Data Manager
2 18 (32.73%) Data Manager
2 18 (32.73%) Platform Manager
3 4 (7.27%) Azure Engineer
4 3 (5.45%) AWS Engineer
4 3 (5.45%) Big Data Engineer
4 3 (5.45%) Developer
4 3 (5.45%) QA Engineer
4 3 (5.45%) Senior Data Engineer
5 2 (3.64%) Hadoop Developer
6 1 (1.82%) Big Data Architect
6 1 (1.82%) Cloud Engineer
6 1 (1.82%) Core Java Developer
6 1 (1.82%) Cybersecurity Engineer
6 1 (1.82%) Data Architect
6 1 (1.82%) Java Developer
6 1 (1.82%) Java Engineer
6 1 (1.82%) Platform Engineer
6 1 (1.82%) Solutions Architect
Libraries, Frameworks & Software Standards
1 47 (85.45%) Kafka
2 10 (18.18%) PySpark
3 3 (5.45%) .NET
4 1 (1.82%) Apache NiFi
4 1 (1.82%) ARM Templates
Miscellaneous
1 4 (7.27%) Data Structures
1 4 (7.27%) iPad
1 4 (7.27%) Linux Command Line
2 3 (5.45%) Algorithms
2 3 (5.45%) Self-Motivation
3 1 (1.82%) Data Centre
3 1 (1.82%) Greenfield Project
3 1 (1.82%) Legacy Systems
Operating Systems
1 19 (34.55%) Linux
Processes & Methodologies
1 34 (61.82%) Analytics
2 22 (40.00%) Data Pipeline
3 21 (38.18%) ETL
4 20 (36.36%) Data Analytics
5 17 (30.91%) Business Intelligence
5 17 (30.91%) Data Management
5 17 (30.91%) Use Case
6 16 (29.09%) Data Engineering
6 16 (29.09%) Performance Tuning
7 14 (25.45%) Continuous Improvement
7 14 (25.45%) Problem-Solving
8 12 (21.82%) Agile Software Development
8 12 (21.82%) BDD
8 12 (21.82%) DevOps
8 12 (21.82%) TDD
9 11 (20.00%) Scrum
10 9 (16.36%) Data Ingestion
10 9 (16.36%) Machine Learning
11 7 (12.73%) Data Transformation
12 6 (10.91%) Data Modelling
Programming Languages
1 25 (45.45%) Python
2 22 (40.00%) SQL
3 16 (29.09%) Java
4 12 (21.82%) Scala
5 5 (9.09%) C#
5 5 (9.09%) PowerShell
6 4 (7.27%) Bash
7 2 (3.64%) CQL
7 2 (3.64%) Perl
7 2 (3.64%) U-SQL
8 1 (1.82%) T-SQL
Qualifications
1 3 (5.45%) FFIEC
2 2 (3.64%) Computer Science Degree
2 2 (3.64%) Degree
2 2 (3.64%) PMI Certification
2 2 (3.64%) PMP
Quality Assurance & Compliance
1 10 (18.18%) Data Quality
2 3 (5.45%) GLBA
2 3 (5.45%) NIST
2 3 (5.45%) PCI DSS
2 3 (5.45%) QA
System Software
1 18 (32.73%) HDFS
2 12 (21.82%) Docker
3 3 (5.45%) Apache Flume
Systems Management
1 15 (27.27%) Kubernetes
2 8 (14.55%) Terraform
3 5 (9.09%) Oozie
4 1 (1.82%) Hadoop YARN
Vendors
1 19 (34.55%) Ab Initio
1 19 (34.55%) MapR
2 12 (21.82%) Databricks
3 5 (9.09%) Cloudera
4 2 (3.64%) StreamSets
5 1 (1.82%) Google
5 1 (1.82%) Hortonworks
5 1 (1.82%) Microsoft