Period
to 26 September 2021

The following table provides summary statistics for contract job vacancies advertised in England with a requirement for Spark Streaming skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Spark Streaming over the 6 months to 26 September 2021 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark Streaming
UK > England
6 months to
26 Sep 2021
Same period 2020 Same period 2019
Rank 644 473 732
Rank change year-on-year -171 +259 +10
Contract jobs citing Spark Streaming 94 18 78
As % of all contract jobs advertised in England 0.15% 0.070% 0.097%
As % of the Libraries, Frameworks & Software Standards category 0.50% 0.22% 0.31%
Number of daily rates quoted 66 8 65
10th Percentile £413 £448 £471
25th Percentile £452 £509 £525
Median daily rate (50th Percentile) £497 £563 £575
Median % change year-on-year -11.60% -2.17% +4.55%
75th Percentile £645 £622 £668
90th Percentile £675 £665 £688
UK median daily rate £525 £563 £575
% change year-on-year -6.67% -2.17% +4.55%

Spark Streaming is in the Software Libraries and Frameworks category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in England with a requirement for technical specification, industry standards, software libraries and framework skills.

All Software Libraries and Frameworks
England
Contract vacancies with a requirement for technical specification, industry standards, software libraries and framework skills 18,884 8,186 25,403
As % of all contract IT jobs advertised in England 29.62% 31.98% 31.57%
Number of daily rates quoted 13,750 5,866 18,041
10th Percentile £363 £320 £340
25th Percentile £425 £408 £408
Median daily rate (50th Percentile) £525 £500 £500
Median % change year-on-year +5.00% - +5.26%
75th Percentile £625 £600 £575
90th Percentile £700 £688 £650
UK median daily rate £525 £500 £495
% change year-on-year +5.00% +1.01% +4.21%
Number of hourly rates quoted 203 64 232
10th Percentile £18.52 £21.80 £16.25
25th Percentile £30.63 £30.00 £30.90
Median hourly rate £49.00 £43.75 £46.45
Median % change year-on-year +12.00% -5.81% +10.60%
75th Percentile £63.00 £53.81 £53.75
90th Percentile £71.50 £60.13 £68.38
UK median hourly rate £48.88 £43.00 £45.00
% change year-on-year +13.68% -4.44% +7.64%

Spark Streaming
Job Vacancy Trend in England

Job postings citing Spark Streaming as a proportion of all IT jobs advertised in England.

Job vacancy trend for Spark Streaming in England

Spark Streaming
Contractor Daily Rate Trend in England

3-month moving average daily rate quoted in jobs citing Spark Streaming in England.

Daily rate trend for Spark Streaming in England

Spark Streaming
Daily Rate Histogram in England

Daily rate distribution for jobs citing Spark Streaming in England over the 6 months to 26 September 2021.

Daily rate histogram for Spark Streaming in England

Spark Streaming
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Spark Streaming within the England region over the 6 months to 26 September 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London -149 58 £500 -11.11% 6
North of England - 13 £413 - 1
South East - 12 £563 - 1
Yorkshire - 8 £388 -
North West - 5 £489 - 1
Midlands - 4 £463 -
South West - 4 £500 - 1
West Midlands - 2 £463 -
East of England - 2 - - 1
Spark Streaming
UK

For the 6 months to 26 September 2021, IT contractor jobs citing Spark Streaming also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region with a requirement for Spark Streaming.

1 92 (97.87%) Apache Spark
2 82 (87.23%) Kafka
3 56 (59.57%) Data Pipeline
4 53 (56.38%) Analytics
4 53 (56.38%) Hadoop
5 46 (48.94%) Java
6 45 (47.87%) Azure
7 44 (46.81%) Python
8 43 (45.74%) HBase
9 39 (41.49%) SQL
10 38 (40.43%) Scala
11 37 (39.36%) DevOps
12 36 (38.30%) NoSQL
13 34 (36.17%) Apache Cassandra
14 33 (35.11%) Big Data
15 31 (32.98%) Data Ingestion
15 31 (32.98%) Apache Hive
16 30 (31.91%) Databricks
17 29 (30.85%) Data Engineering
18 28 (29.79%) Kubernetes
18 28 (29.79%) MongoDB
19 27 (28.72%) Flink
20 25 (26.60%) GCP
21 24 (25.53%) AWS
22 23 (24.47%) Machine Learning
22 23 (24.47%) Data Warehouse
23 21 (22.34%) Data Modelling
23 21 (22.34%) MySQL
23 21 (22.34%) Apache Airflow
24 19 (20.21%) Azure Data Factory

Spark Streaming
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 92 (97.87%) Apache Spark
2 21 (22.34%) Apache Airflow
3 9 (9.57%) Apache
4 8 (8.51%) Luigi
5 5 (5.32%) Oracle Workflow
6 3 (3.19%) Elasticsearch
7 2 (2.13%) SAS
8 1 (1.06%) Apache Pig
8 1 (1.06%) Apache Solr
8 1 (1.06%) Confluence
Cloud Services
1 45 (47.87%) Azure
2 25 (26.60%) GCP
3 24 (25.53%) AWS
4 19 (20.21%) Azure Data Factory
5 12 (12.77%) Azure Cosmos DB
6 11 (11.70%) Amazon EMR
7 9 (9.57%) Azure Synapse Analytics
8 8 (8.51%) Amazon EC2
8 8 (8.51%) Serverless
9 7 (7.45%) AWS Lambda
9 7 (7.45%) Snowflake
10 6 (6.38%) AWS Data Pipeline
11 5 (5.32%) Amazon S3
11 5 (5.32%) Azure Active Directory
11 5 (5.32%) Azure Key Vault
11 5 (5.32%) Azure Logic Apps
11 5 (5.32%) Google Analytics
12 3 (3.19%) GitHub
13 2 (2.13%) Amazon Kinesis
13 2 (2.13%) Azure Data Lake Analytics
Communications & Networking
1 5 (5.32%) Cisco ASA
2 2 (2.13%) Kerberos
2 2 (2.13%) PuTTY
3 1 (1.06%) Firewall
3 1 (1.06%) FTP
3 1 (1.06%) SSL
Database & Business Intelligence
1 53 (56.38%) Hadoop
2 43 (45.74%) HBase
3 36 (38.30%) NoSQL
4 34 (36.17%) Apache Cassandra
5 33 (35.11%) Big Data
6 31 (32.98%) Apache Hive
7 28 (29.79%) MongoDB
8 27 (28.72%) Flink
9 23 (24.47%) Data Warehouse
10 21 (22.34%) MySQL
11 18 (19.15%) Data Lake
12 12 (12.77%) HDInsight
13 10 (10.64%) Amazon Redshift
13 10 (10.64%) BigQuery
13 10 (10.64%) Metadata
13 10 (10.64%) PostgreSQL
13 10 (10.64%) Relational Database
14 8 (8.51%) Amazon RDS
14 8 (8.51%) Azkaban
14 8 (8.51%) Sqoop
Development Applications
1 19 (20.21%) Jenkins
2 18 (19.15%) Git (software)
3 5 (5.32%) kappa
3 5 (5.32%) Oracle APEX
3 5 (5.32%) Visual Studio
4 4 (4.26%) Maven
5 3 (3.19%) JIRA
6 2 (2.13%) ProC
7 1 (1.06%) Octopus Deploy
General
1 19 (20.21%) Banking
2 10 (10.64%) Social Skills
3 9 (9.57%) Investment Banking
4 5 (5.32%) Retail Banking
5 4 (4.26%) Finance
6 2 (2.13%) Games
6 2 (2.13%) Public Sector
7 1 (1.06%) Presentation Skills
Job Titles
1 59 (62.77%) Data Engineer
2 15 (15.96%) Azure Engineer
3 10 (10.64%) Big Data Manager
3 10 (10.64%) Data Manager
3 10 (10.64%) Platform Manager
4 7 (7.45%) Scala Engineer
5 6 (6.38%) Developer
6 5 (5.32%) Big Data Engineer
7 4 (4.26%) Architect
7 4 (4.26%) Java Engineer
7 4 (4.26%) Senior Data Engineer
8 3 (3.19%) Azure DevOps Engineer
8 3 (3.19%) DevOps Engineer
8 3 (3.19%) Java Developer
8 3 (3.19%) Platform Engineer
9 2 (2.13%) AWS Engineer
9 2 (2.13%) Hadoop Developer
9 2 (2.13%) Hadoop Engineer
9 2 (2.13%) Scala Developer
9 2 (2.13%) Scala Software Engineer
Libraries, Frameworks & Software Standards
1 82 (87.23%) Kafka
2 16 (17.02%) PySpark
3 6 (6.38%) JSON
4 5 (5.32%) ARM Templates
4 5 (5.32%) Oracle GoldenGate
4 5 (5.32%) SAML
5 4 (4.26%) Apache NiFi
6 2 (2.13%) .NET
6 2 (2.13%) Apache Avro
6 2 (2.13%) Dojo
7 1 (1.06%) J2EE
7 1 (1.06%) JDBC
7 1 (1.06%) Lucene
7 1 (1.06%) MLlib
7 1 (1.06%) Node.js
7 1 (1.06%) Protocol Buffers
7 1 (1.06%) SparkR
Miscellaneous
1 18 (19.15%) iPad
2 11 (11.70%) Data Structures
3 7 (7.45%) Management Information System
4 3 (3.19%) Data Centre
5 1 (1.06%) Algorithms
5 1 (1.06%) Cloud Native
5 1 (1.06%) Greenfield Project
5 1 (1.06%) Self-Motivation
Operating Systems
1 11 (11.70%) Linux
2 3 (3.19%) Unix
3 1 (1.06%) Red Hat Enterprise Linux
Processes & Methodologies
1 56 (59.57%) Data Pipeline
2 53 (56.38%) Analytics
3 37 (39.36%) DevOps
4 31 (32.98%) Data Ingestion
5 29 (30.85%) Data Engineering
6 23 (24.47%) Machine Learning
7 21 (22.34%) Data Modelling
8 19 (20.21%) Data Analytics
8 19 (20.21%) Data Transformation
9 18 (19.15%) Agile Software Development
9 18 (19.15%) E-Commerce
10 17 (18.09%) BDD
10 17 (18.09%) TDD
11 16 (17.02%) Business Intelligence
11 16 (17.02%) ETL
11 16 (17.02%) Use Case
12 15 (15.96%) Computer Science
12 15 (15.96%) Workflow
13 13 (13.83%) Open Source
14 11 (11.70%) Scrum
Programming Languages
1 46 (48.94%) Java
2 44 (46.81%) Python
3 39 (41.49%) SQL
4 38 (40.43%) Scala
5 11 (11.70%) CQL
6 9 (9.57%) Perl
7 7 (7.45%) C++
7 7 (7.45%) U-SQL
8 5 (5.32%) PL/SQL
8 5 (5.32%) PowerShell
9 2 (2.13%) Bash Shell
9 2 (2.13%) R
10 1 (1.06%) PROC SQL
10 1 (1.06%) Ruby
10 1 (1.06%) Shell Script
Qualifications
1 15 (15.96%) Computer Science Degree
1 15 (15.96%) Degree
2 3 (3.19%) Security Cleared
Quality Assurance & Compliance
1 18 (19.15%) Data Quality
2 2 (2.13%) QA
System Software
1 13 (13.83%) HDFS
2 12 (12.77%) Docker
3 5 (5.32%) Active Directory
4 2 (2.13%) Apache Flume
5 1 (1.06%) Apache ZooKeeper
Systems Management
1 28 (29.79%) Kubernetes
2 16 (17.02%) Oozie
3 10 (10.64%) Terraform
4 5 (5.32%) Active Directory Federation Services
4 5 (5.32%) FortiGate
4 5 (5.32%) Opscode Chef
4 5 (5.32%) Puppet
5 2 (2.13%) Ansible
5 2 (2.13%) Hadoop YARN
5 2 (2.13%) Kibana
5 2 (2.13%) logstash
6 1 (1.06%) Ambari
6 1 (1.06%) Salt
Vendors
1 30 (31.91%) Databricks
2 11 (11.70%) Confluent
3 10 (10.64%) Ab Initio
3 10 (10.64%) Cloudera
3 10 (10.64%) Google
3 10 (10.64%) MapR
4 6 (6.38%) CheckPoint
5 5 (5.32%) Cisco
5 5 (5.32%) Microsoft
5 5 (5.32%) Oracle
6 2 (2.13%) Genesys
6 2 (2.13%) Informatica
6 2 (2.13%) StreamSets
7 1 (1.06%) MicroStrategy
7 1 (1.06%) Red Hat