Period
to 20 June 2021

The following table provides summary statistics for permanent job vacancies with a requirement for Spark Streaming skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Spark Streaming over the 6 months to 20 June 2021 with a comparison to the same period in the previous 2 years.

Apache Spark Streaming
UK
6 months to
20 Jun 2021
Same period 2020 Same period 2019
Rank 726 795 987
Rank change year-on-year +69 +192 +100
Permanent jobs citing Spark Streaming 226 71 122
As % of all permanent jobs advertised in the UK 0.22% 0.088% 0.085%
As % of the Libraries, Frameworks & Software Standards category 0.53% 0.22% 0.22%
Number of salaries quoted 110 42 94
10th Percentile " £56,250 £49,337
25th Percentile £46,250 £59,063 £57,188
Median annual salary (50th Percentile) £75,000 £70,000 £66,250
Median % change year-on-year +7.14% +5.66% -10.17%
75th Percentile £87,500 £86,250 £87,500
90th Percentile £97,500 £93,500 £98,875
UK excluding London median annual salary £50,000 £68,750 £63,354
% change year-on-year -27.27% +8.52% -2.53%

Spark Streaming is in the Software Libraries and Frameworks category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for technical specification, industry standards, software libraries and framework skills.

All Software Libraries and Frameworks
UK
Permanent vacancies with a requirement for technical specification, industry standards, software libraries and framework skills 42,775 32,272 54,556
As % of all permanent IT jobs advertised in the UK 41.52% 39.77% 37.96%
Number of salaries quoted 29,152 26,769 45,186
10th Percentile £35,000 £33,750 £31,250
25th Percentile £42,500 £42,500 £38,500
Median annual salary (50th Percentile) £57,500 £55,000 £52,500
Median % change year-on-year +4.55% +4.76% +5.00%
75th Percentile £75,000 £75,000 £71,250
90th Percentile £92,500 £91,250 £85,000
UK excluding London median annual salary £50,000 £47,500 £45,000
% change year-on-year +5.26% +5.56% -

Spark Streaming
Job Vacancy Trend

Job postings citing Spark Streaming as a proportion of all IT jobs advertised.

Job vacancy trend for Spark Streaming in the UK

Spark Streaming
Salary Trend

3-month moving average salary quoted in jobs citing Spark Streaming.

Salary trend for Spark Streaming in the UK

Spark Streaming
Salary Histogram

Salary distribution for jobs citing Spark Streaming over the 6 months to 20 June 2021.

Salary histogram for Spark Streaming in the UK

Spark Streaming
Top 9 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Spark Streaming within the UK over the 6 months to 20 June 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +67 217 £76,250 +8.93% 6
UK excluding London +39 114 £50,000 -27.27% 3
London -2 106 £80,000 +14.29% 3
South East +75 104 £97,500 +41.82%
Work from Home -204 14 £77,500 -3.13% 1
Scotland - 6 £50,000 -
East of England -19 2 £65,000 - 1
North of England -50 2 £47,500 - 2
Yorkshire - 2 £47,500 - 2

For the 6 months to 20 June 2021, IT jobs citing Spark Streaming also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Spark Streaming.

1 225 (99.56%) Apache Spark
2 184 (81.42%) Jenkins
3 183 (80.97%) Python
3 183 (80.97%) Agile Software Development
4 181 (80.09%) Software Engineering
4 181 (80.09%) Ansible
5 175 (77.43%) NoSQL
6 162 (71.68%) AWS
6 162 (71.68%) Big Data
7 146 (64.60%) Kafka
8 141 (62.39%) Java
9 131 (57.96%) Azure
10 129 (57.08%) GCP
11 128 (56.64%) Data Pipeline
12 120 (53.10%) DevOps
13 117 (51.77%) Degree
14 116 (51.33%) Finance
15 114 (50.44%) PostgreSQL
16 112 (49.56%) SQL
17 111 (49.12%) Git (software)
18 109 (48.23%) Computer Science
19 107 (47.35%) React
19 107 (47.35%) AngularJS
20 106 (46.90%) HDFS
20 106 (46.90%) Hadoop
21 105 (46.46%) JSON
21 105 (46.46%) Spring
21 105 (46.46%) Linux
21 105 (46.46%) Apache
22 104 (46.02%) CSS

Spark Streaming
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 225 (99.56%) Apache Spark
2 105 (46.46%) Apache
3 101 (44.69%) GemFire
4 20 (8.85%) Apache Airflow
4 20 (8.85%) Luigi
5 4 (1.77%) Oracle Workflow
6 3 (1.33%) Cloud Foundry
6 3 (1.33%) Elasticsearch
Cloud Services
1 162 (71.68%) AWS
2 131 (57.96%) Azure
3 129 (57.08%) GCP
4 101 (44.69%) GitHub
5 86 (38.05%) Amazon EMR
6 71 (31.42%) IBM Cloud
7 64 (28.32%) Amazon Kinesis
8 60 (26.55%) Snowflake
9 32 (14.16%) Serverless
10 30 (13.27%) Cloud Functions
10 30 (13.27%) Google App Engine
10 30 (13.27%) Google Compute Engine
10 30 (13.27%) Google Container Engine
11 26 (11.50%) OpenShift
12 11 (4.87%) Amazon EC2
13 8 (3.54%) AWS CloudFormation
14 7 (3.10%) PaaS
15 6 (2.65%) Amazon S3
15 6 (2.65%) Azure Data Factory
15 6 (2.65%) Power Platform
Database & Business Intelligence
1 175 (77.43%) NoSQL
2 162 (71.68%) Big Data
3 114 (50.44%) PostgreSQL
4 106 (46.90%) Hadoop
5 103 (45.58%) HBase
6 98 (43.36%) MariaDB
6 98 (43.36%) Redis
7 88 (38.94%) Relational Database
8 87 (38.50%) Amazon Redshift
9 80 (35.40%) Neo4j
10 75 (33.19%) Amazon Athena
11 67 (29.65%) Electronic Patient Records
12 18 (7.96%) Azkaban
12 18 (7.96%) Data Warehouse
12 18 (7.96%) Flink
13 11 (4.87%) Amazon RDS
13 11 (4.87%) Apache Hive
14 8 (3.54%) Amazon DynamoDB
14 8 (3.54%) BigQuery
14 8 (3.54%) SQL Server Integration Services
Development Applications
1 184 (81.42%) Jenkins
2 111 (49.12%) Git (software)
3 101 (44.69%) Cucumber
3 101 (44.69%) Cypress.io
3 101 (44.69%) Eclipse
3 101 (44.69%) Gradle
3 101 (44.69%) IntelliJ
3 101 (44.69%) JUnit
3 101 (44.69%) Protractor
3 101 (44.69%) Sonatype Nexus
3 101 (44.69%) Storybook
4 98 (43.36%) Mockito
5 76 (33.63%) Atlassian Bamboo
5 76 (33.63%) Vagrant
6 54 (23.89%) Artifactory
7 36 (15.93%) SonarQube
8 4 (1.77%) Bitbucket
8 4 (1.77%) Oracle APEX
8 4 (1.77%) Spinnaker
9 3 (1.33%) Visual Studio
General
1 116 (51.33%) Finance
2 102 (45.13%) Banking
3 101 (44.69%) Law
4 14 (6.19%) Social Skills
5 6 (2.65%) Analytical Skills
6 5 (2.21%) Manufacturing
6 5 (2.21%) Marketing
6 5 (2.21%) Retail
7 2 (0.88%) Czech Language
7 2 (0.88%) Presentation Skills
Job Titles
1 76 (33.63%) Software Engineer
2 73 (32.30%) Data Engineer
3 53 (23.45%) Architect
4 41 (18.14%) Principal Engineer
4 41 (18.14%) Principal Software Engineer
5 40 (17.70%) Data Architect
6 38 (16.81%) Platform Architect
7 36 (15.93%) Platform Engineer
8 31 (13.72%) Lead Software Engineer
9 27 (11.95%) AWS Engineer
10 22 (9.73%) AWS Architect
11 21 (9.29%) Developer
12 18 (7.96%) Software Developer
13 17 (7.52%) Lead Developer
13 17 (7.52%) Lead Software Developer
14 8 (3.54%) Principal Architect
14 8 (3.54%) Solutions Architect
15 6 (2.65%) Big Data Engineer
15 6 (2.65%) DevOps Engineer
16 5 (2.21%) Data Scientist
Libraries, Frameworks & Software Standards
1 146 (64.60%) Kafka
2 107 (47.35%) AngularJS
2 107 (47.35%) React
3 105 (46.46%) JSON
3 105 (46.46%) Spring
4 104 (46.02%) CSS
4 104 (46.02%) Node.js
4 104 (46.02%) RESTful
4 104 (46.02%) Twitter Bootstrap
5 101 (44.69%) Express
5 101 (44.69%) Redux
5 101 (44.69%) Tailwind CSS
6 98 (43.36%) WireMock
7 84 (37.17%) Play Framework
8 63 (27.88%) LAMP
8 63 (27.88%) LAPP Stack
9 58 (25.66%) RabbitMQ
10 15 (6.64%) Java EE
11 11 (4.87%) Web Services
12 9 (3.98%) TensorFlow
Miscellaneous
1 101 (44.69%) Linux Command Line
2 44 (19.47%) User Experience
3 43 (19.03%) Self-Motivation
4 13 (5.75%) Data Structures
5 7 (3.10%) Management Information System
6 5 (2.21%) Blog
6 5 (2.21%) Social Media
6 5 (2.21%) Virtual Team
7 4 (1.77%) Acoustics
8 3 (1.33%) Cloud Native
8 3 (1.33%) Data Centre
8 3 (1.33%) Media Streaming
8 3 (1.33%) Video\Live Streaming
9 1 (0.44%) Derivative
9 1 (0.44%) Enterprise Software
9 1 (0.44%) Equities
9 1 (0.44%) Equity Derivative
9 1 (0.44%) Fixed Income
9 1 (0.44%) Housing Association
Operating Systems
1 105 (46.46%) Linux
2 101 (44.69%) Unix
Processes & Methodologies
1 183 (80.97%) Agile Software Development
2 181 (80.09%) Software Engineering
3 128 (56.64%) Data Pipeline
4 120 (53.10%) DevOps
5 109 (48.23%) Computer Science
6 102 (45.13%) Microservices
7 101 (44.69%) Cross-Browser Testing
7 101 (44.69%) Digital Banking
7 101 (44.69%) Validation
8 99 (43.81%) Data Engineering
9 97 (42.92%) Analytics
10 96 (42.48%) ETL
11 93 (41.15%) Continuous Integration
11 93 (41.15%) Unit Testing
12 90 (39.82%) Data Munging
13 82 (36.28%) Artificial Intelligence
14 80 (35.40%) Data Science
15 76 (33.63%) Configuration Management
15 76 (33.63%) Project Management
16 74 (32.74%) Business Continuity
Programming Languages
1 183 (80.97%) Python
2 141 (62.39%) Java
3 112 (49.56%) SQL
4 103 (45.58%) Groovy
5 101 (44.69%) Shell Script
6 38 (16.81%) Scala
7 17 (7.52%) R
8 15 (6.64%) C++
9 5 (2.21%) C#
10 4 (1.77%) JavaScript
10 4 (1.77%) PL/SQL
11 3 (1.33%) Clojure
12 2 (0.88%) Go
13 1 (0.44%) Java 8
13 1 (0.44%) PowerShell
13 1 (0.44%) T-SQL
Qualifications
1 117 (51.77%) Degree
2 98 (43.36%) OCP
2 98 (43.36%) Oracle Certification
3 12 (5.31%) Computer Science Degree
4 4 (1.77%) AWS Certification
4 4 (1.77%) TOGAF Certification
5 3 (1.33%) Security Cleared
Quality Assurance & Compliance
1 1 (0.44%) QA
System Software
1 106 (46.90%) HDFS
2 85 (37.61%) Docker
3 76 (33.63%) VirtualBox
4 70 (30.97%) VMware Infrastructure
5 5 (2.21%) Apache ZooKeeper
6 4 (1.77%) Virtual Servers
7 1 (0.44%) Apache Flume
Systems Management
1 181 (80.09%) Ansible
2 90 (39.82%) Kubernetes
3 85 (37.61%) Terraform
4 76 (33.63%) Docker Swarm
5 26 (11.50%) Opscode Chef
6 3 (1.33%) Prometheus
Vendors
1 101 (44.69%) Okta
1 101 (44.69%) VMware
2 89 (39.38%) Google
3 81 (35.84%) Cloudera
3 81 (35.84%) Hortonworks
4 71 (31.42%) IBM
5 67 (29.65%) Informatica
5 67 (29.65%) Talend
6 11 (4.87%) Databricks
7 8 (3.54%) Microsoft
8 5 (2.21%) Dataiku
9 4 (1.77%) Oracle
10 3 (1.33%) Splunk
11 2 (0.88%) StreamSets