Period
to 13 December 2019

The following table provides summary statistics for contract job vacancies advertised in England with a requirement for Spark Streaming skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Spark Streaming over the 6 months to 13 December 2019 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark Streaming
UK > England
6 months to
13 Dec 2019
Same period 2018 Same period 2017
Rank 686 727 755
Rank change year-on-year +41 +28 +23
Contract jobs citing Spark Streaming 61 143 122
As % of all contract IT jobs advertised in England 0.084% 0.15% 0.13%
As % of the Libraries, Frameworks & Software Standards category 0.27% 0.42% 0.39%
Number of daily rates quoted 43 100 72
England median daily rate £600 £575 £515
Median daily rate % change year-on-year +4.35% +11.65% -6.36%
10th Percentile £451 £463 £419
90th Percentile £698 £775 £591
UK median daily rate £600 £575 £520
% change year-on-year +4.35% +10.58% -5.45%
Number of hourly rates quoted 0 0 1
England median hourly rate - - £35.00
UK median hourly rate - - £35.00

Spark Streaming is in the Software Libraries and Frameworks category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in England with a requirement for technical specification, industry standards, software libraries and framework skills.

All Software Libraries and Frameworks
England
Contract vacancies with a requirement for technical specification, industry standards, software libraries and framework skills 22,234 33,737 30,993
As % of all contract IT jobs advertised in England 30.56% 35.09% 32.59%
Number of daily rates quoted 15,539 24,077 22,087
England median daily rate £500 £480 £470
Median daily rate % change year-on-year +4.17% +2.13% +10.59%
10th Percentile £338 £325 £313
90th Percentile £650 £638 £625
UK median daily rate £493 £475 £450
% change year-on-year +3.68% +5.56% +5.88%
Number of hourly rates quoted 237 284 354
England median hourly rate £42.50 £41.50 £40.00
Median hourly rate % change year-on-year +2.41% +3.75% -
10th Percentile £17.23 £18.71 £18.00
90th Percentile £78.75 £57.50 £52.35
UK median hourly rate £42.50 £41.50 £40.00
% change year-on-year +2.41% +3.75% -

Spark Streaming
Job Vacancy Trend in England

Job postings citing Spark Streaming as a percentage of all IT jobs advertised in England.

Job vacancy trend for Spark Streaming in England

Spark Streaming
Contractor Daily Rate Trend in England

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Spark Streaming in England.

Contractor daily rate trend for Spark Streaming in England

Spark Streaming
Contractor Daily Rate Histogram in England

The daily rate distribution of IT jobs citing Spark Streaming in England over the 6 months to 13 December 2019.

Contractor daily rate histogram for Spark Streaming in England

Spark Streaming
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Spark Streaming within the England region over the 6 months to 13 December 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London +77 60 £600 -9.43% 11
South West +8 1 £750 +36.36%
Spark Streaming
UK

For the 6 months to 13 December 2019, IT contractor jobs citing Spark Streaming also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region with a requirement for Spark Streaming.

1 61 (100.00%) Apache Spark
2 44 (72.13%) Hadoop
2 44 (72.13%) Kafka
3 39 (63.93%) Python
4 35 (57.38%) SQL
5 34 (55.74%) Scala
6 32 (52.46%) Analytics
7 31 (50.82%) AWS
8 27 (44.26%) Java
9 25 (40.98%) Big Data
9 25 (40.98%) Kubernetes
10 24 (39.34%) Docker
11 23 (37.70%) Apache Hive
12 20 (32.79%) PostgreSQL
12 20 (32.79%) Agile Software Development
13 19 (31.15%) Apache Cassandra
14 18 (29.51%) NoSQL
14 18 (29.51%) Jenkins
15 17 (27.87%) Data Science
15 17 (27.87%) Azure
16 15 (24.59%) ETL
16 15 (24.59%) Apache Airflow
17 14 (22.95%) PySpark
17 14 (22.95%) Microservices
18 13 (21.31%) Ansible
18 13 (21.31%) Amazon Redshift
18 13 (21.31%) GCP
18 13 (21.31%) Apache
19 12 (19.67%) IBM
19 12 (19.67%) Terraform

Spark Streaming
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 61 (100.00%) Apache Spark
2 15 (24.59%) Apache Airflow
3 13 (21.31%) Apache
4 9 (14.75%) SAS
5 8 (13.11%) Luigi
6 5 (8.20%) Elasticsearch
7 2 (3.28%) Apache Solr
8 1 (1.64%) Oracle Workflow
Cloud Services
1 31 (50.82%) AWS
2 17 (27.87%) Azure
3 13 (21.31%) GCP
4 9 (14.75%) Amazon EKS
5 6 (9.84%) Amazon EMR
6 4 (6.56%) Amazon EC2
7 3 (4.92%) AWS IoT
7 3 (4.92%) GitHub
8 1 (1.64%) Amazon S3
8 1 (1.64%) OpenShift
Communications & Networking
1 1 (1.64%) Kerberos
Database & Business Intelligence
1 44 (72.13%) Hadoop
2 25 (40.98%) Big Data
3 23 (37.70%) Apache Hive
4 20 (32.79%) PostgreSQL
5 19 (31.15%) Apache Cassandra
6 18 (29.51%) NoSQL
7 13 (21.31%) Amazon Redshift
8 12 (19.67%) Relational Database
9 11 (18.03%) Flink
9 11 (18.03%) Neo4j
9 11 (18.03%) Spark SQL
10 9 (14.75%) Amazon Athena
10 9 (14.75%) SAS Enterprise Guide
11 7 (11.48%) Azkaban
11 7 (11.48%) HBase
12 6 (9.84%) Data Lake
13 5 (8.20%) MongoDB
13 5 (8.20%) Sqoop
14 4 (6.56%) Amazon RDS
14 4 (6.56%) Bigtable
Development Applications
1 18 (29.51%) Jenkins
2 9 (14.75%) Atlassian Bamboo
2 9 (14.75%) Git (software)
2 9 (14.75%) Vagrant
3 6 (9.84%) Maven
4 3 (4.92%) JIRA
General
1 8 (13.11%) Finance
2 6 (9.84%) Banking
3 4 (6.56%) Legal
4 3 (4.92%) Publishing
5 2 (3.28%) Investment Banking
6 1 (1.64%) Public Sector
6 1 (1.64%) Telecoms
Job Titles
1 31 (50.82%) Data Engineer
2 13 (21.31%) Developer
3 8 (13.11%) AWS Engineer
3 8 (13.11%) Big Data Engineer
3 8 (13.11%) Java Engineer
4 7 (11.48%) Architect
4 7 (11.48%) ETL Developer
5 5 (8.20%) Platform Architect
5 5 (8.20%) Python Engineer
5 5 (8.20%) Senior Data Engineer
6 4 (6.56%) Data Architect
7 3 (4.92%) Big Data Architect
7 3 (4.92%) Java Developer
7 3 (4.92%) Kubernetes Engineer
7 3 (4.92%) Lead Architect
8 2 (3.28%) Database Developer
8 2 (3.28%) Hadoop Architect
8 2 (3.28%) Senior Java Engineer
9 1 (1.64%) Informatica Developer
9 1 (1.64%) Solutions Architect
Libraries, Frameworks & Software Standards
1 44 (72.13%) Kafka
2 14 (22.95%) PySpark
3 11 (18.03%) Spring
4 10 (16.39%) Java Concurrency
5 8 (13.11%) Lambda Expressions
6 6 (9.84%) Akka
6 6 (9.84%) REST
6 6 (9.84%) RESTful
6 6 (9.84%) Web Services
7 4 (6.56%) Apache Beam
8 3 (4.92%) AngularJS
8 3 (4.92%) Backbone.js
8 3 (4.92%) Ember.js
8 3 (4.92%) jQuery
8 3 (4.92%) Node.js
8 3 (4.92%) React
8 3 (4.92%) Twitter Bootstrap
9 2 (3.28%) Lucene
10 1 (1.64%) JSON
10 1 (1.64%) XML
Miscellaneous
1 7 (11.48%) Data Structures
2 4 (6.56%) Public Cloud
3 3 (4.92%) Tandem
4 1 (1.64%) Algorithms
4 1 (1.64%) Analytical Skills
4 1 (1.64%) Self-Motivation
Operating Systems
1 9 (14.75%) Linux
2 6 (9.84%) Unix
3 3 (4.92%) Windows
Processes & Methodologies
1 32 (52.46%) Analytics
2 20 (32.79%) Agile Software Development
3 17 (27.87%) Data Science
4 15 (24.59%) ETL
5 14 (22.95%) Microservices
6 10 (16.39%) OO
6 10 (16.39%) Software Engineering
7 9 (14.75%) Configuration Management
7 9 (14.75%) Continuous Integration
7 9 (14.75%) Data Munging
7 9 (14.75%) DevOps
7 9 (14.75%) Translating Business Requirements
7 9 (14.75%) Unit Testing
8 8 (13.11%) Pair Programming
8 8 (13.11%) Use Case
8 8 (13.11%) Workflow
8 8 (13.11%) Workload Management
9 7 (11.48%) Artificial Intelligence
9 7 (11.48%) Root Cause Analysis
9 7 (11.48%) Workflow Management
Programming Languages
1 39 (63.93%) Python
2 35 (57.38%) SQL
3 34 (55.74%) Scala
4 27 (44.26%) Java
5 10 (16.39%) C++
6 6 (9.84%) AWK
6 6 (9.84%) sed
7 4 (6.56%) Clojure
8 3 (4.92%) C
8 3 (4.92%) Ruby
9 2 (3.28%) R
10 1 (1.64%) Bash Shell
10 1 (1.64%) PL/SQL
Qualifications
1 8 (13.11%) Degree
2 5 (8.20%) Computer Science Degree
2 5 (8.20%) Master's Degree
3 1 (1.64%) TOGAF Certification
Quality Assurance & Compliance
1 1 (1.64%) QA
System Software
1 24 (39.34%) Docker
2 9 (14.75%) VirtualBox
3 4 (6.56%) HDFS
4 2 (3.28%) Apache Flume
Systems Management
1 25 (40.98%) Kubernetes
2 13 (21.31%) Ansible
3 12 (19.67%) Terraform
4 9 (14.75%) Docker Swarm
5 4 (6.56%) Hadoop YARN
6 3 (4.92%) Graphite
6 3 (4.92%) Oozie
7 1 (1.64%) Kibana
7 1 (1.64%) logstash
Vendors
1 12 (19.67%) IBM
2 11 (18.03%) Cloudera
2 11 (18.03%) Talend
3 10 (16.39%) Informatica
4 9 (14.75%) Datactics
4 9 (14.75%) Google
4 9 (14.75%) Hortonworks
4 9 (14.75%) Palantir
5 4 (6.56%) Confluent
6 2 (3.28%) Ab Initio
6 2 (3.28%) MapR
7 1 (1.64%) Microsoft
7 1 (1.64%) Oracle
7 1 (1.64%) ParAccel
7 1 (1.64%) Solace