Period
to 21 July 2018

The following table provides summary statistics for contract job vacancies advertised in England with a requirement for Spark Streaming skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Spark Streaming over the 6 months to 21 July 2018 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark Streaming
UK > England
6 months to
21 Jul 2018
Same period 2017 Same period 2016
Rank 707 746 837
Rank change year-on-year +39 +91 +26
Contract jobs citing Spark Streaming 155 87 36
As % of all contract IT jobs advertised in England 0.16% 0.098% 0.035%
As % of the Libraries, Frameworks & Software Standards category 0.48% 0.30% 0.12%
Number of daily rates quoted 104 62 17
England median daily rate £525 £515 £575
Median daily rate % change year-on-year +1.94% -10.43% -4.17%
10th Percentile £425 £400 £486
90th Percentile £650 £650 £835
UK median daily rate £525 £515 £575
% change year-on-year +1.94% -10.43% -4.17%
Number of hourly rates quoted 0 1 0
England median hourly rate - £35.00 -
UK median hourly rate - £35.00 -

Spark Streaming is in the Software Libraries and Frameworks category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in England with a requirement for technical specification, industry standards, software libraries and framework skills.

All Software Libraries and Frameworks
England
Contract vacancies with a requirement for technical specification, industry standards, software libraries and framework skills 32,623 28,701 29,639
As % of all contract IT jobs advertised in England 34.33% 32.48% 29.16%
Number of daily rates quoted 23,185 19,902 20,933
England median daily rate £475 £450 £425
Median daily rate % change year-on-year +5.56% +5.88% +6.25%
10th Percentile £319 £308 £295
90th Percentile £631 £625 £600
UK median daily rate £475 £450 £425
% change year-on-year +5.56% +5.88% +6.25%
Number of hourly rates quoted 358 366 376
England median hourly rate £41.52 £40.75 £37.50
Median hourly rate % change year-on-year +1.88% +8.67% +3.81%
10th Percentile £15.25 £15.54 £12.63
90th Percentile £57.50 £53.75 £52.00
UK median hourly rate £41.00 £40.38 £37.50
% change year-on-year +1.54% +7.68% +2.74%

Spark Streaming
Job Vacancy Trend in England

Job postings citing Spark Streaming as a percentage of all IT jobs advertised in England.

Job vacancy trend for Spark Streaming in England

Spark Streaming
Contractor Daily Rate Trend in England

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Spark Streaming in England.

Contractor daily rate trend for Spark Streaming in England

Spark Streaming
Contractor Daily Rate Histogram in England

The daily rate distribution of IT jobs citing Spark Streaming in England over the 6 months to 21 July 2018.

Contractor daily rate histogram for Spark Streaming in England

Spark Streaming
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Spark Streaming within the England region over the 6 months to 21 July 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London -20 69 £575 +4.55% 8
South East +34 60 £525 +23.53% 3
North of England -25 18 £500 +8.23% 2
Yorkshire -29 14 £500 +8.23%
East Midlands - 8 £500 -
Midlands - 8 £500 -
North West - 4 £513 - 2
Spark Streaming
UK

For the 6 months to 21 July 2018, IT contractor jobs citing Spark Streaming also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region with a requirement for Spark Streaming.

1 153 (98.71%) Apache Spark
2 131 (84.52%) Kafka
3 123 (79.35%) Big Data
4 105 (67.74%) Hadoop
5 99 (63.87%) Java
6 93 (60.00%) Agile Software Development
7 85 (54.84%) Scala
8 64 (41.29%) Apache Hive
9 63 (40.65%) Apache
10 57 (36.77%) HBase
11 56 (36.13%) RESTful
12 53 (34.19%) Finance
13 51 (32.90%) Hadoop YARN
14 50 (32.26%) RabbitMQ
14 50 (32.26%) Linux
15 46 (29.68%) Apache ZooKeeper
16 45 (29.03%) Jenkins
17 44 (28.39%) NoSQL
17 44 (28.39%) TDD
18 42 (27.10%) Spring
19 41 (26.45%) Python
20 40 (25.81%) Unix
20 40 (25.81%) Cloudera
20 40 (25.81%) HDFS
21 38 (24.52%) Microservices
22 36 (23.23%) Impala
23 33 (21.29%) Apache Cassandra
24 29 (18.71%) BDD
24 29 (18.71%) Sqoop
25 28 (18.06%) SQL

Spark Streaming
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 153 (98.71%) Apache Spark
2 63 (40.65%) Apache
3 15 (9.68%) Elasticsearch
4 13 (8.39%) GemFire
5 12 (7.74%) Apache Pig
6 9 (5.81%) Apache Airflow
7 5 (3.23%) Apache Solr
8 2 (1.29%) Cloud Foundry
9 1 (0.65%) Luigi
Cloud Services
1 18 (11.61%) Amazon AWS
2 16 (10.32%) Google Cloud Platform
3 14 (9.03%) GitHub
4 9 (5.81%) Amazon EMR
5 8 (5.16%) Microsoft Azure
6 3 (1.94%) PaaS
7 1 (0.65%) Amazon EC2
7 1 (0.65%) Amazon Kinesis
7 1 (0.65%) Amazon S3
7 1 (0.65%) IaaS
7 1 (0.65%) SaaS
Communications & Networking
1 15 (9.68%) HTTP
1 15 (9.68%) Kerberos
2 6 (3.87%) DSL
3 2 (1.29%) DNS
3 2 (1.29%) SSL
Database & Business Intelligence
1 123 (79.35%) Big Data
2 105 (67.74%) Hadoop
3 64 (41.29%) Apache Hive
4 57 (36.77%) HBase
5 44 (28.39%) NoSQL
6 36 (23.23%) Impala
7 33 (21.29%) Apache Cassandra
8 29 (18.71%) Sqoop
9 25 (16.13%) Data Warehouse
10 23 (14.84%) MongoDB
11 21 (13.55%) SQL Server
12 18 (11.61%) Cloudera CDH
13 17 (10.97%) Relational Database
14 10 (6.45%) MySQL
14 10 (6.45%) Spark SQL
15 8 (5.16%) Data Lake
15 8 (5.16%) Oracle Database
15 8 (5.16%) Oracle Database 11g
15 8 (5.16%) Oracle Database 12c
16 7 (4.52%) OLTP
Development Applications
1 45 (29.03%) Jenkins
2 24 (15.48%) Git (software)
3 14 (9.03%) Team Foundation Server
4 13 (8.39%) JIRA
5 8 (5.16%) Oracle APEX
6 4 (2.58%) Gatling
6 4 (2.58%) TeamCity
6 4 (2.58%) Visual Studio Team System
7 2 (1.29%) Eclipse
7 2 (1.29%) GitLab
7 2 (1.29%) Gradle
7 2 (1.29%) IntelliJ
7 2 (1.29%) JUnit
7 2 (1.29%) Maven
7 2 (1.29%) Travis CI
8 1 (0.65%) Appium
8 1 (0.65%) Cucumber
8 1 (0.65%) Selenium
8 1 (0.65%) SonarQube
8 1 (0.65%) Sonatype Nexus
General
1 53 (34.19%) Finance
2 11 (7.10%) Banking
3 8 (5.16%) Telecoms
4 6 (3.87%) Games
5 4 (2.58%) Investment Banking
6 2 (1.29%) Retail Banking
7 1 (0.65%) Publishing
Job Titles
1 115 (74.19%) Developer
2 63 (40.65%) Big Data Developer
3 44 (28.39%) Java Developer
4 37 (23.87%) Apache Spark Developer
5 27 (17.42%) Data Engineer
6 20 (12.90%) Hadoop Developer
7 16 (10.32%) Scala Developer
8 15 (9.68%) Big Data Engineer
9 13 (8.39%) Java Hadoop Developer
10 10 (6.45%) Consultant
11 8 (5.16%) Architect
11 8 (5.16%) Big Data Consultant
11 8 (5.16%) Business Developer
11 8 (5.16%) Business Intelligence Developer
11 8 (5.16%) Data Architect
11 8 (5.16%) Data Consultant
11 8 (5.16%) Python Developer
12 7 (4.52%) Big Data Architect
13 5 (3.23%) DevOps Engineer
14 4 (2.58%) AWS Engineer
Libraries, Frameworks & Software Standards
1 131 (84.52%) Kafka
2 56 (36.13%) RESTful
3 50 (32.26%) RabbitMQ
4 42 (27.10%) Spring
5 28 (18.06%) J2EE
6 26 (16.77%) JSON
6 26 (16.77%) Middleware
6 26 (16.77%) Spring Boot
7 15 (9.68%) Akka
7 15 (9.68%) Play Framework
7 15 (9.68%) spray
7 15 (9.68%) Spring Integration
8 14 (9.03%) Web Services
9 5 (3.23%) Apache Avro
10 4 (2.58%) CSS
10 4 (2.58%) HTML
10 4 (2.58%) Node.js
11 3 (1.94%) ActiveMQ
11 3 (1.94%) Elastic Stack
11 3 (1.94%) TensorFlow
Miscellaneous
1 18 (11.61%) Greenfield Project
2 7 (4.52%) Analytical Skills
3 4 (2.58%) Self-Motivation
4 3 (1.94%) Algorithms
5 2 (1.29%) Computer Science
5 2 (1.29%) Data Structures
5 2 (1.29%) Fintech
5 2 (1.29%) Hedge funds
5 2 (1.29%) Linux Command Line
5 2 (1.29%) Virtual Team
6 1 (0.65%) Data Centre
6 1 (0.65%) Taxonomies
Operating Systems
1 50 (32.26%) Linux
2 40 (25.81%) Unix
Processes & Methodologies
1 93 (60.00%) Agile Software Development
2 44 (28.39%) TDD
3 38 (24.52%) Microservices
4 29 (18.71%) BDD
5 26 (16.77%) Continuous Integration
5 26 (16.77%) DevOps
6 23 (14.84%) Data Modelling
6 23 (14.84%) ETL
7 22 (14.19%) Business Intelligence
7 22 (14.19%) MapReduce
8 18 (11.61%) Machine Learning
9 16 (10.32%) Pair Programming
10 15 (9.68%) Analytics
10 15 (9.68%) Coaching
11 13 (8.39%) Decision-Making
11 13 (8.39%) Kalman Filter
11 13 (8.39%) Unit Testing
12 11 (7.10%) System Testing
13 10 (6.45%) Scrum
14 7 (4.52%) Refactoring
Programming Languages
1 99 (63.87%) Java
2 85 (54.84%) Scala
3 41 (26.45%) Python
4 28 (18.06%) SQL
5 17 (10.97%) Shell Script
6 6 (3.87%) HiveQL
7 5 (3.23%) C++
7 5 (3.23%) Java 8
8 4 (2.58%) C#
8 4 (2.58%) JavaScript
8 4 (2.58%) PHP
9 3 (1.94%) Bash Shell
10 2 (1.29%) Ruby
11 1 (0.65%) Groovy
11 1 (0.65%) Perl
Qualifications
1 15 (9.68%) Degree
2 8 (5.16%) Security Cleared
3 7 (4.52%) SC Cleared
4 2 (1.29%) Computer Science Degree
Quality Assurance & Compliance
1 10 (6.45%) GCP
2 2 (1.29%) QA
3 1 (0.65%) Data Quality
3 1 (0.65%) GDPR
3 1 (0.65%) PSD2
System Software
1 46 (29.68%) Apache ZooKeeper
2 40 (25.81%) HDFS
3 21 (13.55%) Docker
4 16 (10.32%) Apache Flume
Systems Management
1 51 (32.90%) Hadoop YARN
2 12 (7.74%) Oozie
3 9 (5.81%) Ansible
4 7 (4.52%) Kubernetes
5 3 (1.94%) Kibana
6 2 (1.29%) Prometheus
6 2 (1.29%) Terraform
7 1 (0.65%) logstash
7 1 (0.65%) Opscode Chef
7 1 (0.65%) Packer
7 1 (0.65%) Puppet
Vendors
1 40 (25.81%) Cloudera
2 27 (17.42%) Hortonworks
3 16 (10.32%) Google
4 14 (9.03%) Informatica
4 14 (9.03%) Talend
5 13 (8.39%) Oracle
6 7 (4.52%) MapR
7 4 (2.58%) Databricks
7 4 (2.58%) Tableau
8 2 (1.29%) Ab Initio
9 1 (0.65%) SaltStack
9 1 (0.65%) SAP