Period
to 17 December 2017

The following table provides summary statistics for contract job vacancies advertised in London with a requirement for Spark Streaming skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Spark Streaming over the 6 months to 17 December 2017 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark Streaming
England > London
6 months to
17 Dec 2017
Same period 2016 Same period 2015
Rank 580 537 594
Rank change year-on-year -43 +57 +8
Contract jobs citing Spark Streaming 57 79 2
As % of all contract IT jobs advertised in London 0.12% 0.18% 0.004%
As % of the Libraries, Frameworks & Software Standards category 0.33% 0.52% 0.013%
Number of daily rates quoted 42 55 0
London median daily rate £525 £550 -
Median daily rate % change year-on-year -4.55% - -
England median daily rate £515 £550 £501
% change year-on-year -6.36% +9.89% -9.00%
Number of hourly rates quoted 0 0 0
London median hourly rate - - -
England median hourly rate £35.00 - -

Spark Streaming is in the Software Libraries and Frameworks category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in London with a requirement for technical specification, industry standards, software libraries and framework skills.

All Software Libraries and Frameworks
London
Contract vacancies with a requirement for technical specification, industry standards, software libraries and framework skills 17365 15273 14949
As % of all contract IT jobs advertised in London 36.77% 34.60% 32.42%
Number of daily rates quoted 13314 11397 11163
London median daily rate £500 £471 £450
Median daily rate % change year-on-year +6.27% +4.56% +5.88%
10th Percentile £350 £321 £310
90th Percentile £650 £625 £625
England median daily rate £475 £425 £415
% change year-on-year +11.76% +2.41% +3.75%
Number of hourly rates quoted 99 73 92
London median hourly rate £29.00 £42.25 £37.75
Median hourly rate % change year-on-year -31.36% +11.92% +7.86%
10th Percentile £19.00 £19.23 £14.25
90th Percentile £47.83 £57.38 £54.75
England median hourly rate £40.00 £40.00 £37.50
% change year-on-year - +6.67% +7.14%

Spark Streaming
Job Vacancy Trend in London

Job postings citing Spark Streaming as a percentage of all IT jobs advertised in London.

Job vacancy trend for Spark Streaming in London

Spark Streaming
Contractor Daily Rate Trend in London

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Spark Streaming in London.

Contractor daily rate trend for Spark Streaming in London

Spark Streaming
Contractor Daily Rate Histogram in London

The daily rate distribution of IT jobs citing Spark Streaming in London over the 6 months to 17 December 2017.

Contractor daily rate histogram for Spark Streaming in London

Spark Streaming
Contract Job Locations in London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Spark Streaming within the London region over the 6 months to 17 December 2017. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
West London -8 8 £525 +9.38%
South London - 6 £544 -
Central London -10 4 £450 -18.18% 1
Spark Streaming
England

For the 6 months to 17 December 2017, IT contractor jobs citing Spark Streaming also mentioned the following skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all contract ads across the London region with a requirement for Spark Streaming.

1 57 (100.00%) Apache Spark
2 42 (73.68%) Kafka
3 38 (66.67%) Big Data
4 31 (54.39%) Apache Cassandra
5 28 (49.12%) Scala
5 28 (49.12%) Agile Software Development
5 28 (49.12%) Java
6 27 (47.37%) Hadoop
7 18 (31.58%) RESTful
7 18 (31.58%) NoSQL
7 18 (31.58%) Apache Hive
7 18 (31.58%) HBase
8 17 (29.82%) Microservices
8 17 (29.82%) HDFS
8 17 (29.82%) MongoDB
9 16 (28.07%) Machine Learning
9 16 (28.07%) TDD
9 16 (28.07%) Pair Programming
9 16 (28.07%) Scrum
10 15 (26.32%) ETL
11 14 (24.56%) Python
12 13 (22.81%) Akka
12 13 (22.81%) Apache Flume
12 13 (22.81%) HTTP
13 12 (21.05%) Open Source
13 12 (21.05%) Amazon AWS
14 11 (19.30%) Elasticsearch
14 11 (19.30%) SQL
14 11 (19.30%) Algorithms
15 10 (17.54%) Hadoop YARN

Spark Streaming
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 57 (100.00%) Apache Spark
2 11 (19.30%) Elasticsearch
3 10 (17.54%) Apache
4 6 (10.53%) Apache Pig
5 3 (5.26%) Tomcat
6 1 (1.75%) Jupyter
6 1 (1.75%) OpenStack
Applications
1 1 (1.75%) MS Visio
Cloud Services
1 12 (21.05%) Amazon AWS
2 6 (10.53%) GitHub
3 3 (5.26%) SaaS
4 1 (1.75%) Amazon EC2
4 1 (1.75%) Amazon Kinesis
4 1 (1.75%) Amazon S3
4 1 (1.75%) Amazon SQS
Communications & Networking
1 13 (22.81%) HTTP
2 1 (1.75%) Internet
2 1 (1.75%) Kerberos
2 1 (1.75%) LAN
2 1 (1.75%) SSL
2 1 (1.75%) VPN
2 1 (1.75%) WAN
Database & Business Intelligence
1 38 (66.67%) Big Data
2 31 (54.39%) Apache Cassandra
3 27 (47.37%) Hadoop
4 18 (31.58%) Apache Hive
4 18 (31.58%) HBase
4 18 (31.58%) NoSQL
5 17 (29.82%) MongoDB
6 9 (15.79%) Impala
7 8 (14.04%) Flink
8 6 (10.53%) PostgreSQL
9 5 (8.77%) Amazon Redshift
10 3 (5.26%) Mahout
10 3 (5.26%) Sqoop
11 2 (3.51%) Parquet
11 2 (3.51%) Spark SQL
12 1 (1.75%) Apache Kudu
12 1 (1.75%) Apache Zeppelin
12 1 (1.75%) Data Lake
12 1 (1.75%) Oracle ODI
12 1 (1.75%) QlikView
Development Applications
1 5 (8.77%) Git (software)
1 5 (8.77%) Jenkins
2 4 (7.02%) Maven
3 1 (1.75%) Apache Continuum
3 1 (1.75%) ERwin
3 1 (1.75%) GoCD
3 1 (1.75%) Gradle
3 1 (1.75%) JIRA
3 1 (1.75%) Subversion
3 1 (1.75%) Talend Open Studio
General
1 10 (17.54%) Banking
2 9 (15.79%) Finance
3 5 (8.77%) Marketing
4 4 (7.02%) Publishing
4 4 (7.02%) Retail
5 3 (5.26%) Telecoms
6 2 (3.51%) Electronics
6 2 (3.51%) Manufacturing
7 1 (1.75%) Pharmaceutical
Job Titles
1 36 (63.16%) Developer
2 18 (31.58%) Data Engineer
3 17 (29.82%) Scala Developer
4 14 (24.56%) Big Data Developer
5 8 (14.04%) Big Data Engineer
6 6 (10.53%) Apache Spark Developer
6 6 (10.53%) Architect
7 4 (7.02%) Data Architect
7 4 (7.02%) Hadoop Developer
8 3 (5.26%) Big Data Architect
8 3 (5.26%) Elasticsearch Developer
8 3 (5.26%) Elasticsearch Engineer
8 3 (5.26%) Hadoop Architect
8 3 (5.26%) Hadoop Engineer
8 3 (5.26%) Java Developer
9 2 (3.51%) Java Hadoop Developer
10 1 (1.75%) Big Data Solutions Architect
10 1 (1.75%) Digital Developer
10 1 (1.75%) IoT Developer
10 1 (1.75%) Senior Data Engineer
Libraries, Frameworks & Software Standards
1 42 (73.68%) Kafka
2 18 (31.58%) RESTful
3 13 (22.81%) Akka
4 10 (17.54%) Elastic Stack
5 9 (15.79%) Play Framework
5 9 (15.79%) spray
6 8 (14.04%) REST
6 8 (14.04%) Web Services
7 7 (12.28%) XML
8 6 (10.53%) Ajax
8 6 (10.53%) jQuery
8 6 (10.53%) JSON
8 6 (10.53%) SOAP
9 5 (8.77%) Spring
10 4 (7.02%) Node.js
10 4 (7.02%) RabbitMQ
11 3 (5.26%) Apache Avro
11 3 (5.26%) J2EE
12 1 (1.75%) PySpark
12 1 (1.75%) Sensu
Miscellaneous
1 11 (19.30%) Algorithms
2 4 (7.02%) Computer Science
3 3 (5.26%) Data Structures
3 3 (5.26%) Greenfield Project
4 2 (3.51%) Clustering
5 1 (1.75%) Analytical Skills
5 1 (1.75%) Internet of Things
5 1 (1.75%) Smart Meter
Operating Systems
1 7 (12.28%) Linux
2 1 (1.75%) Red Hat Enterprise Linux
2 1 (1.75%) Unix
Processes & Methodologies
1 28 (49.12%) Agile Software Development
2 17 (29.82%) Microservices
3 16 (28.07%) Machine Learning
3 16 (28.07%) Pair Programming
3 16 (28.07%) Scrum
3 16 (28.07%) TDD
4 15 (26.32%) ETL
5 12 (21.05%) Open Source
6 10 (17.54%) MapReduce
7 9 (15.79%) Analytics
7 9 (15.79%) Coaching
7 9 (15.79%) Continuous Integration
7 9 (15.79%) Kanban
8 8 (14.04%) DevOps
9 7 (12.28%) Data Engineering
9 7 (12.28%) Migration
10 6 (10.53%) Enterprise Search
10 6 (10.53%) OO
10 6 (10.53%) Test Automation
11 5 (8.77%) Digital Marketing
Programming Languages
1 28 (49.12%) Java
1 28 (49.12%) Scala
2 14 (24.56%) Python
3 11 (19.30%) SQL
4 6 (10.53%) JavaScript
5 3 (5.26%) C#
6 2 (3.51%) Shell Script
7 1 (1.75%) Bash Shell
7 1 (1.75%) C++
7 1 (1.75%) Perl
Qualifications
1 5 (8.77%) Security Cleared
2 3 (5.26%) SC Cleared
3 1 (1.75%) DV Cleared
System Software
1 17 (29.82%) HDFS
2 13 (22.81%) Apache Flume
3 9 (15.79%) Docker
4 1 (1.75%) Apache ZooKeeper
Systems Management
1 10 (17.54%) Hadoop YARN
1 10 (17.54%) Kibana
1 10 (17.54%) logstash
2 4 (7.02%) Puppet
3 3 (5.26%) Ansible
4 1 (1.75%) Kubernetes
4 1 (1.75%) Nagios
4 1 (1.75%) Opscode Chef
4 1 (1.75%) Prometheus
4 1 (1.75%) Rancher
Vendors
1 6 (10.53%) Hortonworks
2 5 (8.77%) Cloudera
2 5 (8.77%) MapR
3 3 (5.26%) Teradata
4 2 (3.51%) Google
4 2 (3.51%) Oracle
4 2 (3.51%) SAP
5 1 (1.75%) Cognos
5 1 (1.75%) Informatica
5 1 (1.75%) Red Hat
5 1 (1.75%) Tableau
5 1 (1.75%) Talend