Period
to 11 December 2018

The following table provides summary statistics for contract job vacancies advertised in the North West with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 11 December 2018 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
England > North West
6 months to
11 Dec 2018
Same period 2017 Same period 2016
Rank 182 162 204
Rank change year-on-year -20 +42 +33
Contract jobs citing Apache Spark 41 41 16
As % of all contract IT jobs advertised in the North West 0.68% 0.72% 0.25%
As % of the Application Platforms category 4.63% 4.71% 1.50%
Number of daily rates quoted 33 39 13
North West median daily rate £550 £550 £525
Median daily rate % change year-on-year - +4.76% -8.70%
10th Percentile £458 £400 £423
90th Percentile £765 £605 £600
England median daily rate £550 £550 £550
Number of hourly rates quoted 0 0 0
North West median hourly rate - - -
England median hourly rate £50.00 £23.42 £60.25
% change year-on-year +113.49% -61.13% +79.58%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in the North West with a requirement for application platform skills.

All Application Platform Skills
North West
Contract vacancies with a requirement for application platform skills 886 870 1,064
As % of all contract IT jobs advertised in the North West 14.72% 15.31% 16.37%
Number of daily rates quoted 548 555 681
North West median daily rate £400 £400 £388
Median daily rate % change year-on-year - +3.23% -3.73%
10th Percentile £173 £241 £263
90th Percentile £647 £600 £556
England median daily rate £475 £450 £425
% change year-on-year +5.56% +5.88% +6.25%
Number of hourly rates quoted 41 30 33
North West median hourly rate £20.50 £22.50 £15.00
Median hourly rate % change year-on-year -8.89% +50.00% -14.29%
10th Percentile £12.83 £11.00 £11.25
90th Percentile £55.00 £50.75 £20.73
England median hourly rate £21.50 £20.00 £20.00
% change year-on-year +7.50% - +5.26%

Apache Spark
Job Vacancy Trend in the North West

Job postings citing Apache Spark as a percentage of all IT jobs advertised in the North West.

Job vacancy trend for Apache Spark in the North West

Apache Spark
Contractor Daily Rate Trend in the North West

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Apache Spark in the North West.

Contractor daily rate trend for Apache Spark in the North West

Apache Spark
Contractor Daily Rate Histogram in the North West

The daily rate distribution of IT jobs citing Apache Spark in the North West over the 6 months to 11 December 2018.

Contractor daily rate histogram for Apache Spark in the North West

Apache Spark
Contract Job Locations in the North West

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the North West region over the 6 months to 11 December 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
Lancashire -11 15 £600 +50.00%
Manchester -34 15 £525 -4.55% 6
Cheshire +6 8 £538 - 1
Merseyside -6 2 £563 -6.25% 1
Apache Spark
England

For the 6 months to 11 December 2018, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the North West region with a requirement for Apache Spark.

1 33 (80.49%) Big Data
2 29 (70.73%) Hadoop
3 27 (65.85%) Apache Hive
4 22 (53.66%) Java
5 18 (43.90%) HBase
5 18 (43.90%) Cloudera
6 17 (41.46%) Kafka
7 14 (34.15%) Scala
7 14 (34.15%) HDFS
8 12 (29.27%) Sqoop
8 12 (29.27%) Python
9 11 (26.83%) Agile Software Development
10 10 (24.39%) ETL
10 10 (24.39%) SQL
10 10 (24.39%) Amazon AWS
10 10 (24.39%) Impala
11 9 (21.95%) Microsoft Azure
11 9 (21.95%) Hortonworks
12 8 (19.51%) Relational Database
12 8 (19.51%) NoSQL
12 8 (19.51%) Data Modelling
13 7 (17.07%) MongoDB
13 7 (17.07%) Spark Streaming
13 7 (17.07%) DevOps
13 7 (17.07%) Data Lake
14 6 (14.63%) Jenkins
14 6 (14.63%) Oozie
14 6 (14.63%) Talend
14 6 (14.63%) Data Architecture
14 6 (14.63%) Data Warehouse

Apache Spark
Co-occurring IT Skills in the North West by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 6 (14.63%) Apache
2 5 (12.20%) Elasticsearch
3 3 (7.32%) Apache Pig
4 1 (2.44%) Apache Solr
4 1 (2.44%) Confluence
4 1 (2.44%) Tomcat
Cloud Services
1 10 (24.39%) Amazon AWS
2 9 (21.95%) Microsoft Azure
3 4 (9.76%) Amazon EC2
3 4 (9.76%) Amazon EMR
3 4 (9.76%) Amazon S3
4 3 (7.32%) AWS CloudFormation
5 2 (4.88%) Amazon Kinesis
5 2 (4.88%) Cloud Computing
5 2 (4.88%) Google Cloud Platform
5 2 (4.88%) Virtual Private Cloud
6 1 (2.44%) CloudBees
Communications & Networking
1 2 (4.88%) Kerberos
2 1 (2.44%) DHCP
2 1 (2.44%) DNS
2 1 (2.44%) FTP
2 1 (2.44%) HTTP
2 1 (2.44%) IPsec
2 1 (2.44%) TCP/IP
Database & Business Intelligence
1 33 (80.49%) Big Data
2 29 (70.73%) Hadoop
3 27 (65.85%) Apache Hive
4 18 (43.90%) HBase
5 12 (29.27%) Sqoop
6 10 (24.39%) Impala
7 8 (19.51%) NoSQL
7 8 (19.51%) Relational Database
8 7 (17.07%) Data Lake
8 7 (17.07%) MongoDB
9 6 (14.63%) Data Warehouse
10 5 (12.20%) Apache Kudu
10 5 (12.20%) Parquet
10 5 (12.20%) PostgreSQL
10 5 (12.20%) Power BI
11 4 (9.76%) Apache Cassandra
11 4 (9.76%) Cloudera CDH
12 3 (7.32%) DB2
12 3 (7.32%) SQL Server
13 2 (4.88%) Pentaho
Development Applications
1 6 (14.63%) Jenkins
2 3 (7.32%) Git (software)
3 2 (4.88%) Atlassian Bamboo
3 2 (4.88%) Hudson
3 2 (4.88%) JIRA
3 2 (4.88%) RStudio
3 2 (4.88%) SonarQube
3 2 (4.88%) Subversion
3 2 (4.88%) Team Foundation Server
4 1 (2.44%) ERwin
4 1 (2.44%) JIRA Agile
4 1 (2.44%) Selenium
4 1 (2.44%) Toad
General
1 1 (2.44%) Automotive
Job Titles
1 17 (41.46%) Data Engineer
2 15 (36.59%) Developer
3 9 (21.95%) Big Data Engineer
4 7 (17.07%) Big Data Developer
4 7 (17.07%) Java Developer
5 4 (9.76%) Apache Spark Developer
5 4 (9.76%) Architect
5 4 (9.76%) Cloud Architect
5 4 (9.76%) Cloud Engineer
5 4 (9.76%) Hadoop Developer
5 4 (9.76%) Hadoop Engineer
6 3 (7.32%) AWS Engineer
6 3 (7.32%) ETL Developer
6 3 (7.32%) Hadoop Architect
6 3 (7.32%) Java Engineer
6 3 (7.32%) Java Hadoop Developer
7 2 (4.88%) Big Data Architect
7 2 (4.88%) Cloud Developer
7 2 (4.88%) Data Architect
7 2 (4.88%) Pentaho Developer
Libraries, Frameworks & Software Standards
1 17 (41.46%) Kafka
2 7 (17.07%) Spark Streaming
3 5 (12.20%) Apache Avro
4 4 (9.76%) Web Services
5 3 (7.32%) J2EE
5 3 (7.32%) Node.js
6 2 (4.88%) AngularJS
6 2 (4.88%) React
7 1 (2.44%) Dropwizard
7 1 (2.44%) Java EE
7 1 (2.44%) jQuery
7 1 (2.44%) Natural Language Processing
7 1 (2.44%) Oracle GoldenGate
7 1 (2.44%) Pandas
7 1 (2.44%) RabbitMQ
7 1 (2.44%) REST
7 1 (2.44%) scikit-learn
7 1 (2.44%) SciPy
7 1 (2.44%) Spring
7 1 (2.44%) XML
Miscellaneous
1 2 (4.88%) Algorithms
1 2 (4.88%) Self-Motivation
2 1 (2.44%) Data Structures
2 1 (2.44%) Enterprise Software
Operating Systems
1 4 (9.76%) Linux
1 4 (9.76%) Unix
2 2 (4.88%) Windows
3 1 (2.44%) CentOS
3 1 (2.44%) Red Hat Enterprise Linux
3 1 (2.44%) Solaris
Processes & Methodologies
1 11 (26.83%) Agile Software Development
2 10 (24.39%) ETL
3 8 (19.51%) Data Modelling
4 7 (17.07%) DevOps
5 6 (14.63%) Data Architecture
6 5 (12.20%) Business Intelligence
6 5 (12.20%) Continuous Integration
7 4 (9.76%) Analytics
7 4 (9.76%) Data Engineering
7 4 (9.76%) Kimball Methodology
7 4 (9.76%) Migration
7 4 (9.76%) Open Source
7 4 (9.76%) Scrum
7 4 (9.76%) TDD
8 3 (7.32%) BDD
8 3 (7.32%) Continuous Delivery
8 3 (7.32%) Data Analysis
8 3 (7.32%) Data Reconciliation
8 3 (7.32%) ETL Development
8 3 (7.32%) Test Automation
Programming Languages
1 22 (53.66%) Java
2 14 (34.15%) Scala
3 12 (29.27%) Python
4 10 (24.39%) SQL
5 3 (7.32%) Java 8
5 3 (7.32%) JavaScript
6 2 (4.88%) PL/SQL
6 2 (4.88%) Ruby
6 2 (4.88%) Shell Script
7 1 (2.44%) Bash Shell
7 1 (2.44%) R
Qualifications
1 3 (7.32%) Degree
2 2 (4.88%) AWS Certification
2 2 (4.88%) CCA
2 2 (4.88%) Computer Science Degree
3 1 (2.44%) BPSS Clearance
3 1 (2.44%) Security Cleared
System Software
1 14 (34.15%) HDFS
2 5 (12.20%) Docker
3 1 (2.44%) Apache Flume
3 1 (2.44%) Apache ZooKeeper
3 1 (2.44%) Oracle RAC
Systems Management
1 6 (14.63%) Oozie
2 4 (9.76%) Terraform
3 3 (7.32%) Ansible
3 3 (7.32%) Opscode Chef
3 3 (7.32%) Puppet
3 3 (7.32%) Salt
4 2 (4.88%) Docker Swarm
4 2 (4.88%) Kubernetes
5 1 (2.44%) BMC PATROL
5 1 (2.44%) Ganglia
5 1 (2.44%) Nagios
5 1 (2.44%) NetBackup
Vendors
1 18 (43.90%) Cloudera
2 9 (21.95%) Hortonworks
3 6 (14.63%) Talend
4 5 (12.20%) Oracle
4 5 (12.20%) Tableau
5 3 (7.32%) MapR
6 2 (4.88%) Google
6 2 (4.88%) Teradata
7 1 (2.44%) Atlassian
7 1 (2.44%) BMC
7 1 (2.44%) Informatica
7 1 (2.44%) Red Hat