Period
to 14 August 2018

The following table provides summary statistics for permanent job vacancies advertised in the City of London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 14 August 2018 with a comparison to the same period in the previous 2 years.

Apache Spark
Central London > City of London
6 months to
14 Aug 2018
Same period 2017 Same period 2016
Rank 136 186 215
Rank change year-on-year +50 +29 +166
Permanent jobs citing Apache Spark 251 151 174
As % of all permanent IT jobs advertised in the City of London 2.29% 1.48% 1.33%
As % of the Application Platforms category 13.22% 8.33% 6.89%
Number of salaries quoted 231 135 151
City of London median annual salary £80,000 £72,500 £67,500
Median salary % change year-on-year +10.34% +7.41% -3.57%
10th Percentile £58,750 £52,500 £45,000
90th Percentile £107,500 £109,000 £105,000
Central London median annual salary £80,000 £72,500 £67,500
% change year-on-year +10.34% +7.41% -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in the City of London with a requirement for application platform skills.

All Application Platform Skills
City of London
Permanent vacancies with a requirement for application platform skills 1,898 1,813 2,525
As % of all permanent IT jobs advertised in the City of London 17.35% 17.79% 19.31%
Number of salaries quoted 1,710 1,634 2,294
City of London median annual salary £60,000 £57,500 £55,000
Median salary % change year-on-year +4.35% +4.55% +4.76%
10th Percentile £36,250 £33,500 £31,650
90th Percentile £90,000 £85,000 £83,750
Central London median annual salary £60,000 £56,000 £55,000
% change year-on-year +7.14% +1.82% +4.76%

Apache Spark
Job Vacancy Trend in the City of London

Job postings citing Apache Spark as a percentage of all IT jobs advertised in the City of London.

Job vacancy trend for Apache Spark in the City of London

Apache Spark
Salary Trend in the City of London

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in the City of London.

Salary trend for Apache Spark in the City of London

Apache Spark
Salary Histogram in the City of London

The salary distribution of IT jobs citing Apache Spark in the City of London over the 6 months to 14 August 2018.

Salary histogram for Apache Spark in the City of London

Apache Spark
Job Locations in the City of London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the City of London region over the 6 months to 14 August 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
London Bridge - 4 £80,000 -
Apache Spark
Central London

For the 6 months to 14 August 2018, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the City of London region with a requirement for Apache Spark.

1 157 (62.55%) Hadoop
2 154 (61.35%) Big Data
3 153 (60.96%) Java
4 151 (60.16%) Python
5 134 (53.39%) Scala
6 110 (43.82%) Kafka
7 88 (35.06%) Amazon AWS
8 85 (33.86%) SQL
9 83 (33.07%) Machine Learning
10 80 (31.87%) Finance
11 76 (30.28%) Analytics
11 76 (30.28%) Data Science
12 74 (29.48%) NoSQL
13 63 (25.10%) Agile Software Development
14 49 (19.52%) Apache Hive
15 48 (19.12%) DevOps
15 48 (19.12%) Data Engineering
16 47 (18.73%) Apache Cassandra
17 46 (18.33%) Microsoft Azure
18 44 (17.53%) Docker
18 44 (17.53%) Mathematics
19 43 (17.13%) Artificial Intelligence
20 40 (15.94%) Data Lake
21 39 (15.54%) Software Engineering
21 39 (15.54%) ETL
22 38 (15.14%) Linux
22 38 (15.14%) Degree
22 38 (15.14%) Open Source
23 36 (14.34%) R
23 36 (14.34%) Banking

Apache Spark
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 26 (10.36%) Elasticsearch
2 25 (9.96%) Apache
3 8 (3.19%) Apache Pig
4 6 (2.39%) Apache Solr
5 5 (1.99%) nginx
5 5 (1.99%) Oracle Workflow
6 3 (1.20%) Confluence
6 3 (1.20%) SAS
7 2 (0.80%) IIS
7 2 (0.80%) WebSphere
7 2 (0.80%) WebSphere Application Server
8 1 (0.40%) CMS
8 1 (0.40%) NServiceBus
8 1 (0.40%) SharePoint Server
Applications
1 5 (1.99%) Microsoft Excel
2 1 (0.40%) GNU Octave
2 1 (0.40%) Microsoft Office
2 1 (0.40%) Microsoft PowerPoint
2 1 (0.40%) MS Visio
Cloud Services
1 88 (35.06%) Amazon AWS
2 46 (18.33%) Microsoft Azure
3 15 (5.98%) Amazon EMR
4 12 (4.78%) Google Cloud Platform
5 11 (4.38%) Amazon Kinesis
6 6 (2.39%) Amazon EC2
7 3 (1.20%) Cloud Computing
7 3 (1.20%) OpenShift
8 2 (0.80%) Amazon ElastiCache
8 2 (0.80%) AWS Lambda
8 2 (0.80%) Azure Data Factory
8 2 (0.80%) Datadog
8 2 (0.80%) GitHub
8 2 (0.80%) Route 53
8 2 (0.80%) SaaS
8 2 (0.80%) Serverless
9 1 (0.40%) Amazon SQS
9 1 (0.40%) Azure Machine Learning
9 1 (0.40%) CloudBees
9 1 (0.40%) PaaS
Communications & Networking
1 2 (0.80%) DNS
1 2 (0.80%) Firewall
1 2 (0.80%) HTTP
1 2 (0.80%) SMTP
1 2 (0.80%) TCP/IP
1 2 (0.80%) VPN
2 1 (0.40%) Internet
Database & Business Intelligence
1 157 (62.55%) Hadoop
2 154 (61.35%) Big Data
3 74 (29.48%) NoSQL
4 49 (19.52%) Apache Hive
5 47 (18.73%) Apache Cassandra
6 40 (15.94%) Data Lake
7 28 (11.16%) Relational Database
8 23 (9.16%) MongoDB
9 19 (7.57%) Data Warehouse
9 19 (7.57%) MySQL
9 19 (7.57%) SQL Server
10 16 (6.37%) Amazon Redshift
10 16 (6.37%) HBase
11 15 (5.98%) Cosmos DB
12 12 (4.78%) SQL Server Integration Services
13 11 (4.38%) Sqoop
14 10 (3.98%) PostgreSQL
15 9 (3.59%) Power BI
16 8 (3.19%) RDBMS
17 7 (2.79%) Data Mining
Development Applications
1 17 (6.77%) Jenkins
2 12 (4.78%) Git (software)
3 7 (2.79%) JIRA
4 5 (1.99%) RStudio
5 4 (1.59%) JUnit
5 4 (1.59%) Sonatype Nexus
5 4 (1.59%) Travis CI
6 3 (1.20%) Artifactory
7 2 (0.80%) Atlassian Bamboo
7 2 (0.80%) GitLab
7 2 (0.80%) Gradle
7 2 (0.80%) Hudson
7 2 (0.80%) Jasmine
7 2 (0.80%) Maven
7 2 (0.80%) Mocha
7 2 (0.80%) Mockito
7 2 (0.80%) Oracle APEX
7 2 (0.80%) Selenium
7 2 (0.80%) TeamCity
7 2 (0.80%) TestNG
General
1 80 (31.87%) Finance
2 36 (14.34%) Banking
3 14 (5.58%) Games
4 11 (4.38%) Marketing
4 11 (4.38%) Publishing
5 10 (3.98%) Pharmaceutical
5 10 (3.98%) Retail
6 9 (3.59%) Investment Banking
7 6 (2.39%) Retail Banking
8 5 (1.99%) Telecoms
9 4 (1.59%) Front Office
10 2 (0.80%) Advertising
10 2 (0.80%) Automotive
10 2 (0.80%) Billing
11 1 (0.40%) Law
Job Titles
1 104 (41.43%) Developer
2 50 (19.92%) Data Engineer
3 44 (17.53%) Java Developer
4 41 (16.33%) Data Scientist
5 30 (11.95%) Big Data Developer
6 29 (11.55%) Scala Developer
6 29 (11.55%) Senior Data Scientist
7 26 (10.36%) Big Data Engineer
8 16 (6.37%) C++ Developer
8 16 (6.37%) Senior Developer
9 13 (5.18%) Java Scala Developer
9 13 (5.18%) Python Developer
10 12 (4.78%) Hadoop Developer
11 11 (4.38%) AWS Engineer
12 10 (3.98%) Apache Spark Developer
12 10 (3.98%) Architect
12 10 (3.98%) Hadoop Engineer
12 10 (3.98%) Java Engineer
13 9 (3.59%) Senior Java Developer
14 8 (3.19%) Java Hadoop Developer
Libraries, Frameworks & Software Standards
1 110 (43.82%) Kafka
2 26 (10.36%) Natural Language Processing
3 21 (8.37%) Akka
3 21 (8.37%) Web Services
4 18 (7.17%) REST
5 15 (5.98%) RabbitMQ
6 14 (5.58%) Pandas
7 13 (5.18%) Spring
8 12 (4.78%) scikit-learn
9 11 (4.38%) .NET
10 10 (3.98%) MLlib
10 10 (3.98%) Node.js
10 10 (3.98%) TensorFlow
11 9 (3.59%) AngularJS
11 9 (3.59%) RESTful
12 8 (3.19%) Elastic Stack
13 6 (2.39%) SciPy
14 5 (1.99%) .NET Framework
14 5 (1.99%) Flask
14 5 (1.99%) Hibernate
Miscellaneous
1 32 (12.75%) Algorithms
2 31 (12.35%) Fintech
2 31 (12.35%) Greenfield Project
3 28 (11.16%) Analytical Skills
4 23 (9.16%) Computer Science
4 23 (9.16%) Self-Motivation
5 12 (4.78%) Derivative
6 10 (3.98%) Data Structures
7 7 (2.79%) Linux Command Line
8 6 (2.39%) Public Cloud
9 5 (1.99%) Market Risk
9 5 (1.99%) Tandem
10 4 (1.59%) Blog
10 4 (1.59%) Life Science
10 4 (1.59%) NHS
10 4 (1.59%) Social Network
11 3 (1.20%) Semantic Web
11 3 (1.20%) Spread Betting
12 2 (0.80%) Equities
12 2 (0.80%) Foreign Exchange (FX)
Operating Systems
1 38 (15.14%) Linux
2 33 (13.15%) Unix
3 2 (0.80%) Ubuntu
Processes & Methodologies
1 83 (33.07%) Machine Learning
2 76 (30.28%) Analytics
2 76 (30.28%) Data Science
3 63 (25.10%) Agile Software Development
4 48 (19.12%) Data Engineering
4 48 (19.12%) DevOps
5 44 (17.53%) Mathematics
6 43 (17.13%) Artificial Intelligence
7 39 (15.54%) ETL
7 39 (15.54%) Software Engineering
8 38 (15.14%) Open Source
9 30 (11.95%) TDD
10 25 (9.96%) BDD
10 25 (9.96%) Statistics
11 23 (9.16%) Deep Learning
11 23 (9.16%) Microservices
12 19 (7.57%) Prototyping
13 18 (7.17%) Change Control
14 17 (6.77%) Business Intelligence
15 16 (6.37%) Problem-Solving
Programming Languages
1 153 (60.96%) Java
2 151 (60.16%) Python
3 134 (53.39%) Scala
4 85 (33.86%) SQL
5 36 (14.34%) R
6 31 (12.35%) JavaScript
7 30 (11.95%) C++
7 30 (11.95%) Java 8
8 21 (8.37%) C#
9 11 (4.38%) MATLAB
10 8 (3.19%) Ruby
11 7 (2.79%) Bash Shell
11 7 (2.79%) Go
11 7 (2.79%) Perl
12 6 (2.39%) Kotlin
12 6 (2.39%) Shell Script
12 6 (2.39%) T-SQL
13 5 (1.99%) Korn
14 4 (1.59%) Haskell
14 4 (1.59%) VBA
Qualifications
1 38 (15.14%) Degree
2 13 (5.18%) PhD
3 4 (1.59%) Computer Science Degree
4 3 (1.20%) Master's Degree
5 2 (0.80%) Numerate Degree
6 1 (0.40%) MBA
6 1 (0.40%) Security Cleared
Quality Assurance & Compliance
1 4 (1.59%) GCP
2 3 (1.20%) QA
3 2 (0.80%) Data Quality
4 1 (0.40%) GDPR
System Software
1 44 (17.53%) Docker
2 21 (8.37%) HDFS
3 18 (7.17%) Apache Flume
4 4 (1.59%) VirtualBox
5 3 (1.20%) Apache ZooKeeper
5 3 (1.20%) Oracle RAC
6 2 (0.80%) Active Directory
6 2 (0.80%) KVM
6 2 (0.80%) LXC
6 2 (0.80%) Virtual Machines
6 2 (0.80%) VMware Infrastructure
6 2 (0.80%) Xen
Systems Management
1 25 (9.96%) Kubernetes
2 22 (8.76%) Mesos
3 14 (5.58%) Ansible
4 11 (4.38%) Terraform
5 10 (3.98%) Prometheus
6 9 (3.59%) Oozie
7 8 (3.19%) Hadoop YARN
7 8 (3.19%) Opscode Chef
8 6 (2.39%) Puppet
9 3 (1.20%) Consul
9 3 (1.20%) Data Guard
9 3 (1.20%) Oracle RMAN
10 2 (0.80%) Docker Swarm
10 2 (0.80%) Kibana
10 2 (0.80%) logstash
10 2 (0.80%) Red Hat Spacewalk
10 2 (0.80%) Salt
11 1 (0.40%) Grafana
11 1 (0.40%) Graphite
11 1 (0.40%) rkt
Vendors
1 22 (8.76%) Oracle
2 15 (5.98%) Google
2 15 (5.98%) Tableau
3 13 (5.18%) Cloudera
4 12 (4.78%) Hortonworks
5 9 (3.59%) Informatica
5 9 (3.59%) MapR
6 7 (2.79%) Solace
7 6 (2.39%) Microsoft
8 5 (1.99%) Sun
9 4 (1.59%) Apigee
9 4 (1.59%) Atlassian
9 4 (1.59%) Datameer
9 4 (1.59%) IBM
9 4 (1.59%) New Relic
9 4 (1.59%) VMware
10 3 (1.20%) Talend
11 2 (0.80%) Alteryx
11 2 (0.80%) Facebook
11 2 (0.80%) SaltStack