Period
to 16 October 2018

The following table provides summary statistics for permanent job vacancies advertised in the City of London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 16 October 2018 with a comparison to the same period in the previous 2 years.

Apache Spark
Central London > City of London
6 months to
16 Oct 2018
Same period 2017 Same period 2016
Rank 151 158 231
Rank change year-on-year +7 +73 +91
Permanent jobs citing Apache Spark 241 188 134
As % of all permanent IT jobs advertised in the City of London 2.18% 1.89% 1.12%
As % of the Application Platforms category 12.30% 10.04% 5.77%
Number of salaries quoted 227 172 115
City of London median annual salary £80,000 £75,000 £70,000
Median salary % change year-on-year +6.67% +7.14% -
10th Percentile £52,500 £52,500 £45,500
90th Percentile £102,500 £116,125 £105,000
Central London median annual salary £80,000 £72,500 £76,250
% change year-on-year +10.34% -4.92% +8.93%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in the City of London with a requirement for application platform skills.

All Application Platform Skills
City of London
Permanent vacancies with a requirement for application platform skills 1,960 1,872 2,323
As % of all permanent IT jobs advertised in the City of London 17.70% 18.77% 19.42%
Number of salaries quoted 1,792 1,715 2,132
City of London median annual salary £60,000 £60,000 £55,000
Median salary % change year-on-year - +9.09% +1.37%
10th Percentile £35,750 £36,250 £34,000
90th Percentile £90,000 £87,500 £83,750
Central London median annual salary £60,000 £55,250 £55,000
% change year-on-year +8.60% +0.45% +4.76%

Apache Spark
Job Vacancy Trend in the City of London

Job postings citing Apache Spark as a percentage of all IT jobs advertised in the City of London.

Job vacancy trend for Apache Spark in the City of London

Apache Spark
Salary Trend in the City of London

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in the City of London.

Salary trend for Apache Spark in the City of London

Apache Spark
Salary Histogram in the City of London

The salary distribution of IT jobs citing Apache Spark in the City of London over the 6 months to 16 October 2018.

Salary histogram for Apache Spark in the City of London

Apache Spark
Job Locations in the City of London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the City of London region over the 6 months to 16 October 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
London Bridge - 5 £80,000 -
Apache Spark
Central London

For the 6 months to 16 October 2018, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the City of London region with a requirement for Apache Spark.

1 151 (62.66%) Hadoop
2 136 (56.43%) Python
3 133 (55.19%) Big Data
4 122 (50.62%) Java
5 117 (48.55%) Scala
6 112 (46.47%) Kafka
7 77 (31.95%) Finance
8 74 (30.71%) SQL
8 74 (30.71%) Amazon AWS
9 73 (30.29%) Machine Learning
10 70 (29.05%) Data Science
11 63 (26.14%) Agile Software Development
12 62 (25.73%) Analytics
13 51 (21.16%) Apache Cassandra
13 51 (21.16%) NoSQL
14 50 (20.75%) Software Engineering
14 50 (20.75%) Microsoft Azure
15 49 (20.33%) Apache Hive
15 49 (20.33%) DevOps
16 44 (18.26%) Open Source
17 41 (17.01%) ETL
18 40 (16.60%) Linux
18 40 (16.60%) Docker
19 39 (16.18%) Data Engineering
19 39 (16.18%) Banking
20 38 (15.77%) Mathematics
21 34 (14.11%) Computer Science
21 34 (14.11%) C++
21 34 (14.11%) Microservices
21 34 (14.11%) Fintech

Apache Spark
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 31 (12.86%) Apache
2 23 (9.54%) Elasticsearch
3 10 (4.15%) Apache Pig
4 8 (3.32%) SAS
5 5 (2.07%) Oracle Workflow
6 4 (1.66%) Confluence
7 3 (1.24%) Apache Solr
7 3 (1.24%) WebSphere
7 3 (1.24%) WebSphere Application Server
8 2 (0.83%) IIS
9 1 (0.41%) nginx
9 1 (0.41%) NServiceBus
9 1 (0.41%) SharePoint Server
Applications
1 2 (0.83%) Microsoft Excel
2 1 (0.41%) GNU Octave
2 1 (0.41%) Microsoft Office
2 1 (0.41%) Microsoft PowerPoint
2 1 (0.41%) MS Visio
Cloud Services
1 74 (30.71%) Amazon AWS
2 50 (20.75%) Microsoft Azure
3 20 (8.30%) Google Cloud Platform
4 15 (6.22%) Amazon EMR
5 7 (2.90%) Amazon Kinesis
6 5 (2.07%) Azure Data Factory
6 5 (2.07%) Cloud Computing
6 5 (2.07%) Serverless
7 3 (1.24%) Amazon EC2
7 3 (1.24%) Azure Functions
7 3 (1.24%) Data Lake Analytics
8 2 (0.83%) Amazon CloudWatch
8 2 (0.83%) AWS Lambda
8 2 (0.83%) Azure Machine Learning
8 2 (0.83%) Google App Engine
9 1 (0.41%) Amazon ElastiCache
9 1 (0.41%) AWS Data Pipeline
9 1 (0.41%) CloudBees
9 1 (0.41%) Cortana Intelligence Suite
9 1 (0.41%) Google Analytics
Communications & Networking
1 3 (1.24%) AMQP
2 1 (0.41%) Firewall
2 1 (0.41%) Internet
2 1 (0.41%) NX-OS
2 1 (0.41%) VLAN
Database & Business Intelligence
1 151 (62.66%) Hadoop
2 133 (55.19%) Big Data
3 51 (21.16%) Apache Cassandra
3 51 (21.16%) NoSQL
4 49 (20.33%) Apache Hive
5 29 (12.03%) Data Lake
6 27 (11.20%) MongoDB
7 24 (9.96%) Relational Database
8 20 (8.30%) SQL Server
9 18 (7.47%) Data Warehouse
10 17 (7.05%) Amazon Redshift
11 14 (5.81%) PostgreSQL
11 14 (5.81%) Sqoop
12 13 (5.39%) Cosmos DB
12 13 (5.39%) MySQL
13 11 (4.56%) HBase
13 11 (4.56%) SQL Server Integration Services
14 9 (3.73%) Power BI
15 7 (2.90%) RDBMS
15 7 (2.90%) Redis
Development Applications
1 23 (9.54%) Jenkins
2 18 (7.47%) Git (software)
3 12 (4.98%) JIRA
4 5 (2.07%) JUnit
4 5 (2.07%) TeamCity
5 4 (1.66%) Octopus Deploy
6 3 (1.24%) GitLab
6 3 (1.24%) IBM UrbanCode
6 3 (1.24%) Jasmine
6 3 (1.24%) Maven
6 3 (1.24%) Mocha
6 3 (1.24%) Mockito
6 3 (1.24%) RStudio
6 3 (1.24%) Sonatype Nexus
6 3 (1.24%) SpecFlow
6 3 (1.24%) TestNG
6 3 (1.24%) tSQLt
6 3 (1.24%) Visual Studio Team System
7 2 (0.83%) Oracle APEX
7 2 (0.83%) Selenium
General
1 77 (31.95%) Finance
2 39 (16.18%) Banking
3 13 (5.39%) Games
4 12 (4.98%) Investment Banking
5 10 (4.15%) Marketing
5 10 (4.15%) Retail
6 9 (3.73%) Retail Banking
7 8 (3.32%) Telecoms
8 5 (2.07%) Advertising
8 5 (2.07%) Pharmaceutical
9 4 (1.66%) Front Office
10 3 (1.24%) Publishing
11 2 (0.83%) Automotive
11 2 (0.83%) Billing
12 1 (0.41%) Financial Institution
12 1 (0.41%) Law
Job Titles
1 103 (42.74%) Developer
2 46 (19.09%) Data Engineer
3 42 (17.43%) Data Scientist
4 39 (16.18%) Java Developer
5 28 (11.62%) Scala Developer
5 28 (11.62%) Senior Data Scientist
6 22 (9.13%) C++ Developer
7 21 (8.71%) Big Data Engineer
8 19 (7.88%) Big Data Developer
9 16 (6.64%) Senior Developer
10 15 (6.22%) Python Developer
11 14 (5.81%) Architect
12 13 (5.39%) Apache Spark Developer
12 13 (5.39%) Java Scala Developer
13 12 (4.98%) AWS Engineer
13 12 (4.98%) Python Engineer
14 11 (4.56%) Hadoop Developer
15 9 (3.73%) AWS Developer
16 8 (3.32%) Hadoop Engineer
16 8 (3.32%) Java Engineer
Libraries, Frameworks & Software Standards
1 112 (46.47%) Kafka
2 26 (10.79%) Natural Language Processing
3 19 (7.88%) AngularJS
3 19 (7.88%) Pandas
4 18 (7.47%) REST
5 17 (7.05%) Web Services
6 16 (6.64%) Akka
6 16 (6.64%) RabbitMQ
7 12 (4.98%) .NET
7 12 (4.98%) scikit-learn
7 12 (4.98%) Spring
7 12 (4.98%) TensorFlow
8 11 (4.56%) MLlib
8 11 (4.56%) Node.js
9 9 (3.73%) SciPy
10 8 (3.32%) NumPy
11 7 (2.90%) RESTful
12 6 (2.49%) Apache Commons
13 5 (2.07%) .NET Framework
13 5 (2.07%) Flask
Miscellaneous
1 34 (14.11%) Computer Science
1 34 (14.11%) Fintech
2 31 (12.86%) Algorithms
3 29 (12.03%) Greenfield Project
4 26 (10.79%) Analytical Skills
5 23 (9.54%) Self-Motivation
6 19 (7.88%) Derivative
7 10 (4.15%) Data Structures
8 9 (3.73%) Life Science
9 8 (3.32%) Social Network
10 7 (2.90%) Market Risk
10 7 (2.90%) Spread Betting
11 4 (1.66%) Public Cloud
12 3 (1.24%) Clustering
12 3 (1.24%) NHS
13 2 (0.83%) Enterprise Cloud
13 2 (0.83%) Equities
13 2 (0.83%) Foreign Exchange (FX)
13 2 (0.83%) Management Information System
13 2 (0.83%) Mobile App
Operating Systems
1 40 (16.60%) Linux
2 20 (8.30%) Unix
3 2 (0.83%) Ubuntu
4 1 (0.41%) CentOS
4 1 (0.41%) Mac OS
4 1 (0.41%) Windows
4 1 (0.41%) Windows Server
4 1 (0.41%) Windows Server 2012
Processes & Methodologies
1 73 (30.29%) Machine Learning
2 70 (29.05%) Data Science
3 63 (26.14%) Agile Software Development
4 62 (25.73%) Analytics
5 50 (20.75%) Software Engineering
6 49 (20.33%) DevOps
7 44 (18.26%) Open Source
8 41 (17.01%) ETL
9 39 (16.18%) Data Engineering
10 38 (15.77%) Mathematics
11 34 (14.11%) Microservices
12 30 (12.45%) Artificial Intelligence
13 27 (11.20%) Deep Learning
14 20 (8.30%) Statistics
14 20 (8.30%) TDD
15 19 (7.88%) Data Analytics
15 19 (7.88%) SDLC
16 16 (6.64%) Visualisation
17 15 (6.22%) Data Analysis
17 15 (6.22%) Data Modelling
Programming Languages
1 136 (56.43%) Python
2 122 (50.62%) Java
3 117 (48.55%) Scala
4 74 (30.71%) SQL
5 34 (14.11%) C++
6 31 (12.86%) JavaScript
6 31 (12.86%) R
7 20 (8.30%) C#
8 15 (6.22%) Java 8
9 9 (3.73%) Bash Shell
9 9 (3.73%) Ruby
9 9 (3.73%) T-SQL
10 8 (3.32%) MATLAB
11 7 (2.90%) Kotlin
11 7 (2.90%) Perl
11 7 (2.90%) Shell Script
12 5 (2.07%) Go
12 5 (2.07%) Korn
12 5 (2.07%) PowerShell
13 4 (1.66%) Haskell
Qualifications
1 33 (13.69%) Degree
2 14 (5.81%) PhD
3 6 (2.49%) Computer Science Degree
4 5 (2.07%) Master's Degree
5 2 (0.83%) Numerate Degree
Quality Assurance & Compliance
1 5 (2.07%) Data Quality
2 3 (1.24%) GDPR
2 3 (1.24%) QA
System Software
1 40 (16.60%) Docker
2 20 (8.30%) HDFS
3 12 (4.98%) Apache Flume
4 4 (1.66%) VirtualBox
5 3 (1.24%) Apache ZooKeeper
5 3 (1.24%) Oracle RAC
5 3 (1.24%) VMware Infrastructure
6 2 (0.83%) Active Directory
6 2 (0.83%) KVM
6 2 (0.83%) Virtual Machines
6 2 (0.83%) Xen
7 1 (0.41%) Firmware
7 1 (0.41%) LXC
7 1 (0.41%) VMware ESXi
Systems Management
1 26 (10.79%) Kubernetes
2 23 (9.54%) Mesos
3 12 (4.98%) Prometheus
4 11 (4.56%) Ansible
5 10 (4.15%) Oozie
6 8 (3.32%) Terraform
7 7 (2.90%) Kibana
8 4 (1.66%) Hadoop YARN
8 4 (1.66%) logstash
9 3 (1.24%) Data Guard
9 3 (1.24%) Grafana
9 3 (1.24%) Graphite
9 3 (1.24%) Opscode Chef
9 3 (1.24%) Oracle RMAN
9 3 (1.24%) Puppet
10 2 (0.83%) Docker Swarm
10 2 (0.83%) Red Hat Spacewalk
11 1 (0.41%) Consul
11 1 (0.41%) rkt
11 1 (0.41%) vCenter Server
Vendors
1 17 (7.05%) Google
1 17 (7.05%) Oracle
1 17 (7.05%) Tableau
2 13 (5.39%) Microsoft
3 12 (4.98%) Hortonworks
4 11 (4.56%) Cloudera
4 11 (4.56%) Informatica
5 8 (3.32%) IBM
5 8 (3.32%) MapR
6 7 (2.90%) Solace
7 6 (2.49%) Sun
8 5 (2.07%) SAP
9 4 (1.66%) Apigee
10 3 (1.24%) Datameer
10 3 (1.24%) Talend
10 3 (1.24%) Teradata
10 3 (1.24%) VMware
11 2 (0.83%) AppDynamics
11 2 (0.83%) Facebook
11 2 (0.83%) New Relic