Period
to 18 November 2018

The following table provides summary statistics for permanent job vacancies advertised in Central London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 18 November 2018 with a comparison to the same period in the previous 2 years.

Apache Spark
London > Central London
6 months to
18 Nov 2018
Same period 2017 Same period 2016
Rank 181 168 235
Rank change year-on-year -13 +67 +56
Permanent jobs citing Apache Spark 258 259 176
As % of all permanent IT jobs advertised in Central London 1.85% 1.89% 1.22%
As % of the Application Platforms category 10.69% 9.87% 6.15%
Number of salaries quoted 238 239 153
Central London median annual salary £80,000 £75,000 £75,000
Median salary % change year-on-year +6.67% - +11.11%
10th Percentile £47,500 £52,250 £48,000
90th Percentile £105,000 £115,000 £105,000
London median annual salary £80,000 £70,000 £75,000
% change year-on-year +14.29% -6.67% +11.11%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in Central London with a requirement for application platform skills.

All Application Platform Skills
Central London
Permanent vacancies with a requirement for application platform skills 2,414 2,623 2,863
As % of all permanent IT jobs advertised in Central London 17.33% 19.15% 19.89%
Number of salaries quoted 2,180 2,369 2,600
Central London median annual salary £60,000 £57,500 £55,000
Median salary % change year-on-year +4.35% +4.55% +4.76%
10th Percentile £33,750 £32,500 £32,975
90th Percentile £90,000 £85,000 £83,750
London median annual salary £60,000 £57,500 £55,000
% change year-on-year +4.35% +4.55% +4.76%

Apache Spark
Job Vacancy Trend in Central London

Job postings citing Apache Spark as a percentage of all IT jobs advertised in Central London.

Job vacancy trend for Apache Spark in Central London

Apache Spark
Salary Trend in Central London

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in Central London.

Salary trend for Apache Spark in Central London

Apache Spark
Salary Histogram in Central London

The salary distribution of IT jobs citing Apache Spark in Central London over the 6 months to 18 November 2018.

Salary histogram for Apache Spark in Central London

Apache Spark
Job Locations in Central London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the Central London region over the 6 months to 18 November 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
City of London -15 219 £80,000 +6.67% 47
West End of London -12 12 £65,000 -3.70% 10
City of Westminster - 4 £88,748 -
Southwark +54 3 £80,000 +6.67%
Kensington and Chelsea - 2 - -
Apache Spark
London

For the 6 months to 18 November 2018, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the Central London region with a requirement for Apache Spark.

1 166 (64.34%) Hadoop
2 153 (59.30%) Python
3 144 (55.81%) Big Data
4 125 (48.45%) Java
5 119 (46.12%) Kafka
5 119 (46.12%) Scala
6 90 (34.88%) Machine Learning
7 88 (34.11%) Data Science
8 84 (32.56%) SQL
9 81 (31.40%) Agile Software Development
10 80 (31.01%) Amazon AWS
11 68 (26.36%) Finance
12 67 (25.97%) Analytics
13 59 (22.87%) Apache Cassandra
14 57 (22.09%) DevOps
15 56 (21.71%) Apache Hive
16 54 (20.93%) Microsoft Azure
17 49 (18.99%) Software Engineering
18 48 (18.60%) NoSQL
19 45 (17.44%) Open Source
19 45 (17.44%) Artificial Intelligence
19 45 (17.44%) ETL
20 44 (17.05%) Linux
21 43 (16.67%) Computer Science
21 43 (16.67%) Docker
22 42 (16.28%) Algorithms
23 39 (15.12%) Fintech
24 38 (14.73%) Mathematics
24 38 (14.73%) Microservices
25 36 (13.95%) MongoDB

Apache Spark
Co-occurring IT Skills in Central London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 35 (13.57%) Apache
2 27 (10.47%) Elasticsearch
3 18 (6.98%) Apache Pig
4 10 (3.88%) SAS
5 3 (1.16%) Confluence
5 3 (1.16%) Oracle Workflow
5 3 (1.16%) WebSphere
5 3 (1.16%) WebSphere Application Server
6 2 (0.78%) Apache Solr
6 2 (0.78%) IIS
7 1 (0.39%) NServiceBus
Applications
1 3 (1.16%) Chatbot
2 2 (0.78%) GNU Octave
2 2 (0.78%) Microsoft Excel
3 1 (0.39%) Microsoft Office
3 1 (0.39%) Microsoft PowerPoint
3 1 (0.39%) MS Visio
Cloud Services
1 80 (31.01%) Amazon AWS
2 54 (20.93%) Microsoft Azure
3 25 (9.69%) Google Cloud Platform
4 12 (4.65%) Amazon EMR
5 7 (2.71%) Cloud Computing
6 6 (2.33%) Amazon EC2
6 6 (2.33%) Azure Data Factory
7 5 (1.94%) Serverless
8 4 (1.55%) Amazon Kinesis
8 4 (1.55%) Apigee
8 4 (1.55%) Azure Functions
8 4 (1.55%) Data Lake Analytics
9 3 (1.16%) Azure Machine Learning
9 3 (1.16%) GitHub
9 3 (1.16%) Google Analytics
10 2 (0.78%) Amazon CloudWatch
10 2 (0.78%) Amazon S3
10 2 (0.78%) AWS Lambda
10 2 (0.78%) Google App Engine
11 1 (0.39%) PaaS
Communications & Networking
1 4 (1.55%) AMQP
2 2 (0.78%) Intrusion Detection
3 1 (0.39%) Firewall
3 1 (0.39%) Internet
3 1 (0.39%) NX-OS
3 1 (0.39%) VLAN
Database & Business Intelligence
1 166 (64.34%) Hadoop
2 144 (55.81%) Big Data
3 59 (22.87%) Apache Cassandra
4 56 (21.71%) Apache Hive
5 48 (18.60%) NoSQL
6 36 (13.95%) MongoDB
7 25 (9.69%) Data Warehouse
8 22 (8.53%) Data Lake
8 22 (8.53%) HBase
9 21 (8.14%) Relational Database
10 19 (7.36%) SQL Server
11 18 (6.98%) Amazon Redshift
11 18 (6.98%) MySQL
11 18 (6.98%) PostgreSQL
12 11 (4.26%) Sqoop
13 10 (3.88%) SQL Server Integration Services
14 9 (3.49%) RDBMS
15 8 (3.10%) Cosmos DB
15 8 (3.10%) Power BI
15 8 (3.10%) Spark SQL
Development Applications
1 25 (9.69%) Jenkins
2 19 (7.36%) Git (software)
3 11 (4.26%) JIRA
4 6 (2.33%) JUnit
5 4 (1.55%) GitLab
5 4 (1.55%) Mockito
5 4 (1.55%) Octopus Deploy
5 4 (1.55%) SpecFlow
5 4 (1.55%) TeamCity
5 4 (1.55%) TestNG
5 4 (1.55%) tSQLt
5 4 (1.55%) Visual Studio Team System
6 3 (1.16%) IBM UrbanCode
6 3 (1.16%) Jasmine
6 3 (1.16%) Maven
6 3 (1.16%) Mocha
6 3 (1.16%) RStudio
6 3 (1.16%) Sonatype Nexus
6 3 (1.16%) Travis CI
7 2 (0.78%) Selenium
General
1 68 (26.36%) Finance
2 30 (11.63%) Banking
3 16 (6.20%) Retail
4 13 (5.04%) Games
5 12 (4.65%) Marketing
6 11 (4.26%) Retail Banking
7 9 (3.49%) Investment Banking
8 8 (3.10%) Telecoms
9 5 (1.94%) Automotive
9 5 (1.94%) Pharmaceutical
10 4 (1.55%) Advertising
11 3 (1.16%) Financial Institution
11 3 (1.16%) Front Office
11 3 (1.16%) Law
12 2 (0.78%) Billing
12 2 (0.78%) Military
Job Titles
1 105 (40.70%) Developer
2 52 (20.16%) Data Engineer
3 49 (18.99%) Data Scientist
4 40 (15.50%) Java Developer
5 32 (12.40%) Scala Developer
6 26 (10.08%) Senior Data Scientist
7 24 (9.30%) Big Data Engineer
8 23 (8.91%) C++ Developer
9 18 (6.98%) Python Developer
9 18 (6.98%) Python Engineer
10 17 (6.59%) Java Scala Developer
11 16 (6.20%) AWS Engineer
12 15 (5.81%) Apache Spark Developer
12 15 (5.81%) Architect
13 14 (5.43%) AWS Developer
14 13 (5.04%) Big Data Developer
15 12 (4.65%) Senior Developer
16 10 (3.88%) DevOps Engineer
16 10 (3.88%) Hadoop Developer
16 10 (3.88%) Java Engineer
Libraries, Frameworks & Software Standards
1 119 (46.12%) Kafka
2 24 (9.30%) Pandas
3 23 (8.91%) AngularJS
4 22 (8.53%) Natural Language Processing
5 19 (7.36%) Web Services
6 17 (6.59%) Akka
6 17 (6.59%) RabbitMQ
6 17 (6.59%) scikit-learn
7 16 (6.20%) NumPy
8 14 (5.43%) REST
9 13 (5.04%) SciPy
9 13 (5.04%) Spring
10 12 (4.65%) MLlib
10 12 (4.65%) TensorFlow
11 11 (4.26%) Node.js
12 10 (3.88%) .NET
12 10 (3.88%) RESTful
13 9 (3.49%) Spark Streaming
14 6 (2.33%) Apache Commons
14 6 (2.33%) React
Miscellaneous
1 43 (16.67%) Computer Science
2 42 (16.28%) Algorithms
3 39 (15.12%) Fintech
4 33 (12.79%) Analytical Skills
5 28 (10.85%) Greenfield Project
6 27 (10.47%) Self-Motivation
7 20 (7.75%) Derivative
8 12 (4.65%) Data Structures
9 10 (3.88%) Market Risk
10 9 (3.49%) Life Science
10 9 (3.49%) Spread Betting
11 7 (2.71%) Public Cloud
11 7 (2.71%) Social Network
12 4 (1.55%) NHS
13 3 (1.16%) Clustering
13 3 (1.16%) Data Centre
14 2 (0.78%) Enterprise Cloud
14 2 (0.78%) Mobile App
14 2 (0.78%) Smart City
14 2 (0.78%) Social Media
Operating Systems
1 44 (17.05%) Linux
2 15 (5.81%) Unix
3 4 (1.55%) Ubuntu
4 1 (0.39%) CentOS
4 1 (0.39%) Mac OS
4 1 (0.39%) Windows
4 1 (0.39%) Windows Server
4 1 (0.39%) Windows Server 2012
Processes & Methodologies
1 90 (34.88%) Machine Learning
2 88 (34.11%) Data Science
3 81 (31.40%) Agile Software Development
4 67 (25.97%) Analytics
5 57 (22.09%) DevOps
6 49 (18.99%) Software Engineering
7 45 (17.44%) Artificial Intelligence
7 45 (17.44%) ETL
7 45 (17.44%) Open Source
8 38 (14.73%) Mathematics
8 38 (14.73%) Microservices
9 29 (11.24%) Data Engineering
10 25 (9.69%) Deep Learning
11 22 (8.53%) Data Modelling
11 22 (8.53%) TDD
12 21 (8.14%) Data Analytics
13 20 (7.75%) Problem-Solving
14 19 (7.36%) SDLC
15 18 (6.98%) Business Intelligence
15 18 (6.98%) Visualisation
Programming Languages
1 153 (59.30%) Python
2 125 (48.45%) Java
3 119 (46.12%) Scala
4 84 (32.56%) SQL
5 36 (13.95%) C++
6 35 (13.57%) R
7 28 (10.85%) JavaScript
8 22 (8.53%) C#
9 14 (5.43%) Java 8
10 11 (4.26%) Bash Shell
10 11 (4.26%) Shell Script
11 9 (3.49%) Ruby
12 8 (3.10%) Kotlin
12 8 (3.10%) MATLAB
12 8 (3.10%) T-SQL
13 6 (2.33%) PowerShell
14 5 (1.94%) Perl
15 4 (1.55%) Go
15 4 (1.55%) Haskell
15 4 (1.55%) U-SQL
Qualifications
1 33 (12.79%) Degree
2 17 (6.59%) PhD
3 9 (3.49%) Computer Science Degree
4 3 (1.16%) Master's Degree
4 3 (1.16%) Numerate Degree
5 1 (0.39%) Postgraduate
Quality Assurance & Compliance
1 5 (1.94%) Data Quality
2 4 (1.55%) QA
3 3 (1.16%) GDPR
4 2 (0.78%) ISO/IEC 27001
5 1 (0.39%) MiFID
System Software
1 43 (16.67%) Docker
2 16 (6.20%) HDFS
3 14 (5.43%) Apache Flume
4 5 (1.94%) Apache ZooKeeper
4 5 (1.94%) VMware Infrastructure
5 3 (1.16%) VirtualBox
6 2 (0.78%) Active Directory
6 2 (0.78%) Hyper-V
6 2 (0.78%) KVM
6 2 (0.78%) Virtual Machines
6 2 (0.78%) Xen
7 1 (0.39%) Firmware
7 1 (0.39%) Oracle RAC
7 1 (0.39%) VMware ESXi
Systems Management
1 26 (10.08%) Mesos
2 24 (9.30%) Kubernetes
3 19 (7.36%) Oozie
4 14 (5.43%) Ansible
4 14 (5.43%) Prometheus
5 12 (4.65%) Terraform
6 9 (3.49%) Kibana
7 6 (2.33%) Hadoop YARN
8 5 (1.94%) logstash
9 4 (1.55%) Grafana
10 3 (1.16%) Graphite
11 2 (0.78%) Docker Swarm
11 2 (0.78%) Opscode Chef
11 2 (0.78%) Puppet
11 2 (0.78%) Red Hat Spacewalk
12 1 (0.39%) Ambari
12 1 (0.39%) Consul
12 1 (0.39%) Data Guard
12 1 (0.39%) Oracle RMAN
12 1 (0.39%) vCenter Server
Vendors
1 24 (9.30%) Google
2 19 (7.36%) Oracle
3 17 (6.59%) Tableau
4 14 (5.43%) Cloudera
4 14 (5.43%) Hortonworks
5 12 (4.65%) Microsoft
6 9 (3.49%) IBM
6 9 (3.49%) Informatica
6 9 (3.49%) MapR
7 8 (3.10%) Solace
8 5 (1.94%) SAP
8 5 (1.94%) Talend
8 5 (1.94%) VMware
9 4 (1.55%) Sun
10 3 (1.16%) Teradata
11 2 (0.78%) AppDynamics
11 2 (0.78%) Databricks
11 2 (0.78%) Facebook
11 2 (0.78%) New Relic
11 2 (0.78%) Salesforce.com