Period
to 20 January 2019

The following table provides summary statistics for permanent job vacancies advertised in the City of London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 20 January 2019 with a comparison to the same period in the previous 2 years.

Apache Spark
Central London > City of London
6 months to
20 Jan 2019
Same period 2018 Same period 2017
Rank 173 149 210
Rank change year-on-year -24 +61 +28
Permanent jobs citing Apache Spark 178 219 141
As % of all permanent IT jobs advertised in the City of London 1.75% 2.10% 1.32%
As % of the Application Platforms category 10.85% 11.78% 7.50%
Number of salaries quoted 171 204 128
City of London median annual salary £80,000 £77,500 £71,250
Median salary % change year-on-year +3.23% +8.77% +1.79%
10th Percentile £47,500 £51,625 £51,750
90th Percentile £105,000 £115,000 £106,500
Central London median annual salary £77,500 £75,000 £71,250
% change year-on-year +3.33% +5.26% +5.56%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in the City of London with a requirement for application platform skills.

All Application Platform Skills
City of London
Permanent vacancies with a requirement for application platform skills 1,641 1,859 1,879
As % of all permanent IT jobs advertised in the City of London 16.17% 17.79% 17.63%
Number of salaries quoted 1,484 1,693 1,732
City of London median annual salary £60,000 £60,000 £57,000
Median salary % change year-on-year - +5.26% +3.64%
10th Percentile £32,500 £36,250 £33,750
90th Percentile £91,250 £87,500 £86,250
Central London median annual salary £60,000 £55,000 £55,000
% change year-on-year +9.09% - -

Apache Spark
Job Vacancy Trend in the City of London

Job postings citing Apache Spark as a percentage of all IT jobs advertised in the City of London.

Job vacancy trend for Apache Spark in the City of London

Apache Spark
Salary Trend in the City of London

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in the City of London.

Salary trend for Apache Spark in the City of London

Apache Spark
Salary Histogram in the City of London

The salary distribution of IT jobs citing Apache Spark in the City of London over the 6 months to 20 January 2019.

Salary histogram for Apache Spark in the City of London

Apache Spark
Job Locations in the City of London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the City of London region over the 6 months to 20 January 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
London Bridge - 1 £87,500 -
Apache Spark
Central London

For the 6 months to 20 January 2019, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the City of London region with a requirement for Apache Spark.

1 109 (61.24%) Hadoop
2 100 (56.18%) Big Data
3 88 (49.44%) Java
4 87 (48.88%) Kafka
5 86 (48.31%) Python
6 78 (43.82%) Scala
7 65 (36.52%) Agile Software Development
8 62 (34.83%) SQL
9 59 (33.15%) Amazon AWS
10 56 (31.46%) Apache Cassandra
11 53 (29.78%) Analytics
12 52 (29.21%) Machine Learning
13 51 (28.65%) Data Science
14 48 (26.97%) Finance
15 45 (25.28%) NoSQL
16 43 (24.16%) Linux
17 42 (23.60%) Microsoft Azure
17 42 (23.60%) DevOps
18 41 (23.03%) Software Engineering
19 37 (20.79%) Apache Hive
20 36 (20.22%) MongoDB
21 34 (19.10%) ETL
22 33 (18.54%) Open Source
23 31 (17.42%) Algorithms
24 29 (16.29%) C++
25 25 (14.04%) Microservices
25 25 (14.04%) Computer Science
26 24 (13.48%) Artificial Intelligence
26 24 (13.48%) Mesos
26 24 (13.48%) Google Cloud Platform

Apache Spark
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 21 (11.80%) Apache
2 20 (11.24%) Elasticsearch
3 7 (3.93%) Apache Pig
3 7 (3.93%) SAS
4 3 (1.69%) WebSphere
4 3 (1.69%) WebSphere Application Server
5 1 (0.56%) Apache Airflow
5 1 (0.56%) Apache Solr
5 1 (0.56%) Confluence
5 1 (0.56%) IIS
Applications
1 1 (0.56%) GNU Octave
Cloud Services
1 59 (33.15%) Amazon AWS
2 42 (23.60%) Microsoft Azure
3 24 (13.48%) Google Cloud Platform
4 8 (4.49%) GitHub
5 7 (3.93%) Cloud Computing
6 6 (3.37%) Amazon EMR
7 5 (2.81%) Azure Data Factory
7 5 (2.81%) Serverless
8 4 (2.25%) Azure Functions
8 4 (2.25%) Data Lake Analytics
9 3 (1.69%) Amazon EC2
9 3 (1.69%) Azure Machine Learning
10 2 (1.12%) Amazon CloudWatch
10 2 (1.12%) Amazon S3
10 2 (1.12%) Google App Engine
11 1 (0.56%) Amazon Kinesis
11 1 (0.56%) AWS Lambda
11 1 (0.56%) Cortana Intelligence Suite
Communications & Networking
1 8 (4.49%) AMQP
2 1 (0.56%) DNS
2 1 (0.56%) Firewall
2 1 (0.56%) HTTP
2 1 (0.56%) NX-OS
2 1 (0.56%) VLAN
Database & Business Intelligence
1 109 (61.24%) Hadoop
2 100 (56.18%) Big Data
3 56 (31.46%) Apache Cassandra
4 45 (25.28%) NoSQL
5 37 (20.79%) Apache Hive
6 36 (20.22%) MongoDB
7 23 (12.92%) HBase
8 15 (8.43%) Amazon Redshift
8 15 (8.43%) Data Warehouse
9 14 (7.87%) Data Lake
10 13 (7.30%) PostgreSQL
10 13 (7.30%) Sqoop
11 12 (6.74%) Spark SQL
12 9 (5.06%) SQL Server
13 6 (3.37%) MySQL
14 5 (2.81%) Azure SQL Data Warehouse
14 5 (2.81%) Cosmos DB
14 5 (2.81%) Impala
14 5 (2.81%) Power BI
15 4 (2.25%) Apache Ignite
Development Applications
1 22 (12.36%) Jenkins
2 18 (10.11%) Git (software)
3 9 (5.06%) JIRA
4 5 (2.81%) GitLab
5 4 (2.25%) JUnit
5 4 (2.25%) Octopus Deploy
5 4 (2.25%) SpecFlow
5 4 (2.25%) TeamCity
5 4 (2.25%) Travis CI
5 4 (2.25%) tSQLt
5 4 (2.25%) Visual Studio Team System
6 3 (1.69%) IBM UrbanCode
6 3 (1.69%) Jasmine
6 3 (1.69%) Maven
6 3 (1.69%) Mocha
6 3 (1.69%) Mockito
6 3 (1.69%) Sonatype Nexus
6 3 (1.69%) TestNG
7 1 (0.56%) Hudson
7 1 (0.56%) SQL Server Management Studio (SSMS)
General
1 48 (26.97%) Finance
2 22 (12.36%) Banking
3 13 (7.30%) Retail Banking
4 6 (3.37%) Pharmaceutical
4 6 (3.37%) Telecoms
5 5 (2.81%) Investment Banking
5 5 (2.81%) Retail
6 4 (2.25%) Advertising
6 4 (2.25%) Games
6 4 (2.25%) Marketing
7 3 (1.69%) Financial Institution
7 3 (1.69%) Front Office
7 3 (1.69%) Law
8 2 (1.12%) Automotive
8 2 (1.12%) Billing
9 1 (0.56%) Publishing
Job Titles
1 84 (47.19%) Developer
2 37 (20.79%) Data Engineer
3 32 (17.98%) Java Developer
3 32 (17.98%) Scala Developer
4 28 (15.73%) Data Scientist
5 20 (11.24%) Python Engineer
6 19 (10.67%) Apache Spark Developer
6 19 (10.67%) C++ Developer
7 16 (8.99%) Big Data Engineer
7 16 (8.99%) Java Scala Developer
8 12 (6.74%) AWS Developer
8 12 (6.74%) Senior Data Scientist
9 11 (6.18%) Hadoop Developer
10 10 (5.62%) Architect
11 9 (5.06%) Python Developer
12 8 (4.49%) DevOps Engineer
13 6 (3.37%) AWS Engineer
13 6 (3.37%) Big Data Developer
13 6 (3.37%) Data Analytics Engineer
13 6 (3.37%) Java Hadoop Developer
Libraries, Frameworks & Software Standards
1 87 (48.88%) Kafka
2 23 (12.92%) AngularJS
3 18 (10.11%) Pandas
4 13 (7.30%) NumPy
5 12 (6.74%) Natural Language Processing
5 12 (6.74%) RabbitMQ
6 11 (6.18%) SciPy
6 11 (6.18%) Spark Streaming
6 11 (6.18%) Spring
6 11 (6.18%) TensorFlow
7 7 (3.93%) Akka
7 7 (3.93%) React
8 6 (3.37%) Apache Commons
8 6 (3.37%) XGBoost
9 5 (2.81%) Hibernate
9 5 (2.81%) MLlib
9 5 (2.81%) scikit-learn
9 5 (2.81%) Spring MVC
10 4 (2.25%) Velocity
10 4 (2.25%) Web Services
Miscellaneous
1 31 (17.42%) Algorithms
2 19 (10.67%) Analytical Skills
2 19 (10.67%) Greenfield Project
3 18 (10.11%) Derivative
3 18 (10.11%) Fintech
4 17 (9.55%) Life Science
5 15 (8.43%) Self-Motivation
6 12 (6.74%) Data Structures
7 11 (6.18%) Market Risk
7 11 (6.18%) Spread Betting
8 6 (3.37%) Social Network
9 5 (2.81%) Clustering
10 3 (1.69%) Enterprise Software
11 2 (1.12%) Enterprise Cloud
11 2 (1.12%) Mobile App
11 2 (1.12%) Smart Meter
12 1 (0.56%) e-Learning
12 1 (0.56%) Smartphone
12 1 (0.56%) Social Media
12 1 (0.56%) Wiki
Operating Systems
1 43 (24.16%) Linux
2 9 (5.06%) Unix
3 1 (0.56%) CentOS
3 1 (0.56%) Mac OS
3 1 (0.56%) Ubuntu
3 1 (0.56%) Windows
3 1 (0.56%) Windows Server
3 1 (0.56%) Windows Server 2012
Processes & Methodologies
1 65 (36.52%) Agile Software Development
2 53 (29.78%) Analytics
3 52 (29.21%) Machine Learning
4 51 (28.65%) Data Science
5 42 (23.60%) DevOps
6 41 (23.03%) Software Engineering
7 34 (19.10%) ETL
8 33 (18.54%) Open Source
9 25 (14.04%) Computer Science
9 25 (14.04%) Microservices
10 24 (13.48%) Artificial Intelligence
11 22 (12.36%) SDLC
12 21 (11.80%) Data Analytics
12 21 (11.80%) Deep Learning
12 21 (11.80%) Mathematics
13 20 (11.24%) Data Engineering
14 19 (10.67%) Data Analysis
14 19 (10.67%) TDD
15 17 (9.55%) OO
15 17 (9.55%) Quantitative Finance
Programming Languages
1 88 (49.44%) Java
2 86 (48.31%) Python
3 78 (43.82%) Scala
4 62 (34.83%) SQL
5 29 (16.29%) C++
6 18 (10.11%) R
7 12 (6.74%) JavaScript
8 11 (6.18%) C#
8 11 (6.18%) Java 8
9 10 (5.62%) Kotlin
10 7 (3.93%) Ruby
11 6 (3.37%) Bash Shell
12 5 (2.81%) MATLAB
12 5 (2.81%) PowerShell
12 5 (2.81%) T-SQL
13 4 (2.25%) Shell Script
13 4 (2.25%) U-SQL
14 3 (1.69%) CoffeeScript
14 3 (1.69%) F#
15 2 (1.12%) C
Qualifications
1 12 (6.74%) Degree
2 5 (2.81%) PhD
3 3 (1.69%) Computer Science Degree
4 2 (1.12%) Master's Degree
4 2 (1.12%) Numerate Degree
4 2 (1.12%) Postgraduate
Quality Assurance & Compliance
1 4 (2.25%) Data Quality
2 3 (1.69%) GDPR
3 2 (1.12%) QA
4 1 (0.56%) MiFID
System Software
1 23 (12.92%) Docker
2 17 (9.55%) HDFS
3 8 (4.49%) Apache Flume
4 3 (1.69%) Apache ZooKeeper
5 1 (0.56%) Firmware
5 1 (0.56%) VMware ESXi
5 1 (0.56%) VMware Infrastructure
Systems Management
1 24 (13.48%) Kubernetes
1 24 (13.48%) Mesos
2 16 (8.99%) Oozie
3 15 (8.43%) Prometheus
4 11 (6.18%) Kibana
4 11 (6.18%) Terraform
5 8 (4.49%) Ansible
6 7 (3.93%) Hadoop YARN
7 3 (1.69%) Graphite
7 3 (1.69%) logstash
8 2 (1.12%) Grafana
9 1 (0.56%) Ambari
9 1 (0.56%) vCenter Server
Vendors
1 18 (10.11%) Google
2 17 (9.55%) Cloudera
3 16 (8.99%) Hortonworks
4 12 (6.74%) Tableau
5 11 (6.18%) Microsoft
5 11 (6.18%) Oracle
6 9 (5.06%) IBM
7 7 (3.93%) MapR
8 5 (2.81%) SAP
9 4 (2.25%) Greenplum
10 3 (1.69%) Informatica
10 3 (1.69%) Talend
10 3 (1.69%) Teradata
11 2 (1.12%) AppDynamics
11 2 (1.12%) New Relic
12 1 (0.56%) Ab Initio
12 1 (0.56%) Databricks
12 1 (0.56%) Nimble Storage
12 1 (0.56%) Palo Alto
12 1 (0.56%) Solace