Period
to 30 May 2020

The following table provides summary statistics for permanent job vacancies advertised in the City of London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 30 May 2020 with a comparison to the same period in the previous 2 years.

Apache Spark
Central London > City of London
6 months to
30 May 2020
Same period 2019 Same period 2018
Rank 129 161 133
Rank change year-on-year +32 -28 +73
Permanent jobs citing Apache Spark 135 180 252
As % of all permanent jobs advertised in the City of London 1.99% 1.79% 2.37%
As % of the Application Platforms category 14.67% 11.73% 13.81%
Number of salaries quoted 125 165 263
Median annual salary £72,500 £80,000 £80,000
Median salary % change year-on-year -9.38% - +14.29%
10th Percentile £52,500 £49,337 £60,000
90th Percentile £112,500 £116,250 £107,000
Central London median annual salary £74,000 £76,000 £75,000
% change year-on-year -2.63% +1.33% +7.14%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in the City of London with a requirement for application platform skills.

All Application Platform Skills
City of London
Permanent vacancies with a requirement for application platform skills 920 1,534 1,825
As % of all permanent IT jobs advertised in the City of London 13.55% 15.28% 17.17%
Number of salaries quoted 826 1,347 1,626
Median annual salary £65,000 £62,500 £60,000
Median salary % change year-on-year +4.00% +4.17% +4.35%
10th Percentile £38,813 £31,250 £37,500
90th Percentile £92,500 £93,750 £92,500
Central London median annual salary £65,000 £62,500 £60,000
% change year-on-year +4.00% +4.17% +9.09%

Apache Spark
Job Vacancy Trend in the City of London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in the City of London.

Job vacancy trend for Apache Spark in the City of London

Apache Spark
Salary Trend in the City of London

3-month moving average salary quoted in jobs citing Apache Spark in the City of London.

Salary trend for Apache Spark in the City of London

Apache Spark
Salary Histogram in the City of London

Salary distribution for jobs citing Apache Spark in the City of London over the 6 months to 30 May 2020.

Salary histogram for Apache Spark in the City of London

Apache Spark
Job Locations in the City of London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the City of London region over the 6 months to 30 May 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
Moorgate - 8 £65,000 - 1
Apache Spark
Central London

For the 6 months to 30 May 2020, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the City of London region with a requirement for Apache Spark.

1 98 (72.59%) Hadoop
2 84 (62.22%) Big Data
3 83 (61.48%) Python
4 70 (51.85%) Kafka
4 70 (51.85%) AWS
5 68 (50.37%) Scala
5 68 (50.37%) SQL
6 66 (48.89%) Java
7 43 (31.85%) NoSQL
7 43 (31.85%) Kubernetes
8 42 (31.11%) Data Science
9 41 (30.37%) Apache Hive
10 39 (28.89%) Machine Learning
10 39 (28.89%) Artificial Intelligence
11 37 (27.41%) Agile Software Development
11 37 (27.41%) Docker
12 34 (25.19%) Azure
12 34 (25.19%) Finance
13 32 (23.70%) Degree
13 32 (23.70%) Analytics
14 30 (22.22%) ETL
15 29 (21.48%) Computer Science
16 28 (20.74%) Apache
16 28 (20.74%) DevOps
17 27 (20.00%) Git (software)
17 27 (20.00%) JavaScript
17 27 (20.00%) Data Engineering
18 26 (19.26%) Fintech
18 26 (19.26%) Data Warehouse
19 25 (18.52%) Apache Cassandra

Apache Spark
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 28 (20.74%) Apache
2 14 (10.37%) Apache Airflow
3 12 (8.89%) Luigi
4 5 (3.70%) Elasticsearch
5 4 (2.96%) Apache Pig
6 3 (2.22%) WebSphere
Applications
1 1 (0.74%) Chatbot
Cloud Services
1 70 (51.85%) AWS
2 34 (25.19%) Azure
3 25 (18.52%) GCP
4 17 (12.59%) Amazon S3
5 16 (11.85%) Amazon EMR
6 8 (5.93%) Amazon Kinesis
7 6 (4.44%) Serverless
8 4 (2.96%) Amazon EC2
9 3 (2.22%) Amazon CloudWatch
9 3 (2.22%) AWS IoT
9 3 (2.22%) AWS Lambda
9 3 (2.22%) Cloud Computing
9 3 (2.22%) SaaS
10 2 (1.48%) Amazon GuardDuty
10 2 (1.48%) AWS CloudTrail
10 2 (1.48%) Azure Data Factory
11 1 (0.74%) Amazon SQS
11 1 (0.74%) Datadog
11 1 (0.74%) GitHub
11 1 (0.74%) IBM Cloud
Communications & Networking
1 3 (2.22%) AMQP
1 3 (2.22%) HTTP
1 3 (2.22%) TCP/IP
2 2 (1.48%) FTP
Database & Business Intelligence
1 98 (72.59%) Hadoop
2 84 (62.22%) Big Data
3 43 (31.85%) NoSQL
4 41 (30.37%) Apache Hive
5 26 (19.26%) Data Warehouse
6 25 (18.52%) Apache Cassandra
7 16 (11.85%) MongoDB
8 15 (11.11%) Amazon Redshift
9 14 (10.37%) PostgreSQL
10 12 (8.89%) Relational Database
11 8 (5.93%) Data Lake
11 8 (5.93%) HBase
11 8 (5.93%) RDBMS
12 7 (5.19%) HDInsight
13 6 (4.44%) Amazon Athena
13 6 (4.44%) BigQuery
13 6 (4.44%) Electronic Patient Records
13 6 (4.44%) SQL Server
13 6 (4.44%) Sqoop
14 5 (3.70%) Impala
Development Applications
1 27 (20.00%) Git (software)
2 11 (8.15%) Jenkins
3 6 (4.44%) JUnit
4 3 (2.22%) JIRA
4 3 (2.22%) Octopus Deploy
4 3 (2.22%) TeamCity
4 3 (2.22%) Visual Studio Team System
5 2 (1.48%) Maven
6 1 (0.74%) Atlassian Bamboo
6 1 (0.74%) GoCD
6 1 (0.74%) Vagrant
General
1 34 (25.19%) Finance
2 15 (11.11%) Marketing
3 9 (6.67%) Banking
4 8 (5.93%) Retail
5 5 (3.70%) Games
6 4 (2.96%) Advertising
7 3 (2.22%) Automotive
8 2 (1.48%) Financial Institution
8 2 (1.48%) Investment Banking
8 2 (1.48%) Law
8 2 (1.48%) Retail Banking
9 1 (0.74%) Front Office
9 1 (0.74%) Manufacturing
9 1 (0.74%) Pharmaceutical
9 1 (0.74%) Public Sector
9 1 (0.74%) Publishing
Job Titles
1 57 (42.22%) Data Engineer
2 48 (35.56%) Developer
3 25 (18.52%) Java Developer
4 19 (14.07%) Scala Developer
5 12 (8.89%) Full Stack Developer
5 12 (8.89%) Senior Data Engineer
6 11 (8.15%) Apache Spark Developer
7 10 (7.41%) Data Scientist
8 9 (6.67%) Big Data Engineer
9 8 (5.93%) Java Scala Developer
9 8 (5.93%) JavaScript Developer
10 7 (5.19%) Automation Developer
10 7 (5.19%) AWS Engineer
11 6 (4.44%) Senior Developer
12 5 (3.70%) Java Engineer
13 4 (2.96%) Kubernetes Engineer
13 4 (2.96%) Senior Data Scientist
13 4 (2.96%) Senior Java Developer
13 4 (2.96%) Site Engineer
13 4 (2.96%) Software Engineer
Libraries, Frameworks & Software Standards
1 70 (51.85%) Kafka
2 18 (13.33%) RESTful
3 14 (10.37%) Node.js
4 12 (8.89%) React
5 11 (8.15%) AngularJS
6 9 (6.67%) Apache Beam
6 9 (6.67%) CSS
6 9 (6.67%) HTML
6 9 (6.67%) HTML5
6 9 (6.67%) Spring
7 8 (5.93%) RabbitMQ
8 7 (5.19%) GraphQL
8 7 (5.19%) Spring Boot
9 6 (4.44%) Elastic Stack
10 5 (3.70%) Akka
10 5 (3.70%) PySpark
10 5 (3.70%) REST
10 5 (3.70%) Spark Streaming
11 3 (2.22%) Java EE
11 3 (2.22%) Redux
Miscellaneous
1 14 (10.37%) Greenfield Project
2 12 (8.89%) Algorithms
3 11 (8.15%) Mobile App
4 10 (7.41%) Cloud Native
5 5 (3.70%) Hedge funds
6 4 (2.96%) Social Media
7 3 (2.22%) Analytical Skills
7 3 (2.22%) Market Risk
8 2 (1.48%) Blog
8 2 (1.48%) Management Information System
8 2 (1.48%) Self-Motivation
9 1 (0.74%) Acoustics
9 1 (0.74%) Client/Server
9 1 (0.74%) Data Structures
9 1 (0.74%) Derivative
9 1 (0.74%) Equities
9 1 (0.74%) Foreign Exchange (FX)
9 1 (0.74%) Marketing Technology
9 1 (0.74%) Robotics
9 1 (0.74%) Video\Live Streaming
Operating Systems
1 7 (5.19%) Linux
2 4 (2.96%) Windows
3 1 (0.74%) Windows Server
Processes & Methodologies
1 42 (31.11%) Data Science
2 39 (28.89%) Artificial Intelligence
2 39 (28.89%) Machine Learning
3 37 (27.41%) Agile Software Development
4 32 (23.70%) Analytics
5 30 (22.22%) ETL
6 29 (21.48%) Computer Science
7 28 (20.74%) DevOps
8 27 (20.00%) Data Engineering
9 26 (19.26%) Fintech
10 24 (17.78%) Microservices
10 24 (17.78%) TDD
11 17 (12.59%) OO
12 15 (11.11%) Open Source
13 13 (9.63%) Software Engineering
14 12 (8.89%) Project Delivery
14 12 (8.89%) Test Automation
15 11 (8.15%) Continuous Integration
16 10 (7.41%) Problem-Solving
17 9 (6.67%) Data Modelling
Programming Languages
1 83 (61.48%) Python
2 68 (50.37%) Scala
2 68 (50.37%) SQL
3 66 (48.89%) Java
4 27 (20.00%) JavaScript
5 15 (11.11%) C#
6 14 (10.37%) Kotlin
7 13 (9.63%) Go
8 12 (8.89%) PHP
8 12 (8.89%) TypeScript
9 9 (6.67%) ES6
9 9 (6.67%) R
10 8 (5.93%) C++
11 7 (5.19%) Java 8
12 5 (3.70%) Shell Script
13 3 (2.22%) T-SQL
14 2 (1.48%) PL/SQL
14 2 (1.48%) PowerShell
15 1 (0.74%) Bash Shell
15 1 (0.74%) C
Qualifications
1 32 (23.70%) Degree
2 23 (17.04%) Computer Science Degree
3 1 (0.74%) Numerate Degree
3 1 (0.74%) PhD
Quality Assurance & Compliance
1 3 (2.22%) Data Quality
System Software
1 37 (27.41%) Docker
2 7 (5.19%) HDFS
3 5 (3.70%) Apache Flume
4 2 (1.48%) Apache ZooKeeper
5 1 (0.74%) VirtualBox
Systems Management
1 43 (31.85%) Kubernetes
2 12 (8.89%) Ansible
3 10 (7.41%) Terraform
4 5 (3.70%) Opscode Chef
4 5 (3.70%) Puppet
4 5 (3.70%) Salt
5 4 (2.96%) Hadoop YARN
5 4 (2.96%) Prometheus
6 2 (1.48%) Oozie
7 1 (0.74%) Docker Swarm
7 1 (0.74%) Nagios
7 1 (0.74%) Zipkin
Vendors
1 16 (11.85%) Google
2 9 (6.67%) Oracle
3 5 (3.70%) Cloudera
3 5 (3.70%) Databricks
3 5 (3.70%) Hortonworks
4 4 (2.96%) Microsoft
5 3 (2.22%) MapR
5 3 (2.22%) Solace
5 3 (2.22%) Splunk
6 1 (0.74%) Foundry
6 1 (0.74%) IBM
6 1 (0.74%) Palantir
6 1 (0.74%) Tableau
6 1 (0.74%) Talend
6 1 (0.74%) Teradata
6 1 (0.74%) TOWER Software