Period
to 25 September 2021

The following table provides summary statistics for permanent job vacancies advertised in the City of London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 25 September 2021 with a comparison to the same period in the previous 2 years.

Apache Spark
Central London > City of London
6 months to
25 Sep 2021
Same period 2020 Same period 2019
Rank 114 107 147
Rank change year-on-year -7 +40 +6
Permanent jobs citing Apache Spark 150 98 212
As % of all permanent jobs advertised in the City of London 2.54% 2.58% 2.10%
As % of the Application Platforms category 18.96% 18.60% 14.02%
Number of salaries quoted 139 76 201
10th Percentile £56,000 £49,750 £55,000
25th Percentile £65,000 £60,938 £65,000
Median annual salary (50th Percentile) £77,500 £75,750 £77,500
Median % change year-on-year +2.31% -2.26% -3.13%
75th Percentile £88,750 £103,750 £97,500
90th Percentile £96,500 £106,875 £113,750
Central London median annual salary £77,500 £75,000 £77,500
% change year-on-year +3.33% -3.23% -3.13%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in the City of London with a requirement for application platform skills.

All Application Platform Skills
City of London
Permanent vacancies with a requirement for application platform skills 791 527 1,512
As % of all permanent IT jobs advertised in the City of London 13.42% 13.86% 14.96%
Number of salaries quoted 694 441 1,357
10th Percentile £42,500 £41,250 £33,250
25th Percentile £52,500 £50,000 £47,500
Median annual salary (50th Percentile) £70,000 £65,000 £65,000
Median % change year-on-year +7.69% - +8.33%
75th Percentile £85,000 £80,000 £81,250
90th Percentile £101,250 £100,000 £95,000
Central London median annual salary £68,625 £65,000 £62,500
% change year-on-year +5.58% +4.00% +4.17%

Apache Spark
Job Vacancy Trend in the City of London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in the City of London.

Job vacancy trend for Apache Spark in the City of London

Apache Spark
Salary Trend in the City of London

3-month moving average salary quoted in jobs citing Apache Spark in the City of London.

Salary trend for Apache Spark in the City of London

Apache Spark
Salary Histogram in the City of London

Salary distribution for jobs citing Apache Spark in the City of London over the 6 months to 25 September 2021.

Salary histogram for Apache Spark in the City of London

For the 6 months to 25 September 2021, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the City of London region with a requirement for Apache Spark.

1 97 (64.67%) Big Data
2 93 (62.00%) Java
3 86 (57.33%) AWS
4 83 (55.33%) Hadoop
4 83 (55.33%) Analytics
5 70 (46.67%) Python
6 61 (40.67%) Kafka
7 59 (39.33%) Scala
8 56 (37.33%) SQL
9 53 (35.33%) Data Science
10 50 (33.33%) Data Engineering
10 50 (33.33%) Agile Software Development
11 47 (31.33%) Azure
12 43 (28.67%) Data Pipeline
13 39 (26.00%) Amazon Redshift
13 39 (26.00%) Software Engineering
14 38 (25.33%) DevOps
15 35 (23.33%) NoSQL
16 34 (22.67%) Data Lake
17 33 (22.00%) GCP
17 33 (22.00%) Artificial Intelligence
18 32 (21.33%) Data Warehouse
19 28 (18.67%) Finance
20 27 (18.00%) Data Modelling
21 26 (17.33%) Open Source
21 26 (17.33%) Elasticsearch
22 24 (16.00%) MongoDB
22 24 (16.00%) OO
23 23 (15.33%) Microservices
24 22 (14.67%) Machine Learning

Apache Spark
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 26 (17.33%) Elasticsearch
2 22 (14.67%) Apache
3 17 (11.33%) Apache Airflow
4 3 (2.00%) Apache Pig
5 1 (0.67%) Confluence
5 1 (0.67%) GemFire
Cloud Services
1 86 (57.33%) AWS
2 47 (31.33%) Azure
3 33 (22.00%) GCP
4 12 (8.00%) Amazon EC2
5 11 (7.33%) Power Platform
6 10 (6.67%) Amazon S3
7 9 (6.00%) Snowflake
8 7 (4.67%) Amazon EMR
9 5 (3.33%) Amazon Kinesis
9 5 (3.33%) AWS Data Pipeline
10 4 (2.67%) AWS Lambda
10 4 (2.67%) AWS Step Functions
10 4 (2.67%) Azure Cosmos DB
10 4 (2.67%) Serverless
11 2 (1.33%) Amazon SQS
11 2 (1.33%) Azure Data Factory
11 2 (1.33%) Azure Synapse Analytics
11 2 (1.33%) SaaS
12 1 (0.67%) IBM Cloud
12 1 (0.67%) PaaS
Communications & Networking
1 3 (2.00%) 5G
1 3 (2.00%) Broadband
2 1 (0.67%) Internet
2 1 (0.67%) Network Security
Database & Business Intelligence
1 97 (64.67%) Big Data
2 83 (55.33%) Hadoop
3 39 (26.00%) Amazon Redshift
4 35 (23.33%) NoSQL
5 34 (22.67%) Data Lake
6 32 (21.33%) Data Warehouse
7 24 (16.00%) MongoDB
8 19 (12.67%) BigQuery
8 19 (12.67%) Relational Database
9 16 (10.67%) Apache Hive
10 15 (10.00%) Amazon RDS
11 12 (8.00%) Apache Cassandra
12 11 (7.33%) Power BI
13 9 (6.00%) Tableau
14 7 (4.67%) MySQL
14 7 (4.67%) Spark SQL
15 6 (4.00%) Data Mining
16 5 (3.33%) Electronic Patient Records
16 5 (3.33%) Flink
16 5 (3.33%) Presto
Development Applications
1 11 (7.33%) Git (software)
2 10 (6.67%) JIRA
3 5 (3.33%) Gradle
4 2 (1.33%) ER Studio
4 2 (1.33%) ERwin
4 2 (1.33%) Jenkins
4 2 (1.33%) PowerDesigner
4 2 (1.33%) Subversion
5 1 (0.67%) Artifactory
5 1 (0.67%) Atlassian Bamboo
5 1 (0.67%) Bitbucket
5 1 (0.67%) Maven
5 1 (0.67%) Octopus Deploy
5 1 (0.67%) Sonatype Nexus
5 1 (0.67%) TeamCity
5 1 (0.67%) Vagrant
General
1 28 (18.67%) Finance
2 14 (9.33%) Publishing
3 10 (6.67%) Analytical Skills
4 7 (4.67%) Advertising
5 6 (4.00%) Front Office
5 6 (4.00%) Retail
5 6 (4.00%) Social Skills
6 5 (3.33%) Marketing
6 5 (3.33%) Retail Banking
7 2 (1.33%) Banking
7 2 (1.33%) Law
7 2 (1.33%) Legal
8 1 (0.67%) Documentation Skills
8 1 (0.67%) Financial Institution
8 1 (0.67%) Manufacturing
8 1 (0.67%) Pharmaceutical
8 1 (0.67%) Presentation Skills
Job Titles
1 64 (42.67%) Data Engineer
2 36 (24.00%) Developer
3 33 (22.00%) Big Data Engineer
4 27 (18.00%) Java Engineer
5 21 (14.00%) Senior Data Engineer
6 20 (13.33%) Software Engineer
7 19 (12.67%) Java Software Engineer
8 16 (10.67%) AWS Engineer
8 16 (10.67%) Data Analytics Engineer
8 16 (10.67%) Java Developer
8 16 (10.67%) Scala Developer
9 15 (10.00%) Big Data Software Engineer
10 12 (8.00%) Principal Engineer
11 11 (7.33%) Apache Spark Developer
11 11 (7.33%) Hadoop Developer
12 10 (6.67%) Hadoop Engineer
13 8 (5.33%) Data Scientist
14 7 (4.67%) Senior Developer
15 6 (4.00%) Architect
15 6 (4.00%) Data Architect
Libraries, Frameworks & Software Standards
1 61 (40.67%) Kafka
2 12 (8.00%) AngularJS
2 12 (8.00%) Node.js
3 11 (7.33%) Akka
4 10 (6.67%) RabbitMQ
5 9 (6.00%) React
6 8 (5.33%) RESTful
7 7 (4.67%) HTML
7 7 (4.67%) PySpark
7 7 (4.67%) Spring
8 6 (4.00%) CSS
8 6 (4.00%) HTML5
9 5 (3.33%) TensorFlow
10 4 (2.67%) .NET
11 3 (2.00%) JSON
11 3 (2.00%) Keras
11 3 (2.00%) Redux
12 2 (1.33%) Django
12 2 (1.33%) Middleware
12 2 (1.33%) Spark Streaming
Miscellaneous
1 17 (11.33%) Algorithms
2 16 (10.67%) Greenfield Project
3 9 (6.00%) Data Structures
4 5 (3.33%) Derivative
4 5 (3.33%) Self-Motivation
5 3 (2.00%) Data Centre
5 3 (2.00%) Social Media
6 2 (1.33%) Enterprise Cloud
7 1 (0.67%) Acoustics
7 1 (0.67%) Blog
7 1 (0.67%) Enterprise Software
7 1 (0.67%) Equities
7 1 (0.67%) Mainframe
7 1 (0.67%) Market Risk
7 1 (0.67%) Media Streaming
7 1 (0.67%) Public Cloud
Operating Systems
1 7 (4.67%) Linux
2 4 (2.67%) Unix
Processes & Methodologies
1 83 (55.33%) Analytics
2 53 (35.33%) Data Science
3 50 (33.33%) Agile Software Development
3 50 (33.33%) Data Engineering
4 43 (28.67%) Data Pipeline
5 39 (26.00%) Software Engineering
6 38 (25.33%) DevOps
7 33 (22.00%) Artificial Intelligence
8 27 (18.00%) Data Modelling
9 26 (17.33%) Open Source
10 24 (16.00%) OO
11 23 (15.33%) Microservices
12 22 (14.67%) Machine Learning
13 21 (14.00%) TDD
14 18 (12.00%) Computer Science
15 16 (10.67%) ETL
15 16 (10.67%) Scrum
16 15 (10.00%) Data Transformation
17 14 (9.33%) Data Analytics
18 13 (8.67%) OOD
Programming Languages
1 93 (62.00%) Java
2 70 (46.67%) Python
3 59 (39.33%) Scala
4 56 (37.33%) SQL
5 12 (8.00%) C#
6 11 (7.33%) Kotlin
7 10 (6.67%) JavaScript
8 9 (6.00%) ES6
9 6 (4.00%) Java 8
9 6 (4.00%) PHP
9 6 (4.00%) TypeScript
10 5 (3.33%) R
11 4 (2.67%) Clojure
11 4 (2.67%) Go
12 2 (1.33%) Bash Shell
13 1 (0.67%) C
13 1 (0.67%) C++
13 1 (0.67%) Data Analysis Expressions (DAX)
13 1 (0.67%) Haskell
13 1 (0.67%) PowerShell
Qualifications
1 17 (11.33%) Degree
2 7 (4.67%) Computer Science Degree
3 2 (1.33%) Master's Degree
3 2 (1.33%) SC Cleared
3 2 (1.33%) Security Cleared
4 1 (0.67%) PhD
Quality Assurance & Compliance
1 15 (10.00%) Data Quality
2 8 (5.33%) QA
3 1 (0.67%) GDPR
System Software
1 14 (9.33%) Docker
2 4 (2.67%) HDFS
3 3 (2.00%) Apache Flume
4 1 (0.67%) Apache ZooKeeper
4 1 (0.67%) VirtualBox
Systems Management
1 17 (11.33%) Kubernetes
2 10 (6.67%) Terraform
3 5 (3.33%) Prometheus
4 1 (0.67%) Ansible
4 1 (0.67%) Docker Swarm
Vendors
1 12 (8.00%) Talend
2 11 (7.33%) Cloudera
3 8 (5.33%) Oracle
4 6 (4.00%) Databricks
5 4 (2.67%) Greenplum
6 3 (2.00%) Microsoft
7 2 (1.33%) Alteryx
7 2 (1.33%) IBM
7 2 (1.33%) Informatica
8 1 (0.67%) Google
8 1 (0.67%) Hortonworks
8 1 (0.67%) MicroStrategy
8 1 (0.67%) Teradata
8 1 (0.67%) VMware