Period
to 15 January 2021

The following table provides summary statistics for permanent job vacancies advertised in the City of London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 15 January 2021 with a comparison to the same period in the previous 2 years.

Apache Spark
Central London > City of London
6 months to
15 Jan 2021
Same period 2020 Same period 2019
Rank 98 138 166
Rank change year-on-year +40 +28 -21
Permanent jobs citing Apache Spark 126 200 183
As % of all permanent jobs advertised in the City of London 2.94% 2.18% 1.79%
As % of the Application Platforms category 20.90% 15.13% 10.99%
Number of salaries quoted 117 198 184
Median annual salary £77,500 £75,000 £80,000
Median salary % change year-on-year +3.33% -6.25% +6.67%
10th Percentile £52,500 £56,250 £47,875
90th Percentile £105,000 £112,500 £105,000
Central London median annual salary £77,500 £75,000 £80,000
% change year-on-year +3.33% -6.25% +6.67%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in the City of London with a requirement for application platform skills.

All Application Platform Skills
City of London
Permanent vacancies with a requirement for application platform skills 603 1,322 1,665
As % of all permanent IT jobs advertised in the City of London 14.05% 14.43% 16.29%
Number of salaries quoted 517 1,171 1,515
Median annual salary £70,000 £64,750 £60,000
Median salary % change year-on-year +8.11% +7.92% -
10th Percentile £46,250 £36,250 £33,300
90th Percentile £103,750 £97,500 £91,250
Central London median annual salary £70,000 £65,000 £60,000
% change year-on-year +7.69% +8.33% +8.11%

Apache Spark
Job Vacancy Trend in the City of London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in the City of London.

Job vacancy trend for Apache Spark in the City of London

Apache Spark
Salary Trend in the City of London

3-month moving average salary quoted in jobs citing Apache Spark in the City of London.

Salary trend for Apache Spark in the City of London

Apache Spark
Salary Histogram in the City of London

Salary distribution for jobs citing Apache Spark in the City of London over the 6 months to 15 January 2021.

Salary histogram for Apache Spark in the City of London

For the 6 months to 15 January 2021, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the City of London region with a requirement for Apache Spark.

1 82 (65.08%) Big Data
2 67 (53.17%) Hadoop
2 67 (53.17%) Python
3 57 (45.24%) Java
4 49 (38.89%) Scala
5 48 (38.10%) AWS
6 47 (37.30%) Kafka
7 42 (33.33%) Azure
7 42 (33.33%) SQL
8 37 (29.37%) Analytics
9 34 (26.98%) Apache
10 32 (25.40%) Machine Learning
11 31 (24.60%) Docker
11 31 (24.60%) Data Science
12 29 (23.02%) NoSQL
12 29 (23.02%) ETL
13 27 (21.43%) Finance
13 27 (21.43%) GCP
14 25 (19.84%) Marketing
14 25 (19.84%) Analytical Skills
14 25 (19.84%) Kubernetes
15 23 (18.25%) Artificial Intelligence
16 22 (17.46%) Agile Software Development
17 21 (16.67%) Computer Science
17 21 (16.67%) R
18 20 (15.87%) Git (software)
18 20 (15.87%) JavaScript
18 20 (15.87%) C#
19 19 (15.08%) SQL Server
19 19 (15.08%) Databricks

Apache Spark
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 34 (26.98%) Apache
2 11 (8.73%) Apache Airflow
3 7 (5.56%) Elasticsearch
4 6 (4.76%) SAS
5 1 (0.79%) Apache Pig
Business Applications
1 1 (0.79%) SAP APO
Cloud Services
1 48 (38.10%) AWS
2 42 (33.33%) Azure
3 27 (21.43%) GCP
4 13 (10.32%) Power Platform
5 10 (7.94%) SaaS
6 9 (7.14%) Amazon S3
7 7 (5.56%) Azure Data Factory
8 6 (4.76%) Cloud Computing
8 6 (4.76%) PowerApps
9 4 (3.17%) Amazon EC2
9 4 (3.17%) OpenShift
9 4 (3.17%) Serverless
10 2 (1.59%) Amazon EMR
10 2 (1.59%) Amazon Kinesis
10 2 (1.59%) Amazon SQS
10 2 (1.59%) AWS Lambda
11 1 (0.79%) Amazon CloudWatch
11 1 (0.79%) AWS CloudTrail
11 1 (0.79%) Azure Active Directory
11 1 (0.79%) Azure Machine Learning
Communications & Networking
1 5 (3.97%) Kerberos
2 1 (0.79%) Firewall
2 1 (0.79%) HTTP
2 1 (0.79%) HTTPS
2 1 (0.79%) Network Security
2 1 (0.79%) SSL
Database & Business Intelligence
1 82 (65.08%) Big Data
2 67 (53.17%) Hadoop
3 29 (23.02%) NoSQL
4 19 (15.08%) SQL Server
5 17 (13.49%) Apache Hive
6 15 (11.90%) Data Lake
6 15 (11.90%) Presto
7 13 (10.32%) Amazon Redshift
7 13 (10.32%) Power BI
8 12 (9.52%) Apache Cassandra
8 12 (9.52%) PostgreSQL
9 9 (7.14%) BigQuery
9 9 (7.14%) MongoDB
10 8 (6.35%) MySQL
10 8 (6.35%) Tableau
11 7 (5.56%) Bigtable
11 7 (5.56%) Relational Database
12 6 (4.76%) PowerPivot
12 6 (4.76%) RDBMS
12 6 (4.76%) SQL Server Reporting Services
Development Applications
1 20 (15.87%) Git (software)
2 6 (4.76%) Visual Studio
3 5 (3.97%) TeamCity
4 1 (0.79%) Atlassian Bamboo
4 1 (0.79%) JIRA
4 1 (0.79%) Octopus Deploy
General
1 27 (21.43%) Finance
2 25 (19.84%) Marketing
3 17 (13.49%) Banking
4 7 (5.56%) Advertising
5 3 (2.38%) Front Office
6 1 (0.79%) Legal
6 1 (0.79%) Pharmaceutical
6 1 (0.79%) Public Sector
6 1 (0.79%) Retail Banking
Job Titles
1 41 (32.54%) Developer
2 39 (30.95%) Data Engineer
3 26 (20.63%) Java Developer
4 24 (19.05%) Big Data Engineer
5 20 (15.87%) Java Engineer
6 17 (13.49%) Senior Data Engineer
7 16 (12.70%) Big Data Developer
8 12 (9.52%) Architect
8 12 (9.52%) Data Architect
9 11 (8.73%) Full Stack Developer
9 11 (8.73%) Python Developer
10 10 (7.94%) Big Data Architect
10 10 (7.94%) Java Architect
10 10 (7.94%) Software Engineer
11 9 (7.14%) Big Data Software Engineer
12 8 (6.35%) AWS Engineer
12 8 (6.35%) C# Developer
12 8 (6.35%) Data Scientist
12 8 (6.35%) Scala Developer
13 7 (5.56%) Python Engineer
Libraries, Frameworks & Software Standards
1 47 (37.30%) Kafka
2 15 (11.90%) RESTful
3 14 (11.11%) PySpark
4 13 (10.32%) AngularJS
4 13 (10.32%) HTML
5 12 (9.52%) CSS
5 12 (9.52%) HTML5
5 12 (9.52%) React
6 10 (7.94%) Natural Language Processing
6 10 (7.94%) scikit-learn
7 9 (7.14%) Node.js
7 9 (7.14%) TensorFlow
8 7 (5.56%) Apache Beam
8 7 (5.56%) NumPy
8 7 (5.56%) Pandas
8 7 (5.56%) Spring
9 6 (4.76%) Keras
9 6 (4.76%) PyTorch
9 6 (4.76%) Shiny
9 6 (4.76%) XGBoost
Miscellaneous
1 25 (19.84%) Analytical Skills
2 16 (12.70%) Algorithms
3 5 (3.97%) Mobile App
4 4 (3.17%) Public Cloud
5 1 (0.79%) Cloud Native
5 1 (0.79%) Derivative
5 1 (0.79%) Equities
5 1 (0.79%) Reference Data
5 1 (0.79%) Reinsurance
5 1 (0.79%) Self-Motivation
Operating Systems
1 5 (3.97%) Linux
2 4 (3.17%) Red Hat Enterprise Linux
2 4 (3.17%) Unix
Processes & Methodologies
1 37 (29.37%) Analytics
2 32 (25.40%) Machine Learning
3 31 (24.60%) Data Science
4 29 (23.02%) ETL
5 23 (18.25%) Artificial Intelligence
6 22 (17.46%) Agile Software Development
7 21 (16.67%) Computer Science
8 18 (14.29%) Business Intelligence
8 18 (14.29%) Data Engineering
8 18 (14.29%) DevOps
9 17 (13.49%) Open Source
9 17 (13.49%) Software Engineering
10 16 (12.70%) Data Modelling
11 15 (11.90%) Microservices
12 14 (11.11%) Digital Marketing
13 11 (8.73%) Data Analytics
13 11 (8.73%) Fintech
13 11 (8.73%) MapReduce
13 11 (8.73%) Visualisation
14 10 (7.94%) Decision-Making
Programming Languages
1 67 (53.17%) Python
2 57 (45.24%) Java
3 49 (38.89%) Scala
4 42 (33.33%) SQL
5 21 (16.67%) R
6 20 (15.87%) C#
6 20 (15.87%) JavaScript
7 10 (7.94%) Ruby
8 9 (7.14%) PHP
9 8 (6.35%) ES6
9 8 (6.35%) Kotlin
9 8 (6.35%) TypeScript
10 7 (5.56%) T-SQL
11 6 (4.76%) Data Analysis Expressions (DAX)
11 6 (4.76%) MATLAB
12 2 (1.59%) Java 8
13 1 (0.79%) Bash Shell
13 1 (0.79%) Cypher
13 1 (0.79%) Julia
13 1 (0.79%) Shell Script
Qualifications
1 16 (12.70%) Degree
2 14 (11.11%) Computer Science Degree
3 6 (4.76%) Master's Degree
4 3 (2.38%) PhD
5 1 (0.79%) AWS Certification
5 1 (0.79%) SC Cleared
5 1 (0.79%) Security Cleared
Quality Assurance & Compliance
1 8 (6.35%) QA
2 4 (3.17%) GDPR
3 1 (0.79%) Data Quality
System Software
1 31 (24.60%) Docker
2 2 (1.59%) HDFS
3 1 (0.79%) Apache Flume
Systems Management
1 25 (19.84%) Kubernetes
2 12 (9.52%) Terraform
3 6 (4.76%) Ansible
4 1 (0.79%) Ambari
4 1 (0.79%) Hadoop YARN
4 1 (0.79%) Oozie
4 1 (0.79%) Opscode Chef
4 1 (0.79%) Puppet
4 1 (0.79%) Single Sign-On
Vendors
1 19 (15.08%) Databricks
2 8 (6.35%) Oracle
3 7 (5.56%) Cloudera
4 6 (4.76%) Microsoft
4 6 (4.76%) Splunk
4 6 (4.76%) Talend
5 5 (3.97%) Google
6 4 (3.17%) Red Hat
7 3 (2.38%) Alteryx
8 2 (1.59%) Informatica
9 1 (0.79%) Ab Initio
9 1 (0.79%) Hortonworks
9 1 (0.79%) NVIDIA
9 1 (0.79%) Palantir
9 1 (0.79%) SAP
9 1 (0.79%) TIBCO