Job Vacancy Trend Salary Trend Salary Histogram Skill Sets
Period
to 19 January 2022

The following table provides summary statistics for permanent job vacancies advertised in the City of London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 19 January 2022 with a comparison to the same period in the previous 2 years.

Apache Spark
Central London > City of London
6 months to
19 Jan 2022		 Same period 2021 Same period 2020
Rank 139 98 140
Rank change year-on-year -41 +42 +38
Permanent jobs citing Apache Spark 116 128 200
As % of all permanent jobs advertised in the City of London 1.87% 3.00% 2.21%
As % of the Application Platforms category 16.29% 21.44% 15.31%
Number of salaries quoted 100 110 192
10th Percentile £53,750 £54,750 £55,125
25th Percentile £70,938 £65,000 £61,250
Median annual salary (50th Percentile) £80,000 £80,000 £75,000
Median % change year-on-year - +6.67% -6.25%
75th Percentile £99,375 £96,250 £97,500
90th Percentile £110,000 £105,375 £112,500
Central London median annual salary £77,500 £80,000 £75,000
% change year-on-year -3.13% +6.67% -3.23%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in the City of London with a requirement for application platform skills.

All Application Platform Skills
City of London
Permanent vacancies with a requirement for application platform skills 712 597 1,306
As % of all permanent IT jobs advertised in the City of London 11.46% 13.97% 14.41%
Number of salaries quoted 619 520 1,156
10th Percentile £38,000 £46,250 £36,250
25th Percentile £50,500 £55,000 £46,250
Median annual salary (50th Percentile) £70,000 £70,000 £65,000
Median % change year-on-year - +7.69% +8.33%
75th Percentile £87,500 £87,500 £80,125
90th Percentile £103,750 £103,875 £97,500
Central London median annual salary £67,500 £70,000 £65,000
% change year-on-year -3.57% +7.69% +8.33%

Apache Spark
Job Vacancy Trend in the City of London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in the City of London.

Job vacancy trend for Apache Spark in the City of London

Apache Spark
Salary Trend in the City of London

3-month moving average salary quoted in jobs citing Apache Spark in the City of London.

Salary trend for Apache Spark in the City of London

Apache Spark
Salary Histogram in the City of London

Salary distribution for jobs citing Apache Spark in the City of London over the 6 months to 19 January 2022.

Salary histogram for Apache Spark in the City of London

For the 6 months to 19 January 2022, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the City of London region with a requirement for Apache Spark.

1 70 (60.34%) Java
2 68 (58.62%) AWS
3 66 (56.90%) Big Data
3 66 (56.90%) Analytics
3 66 (56.90%) Python
4 65 (56.03%) Scala
5 61 (52.59%) Data Science
6 55 (47.41%) Kafka
7 47 (40.52%) SQL
8 42 (36.21%) Azure
9 39 (33.62%) Hadoop
9 39 (33.62%) Agile Software Development
10 34 (29.31%) Data Engineering
11 31 (26.72%) Data Pipeline
12 29 (25.00%) Data Modelling
13 28 (24.14%) Software Engineering
13 28 (24.14%) DevOps
13 28 (24.14%) Machine Learning
14 26 (22.41%) GCP
14 26 (22.41%) CI/CD
15 25 (21.55%) NoSQL
16 23 (19.83%) Elasticsearch
16 23 (19.83%) Finance
16 23 (19.83%) Snowflake
17 22 (18.97%) Data Warehouse
18 19 (16.38%) Apache
18 19 (16.38%) Publishing
18 19 (16.38%) TDD
19 18 (15.52%) Artificial Intelligence
19 18 (15.52%) ETL

Apache Spark
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 23 (19.83%) Elasticsearch
2 19 (16.38%) Apache
3 10 (8.62%) Apache Airflow
4 1 (0.86%) Apache Pig
4 1 (0.86%) Jupyter
Applications
1 1 (0.86%) Microsoft Office
Business Applications
1 1 (0.86%) Oracle Procure-to-Pay
1 1 (0.86%) SAP R/3
Cloud Services
1 68 (58.62%) AWS
2 42 (36.21%) Azure
3 26 (22.41%) GCP
4 23 (19.83%) Snowflake
5 9 (7.76%) Amazon EMR
6 8 (6.90%) Amazon S3
6 8 (6.90%) Azure Cosmos DB
6 8 (6.90%) Azure Data Factory
7 7 (6.03%) Matillion
8 6 (5.17%) Power Platform
9 5 (4.31%) Azure Machine Learning
10 4 (3.45%) AWS CloudFormation
11 3 (2.59%) Serverless
12 2 (1.72%) AWS Lambda
12 2 (1.72%) AWS Step Functions
12 2 (1.72%) Datadog
13 1 (0.86%) Amazon Kinesis
13 1 (0.86%) Google App Engine
13 1 (0.86%) Google Compute Engine
13 1 (0.86%) OpenShift
Communications & Networking
1 2 (1.72%) Kerberos
2 1 (0.86%) Internet
Database & Business Intelligence
1 66 (56.90%) Big Data
2 39 (33.62%) Hadoop
3 25 (21.55%) NoSQL
4 22 (18.97%) Data Warehouse
5 16 (13.79%) Data Lake
5 16 (13.79%) MongoDB
6 14 (12.07%) Apache Cassandra
7 12 (10.34%) Amazon Redshift
8 10 (8.62%) Tableau
9 7 (6.03%) BigQuery
9 7 (6.03%) Electronic Patient Records
9 7 (6.03%) Flink
10 6 (5.17%) MySQL
10 6 (5.17%) Power BI
11 4 (3.45%) Apache Hive
11 4 (3.45%) Metadata
11 4 (3.45%) Presto
12 3 (2.59%) Looker
13 2 (1.72%) Amazon Athena
13 2 (1.72%) Blockchain
Development Applications
1 9 (7.76%) Git (software)
2 4 (3.45%) Gradle
2 4 (3.45%) Jenkins
3 3 (2.59%) Subversion
4 2 (1.72%) Artifactory
4 2 (1.72%) RStudio
4 2 (1.72%) Sonatype Nexus
5 1 (0.86%) Atlassian Bamboo
5 1 (0.86%) Spinnaker
5 1 (0.86%) Vagrant
General
1 23 (19.83%) Finance
2 19 (16.38%) Publishing
3 9 (7.76%) Social Skills
4 8 (6.90%) Law
5 7 (6.03%) Advertising
5 7 (6.03%) Analytical Skills
6 5 (4.31%) Presentation Skills
7 3 (2.59%) Pharmaceutical
8 2 (1.72%) Retail
9 1 (0.86%) Documentation Skills
9 1 (0.86%) Financial Institution
9 1 (0.86%) Games
9 1 (0.86%) Influencing Skills
9 1 (0.86%) Marketing
9 1 (0.86%) Retail Banking
Job Titles
1 60 (51.72%) Data Engineer
2 22 (18.97%) Senior Data Engineer
3 19 (16.38%) Developer
4 12 (10.34%) Architect
4 12 (10.34%) Data Architect
4 12 (10.34%) Principal Engineer
5 9 (7.76%) Big Data Engineer
6 8 (6.90%) Lead Architect
6 8 (6.90%) Lead Data Architect
6 8 (6.90%) Scala Developer
6 8 (6.90%) Scala Engineer
7 7 (6.03%) Apache Spark Developer
7 7 (6.03%) Data Scientist
7 7 (6.03%) Hadoop Developer
8 6 (5.17%) Cloud Engineer
8 6 (5.17%) Java Developer
8 6 (5.17%) Machine Learning Engineer
8 6 (5.17%) Senior Developer
9 5 (4.31%) Senior Scala Engineer
10 4 (3.45%) Azure Engineer
Libraries, Frameworks & Software Standards
1 55 (47.41%) Kafka
2 11 (9.48%) TensorFlow
3 7 (6.03%) Akka
4 5 (4.31%) AngularJS
4 5 (4.31%) PySpark
5 4 (3.45%) React
6 3 (2.59%) .NET
6 3 (2.59%) Apache Beam
6 3 (2.59%) Keras
6 3 (2.59%) Node.js
6 3 (2.59%) Pandas
6 3 (2.59%) Redux
6 3 (2.59%) Spring
7 2 (1.72%) LDAP
7 2 (1.72%) PyTorch
7 2 (1.72%) Spark Streaming
7 2 (1.72%) XGBoost
8 1 (0.86%) MLlib
8 1 (0.86%) scikit-learn
8 1 (0.86%) Spring Boot
Miscellaneous
1 12 (10.34%) Algorithms
2 9 (7.76%) Greenfield Project
3 4 (3.45%) Data Structures
3 4 (3.45%) Replication
4 2 (1.72%) Derivative
4 2 (1.72%) Equities
4 2 (1.72%) Grid Computing
5 1 (0.86%) CUDA
5 1 (0.86%) Enterprise Software
5 1 (0.86%) Fixed Income
5 1 (0.86%) Hedge funds
5 1 (0.86%) Media Streaming
5 1 (0.86%) Mobile App
5 1 (0.86%) Reference Data
5 1 (0.86%) Self-Motivation
Operating Systems
1 6 (5.17%) Linux
2 4 (3.45%) Unix
3 1 (0.86%) Android
3 1 (0.86%) Apple iOS
Processes & Methodologies
1 66 (56.90%) Analytics
2 61 (52.59%) Data Science
3 39 (33.62%) Agile Software Development
4 34 (29.31%) Data Engineering
5 31 (26.72%) Data Pipeline
6 29 (25.00%) Data Modelling
7 28 (24.14%) DevOps
7 28 (24.14%) Machine Learning
7 28 (24.14%) Software Engineering
8 26 (22.41%) CI/CD
9 19 (16.38%) TDD
10 18 (15.52%) Artificial Intelligence
10 18 (15.52%) ETL
11 15 (12.93%) Computer Science
11 15 (12.93%) Decision-Making
11 15 (12.93%) Microservices
12 14 (12.07%) Task Automation
13 13 (11.21%) Informatics
13 13 (11.21%) Scrum
14 12 (10.34%) Pair Programming
Programming Languages
1 70 (60.34%) Java
2 66 (56.90%) Python
3 65 (56.03%) Scala
4 47 (40.52%) SQL
5 12 (10.34%) R
6 10 (8.62%) C#
7 6 (5.17%) Clojure
8 4 (3.45%) Go
8 4 (3.45%) JavaScript
9 3 (2.59%) Bash
9 3 (2.59%) ES6
10 1 (0.86%) C
10 1 (0.86%) C++
10 1 (0.86%) Java 8
10 1 (0.86%) Kotlin
10 1 (0.86%) PL/SQL
10 1 (0.86%) Ruby
10 1 (0.86%) Shell Script
10 1 (0.86%) T-SQL
Qualifications
1 10 (8.62%) Degree
2 2 (1.72%) PhD
3 1 (0.86%) Computer Science Degree
3 1 (0.86%) Master's Degree
Quality Assurance & Compliance
1 11 (9.48%) Data Quality
2 6 (5.17%) GDPR
3 1 (0.86%) QA
System Software
1 10 (8.62%) Docker
2 2 (1.72%) Samba
3 1 (0.86%) HDFS
3 1 (0.86%) VirtualBox
Systems Management
1 9 (7.76%) Kubernetes
1 9 (7.76%) Terraform
2 5 (4.31%) Ansible
3 2 (1.72%) Ambari
3 2 (1.72%) Hadoop YARN
3 2 (1.72%) vRealize Orchestrator
4 1 (0.86%) Docker Swarm
4 1 (0.86%) Grafana
4 1 (0.86%) Prometheus
Vendors
1 13 (11.21%) Cloudera
2 8 (6.90%) Databricks
2 8 (6.90%) Talend
3 7 (6.03%) Oracle
4 4 (3.45%) Hortonworks
4 4 (3.45%) Salesforce.com
5 3 (2.59%) Google
6 2 (1.72%) Informatica
6 2 (1.72%) Microsoft
6 2 (1.72%) VMware
7 1 (0.86%) Alteryx
7 1 (0.86%) ESRI
7 1 (0.86%) IBM
7 1 (0.86%) PeopleSoft
7 1 (0.86%) SAP
7 1 (0.86%) UiPath