Period
to 16 October 2021

The following table provides summary statistics for permanent job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 16 October 2021 with a comparison to the same period in the previous 2 years.

Apache Spark
England > London
6 months to
16 Oct 2021
Same period 2020 Same period 2019
Rank 175 164 147
Rank change year-on-year -11 -17 +27
Permanent jobs citing Apache Spark 1,090 416 1,310
As % of all permanent jobs advertised in London 2.18% 2.15% 2.39%
As % of the Application Platforms category 15.01% 14.11% 15.53%
Number of salaries quoted 651 324 1,101
10th Percentile £53,750 £51,625 £55,000
25th Percentile £62,750 £62,500 £65,000
Median annual salary (50th Percentile) £80,000 £80,000 £80,000
75th Percentile £96,250 £100,000 £95,000
90th Percentile £115,000 £120,000 £115,000
England median annual salary £80,000 £72,500 £73,593
% change year-on-year +10.34% -1.49% +1.51%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Permanent vacancies with a requirement for application platform skills 7,262 2,949 8,435
As % of all permanent IT jobs advertised in London 14.50% 15.22% 15.39%
Number of salaries quoted 4,474 2,475 6,948
10th Percentile £38,750 £41,250 £32,500
25th Percentile £49,000 £51,250 £45,000
Median annual salary (50th Percentile) £65,000 £67,500 £62,500
Median % change year-on-year -3.70% +8.00% +4.17%
75th Percentile £85,000 £85,000 £80,000
90th Percentile £102,500 £103,750 £95,000
England median annual salary £55,000 £55,000 £50,000
% change year-on-year - +10.00% -

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Salary Trend in London

3-month moving average salary quoted in jobs citing Apache Spark in London.

Salary trend for Apache Spark in London

Apache Spark
Salary Histogram in London

Salary distribution for jobs citing Apache Spark in London over the 6 months to 16 October 2021.

Salary histogram for Apache Spark in London

Apache Spark
Job Locations in London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the London region over the 6 months to 16 October 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London -20 158 £77,500 +3.33% 40
South London -16 11 £60,000 -4.00% 1
East London -14 10 £95,000 +46.15% 8
West London -83 3 - - 11
Apache Spark
England

For the 6 months to 16 October 2021, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the London region with a requirement for Apache Spark.

1 774 (71.01%) Python
2 583 (53.49%) AWS
3 576 (52.84%) SQL
4 528 (48.44%) Big Data
5 515 (47.25%) Analytics
6 502 (46.06%) Hadoop
7 474 (43.49%) Java
8 453 (41.56%) Kafka
9 423 (38.81%) Data Science
10 389 (35.69%) Agile Software Development
11 363 (33.30%) Scala
12 352 (32.29%) Kubernetes
13 347 (31.83%) Azure
14 345 (31.65%) Docker
15 343 (31.47%) Software Engineering
16 339 (31.10%) Finance
16 339 (31.10%) GCP
17 306 (28.07%) Data Pipeline
18 305 (27.98%) Machine Learning
19 269 (24.68%) NoSQL
20 266 (24.40%) ETL
21 249 (22.84%) Data Engineering
22 231 (21.19%) Data Warehouse
23 223 (20.46%) Data Modelling
24 215 (19.72%) DevOps
25 207 (18.99%) Apache
25 207 (18.99%) Artificial Intelligence
26 188 (17.25%) Social Skills
27 180 (16.51%) Data Lake
27 180 (16.51%) Amazon Redshift

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 207 (18.99%) Apache
2 143 (13.12%) Elasticsearch
3 138 (12.66%) Apache Airflow
4 35 (3.21%) Jupyter
5 32 (2.94%) Apache Pig
6 25 (2.29%) Luigi
7 21 (1.93%) Confluence
8 19 (1.74%) Oracle Application Server
9 6 (0.55%) Oracle SOA Suite
9 6 (0.55%) Oracle Workflow
10 5 (0.46%) SAS
11 3 (0.28%) Mule
12 2 (0.18%) Apache Solr
13 1 (0.092%) CMS
13 1 (0.092%) DataSynapse
13 1 (0.092%) GemFire
13 1 (0.092%) IIS
13 1 (0.092%) JBoss
13 1 (0.092%) Tomcat
13 1 (0.092%) WebLogic
Applications
1 8 (0.73%) Spreadsheet
2 5 (0.46%) Adobe Illustrator
2 5 (0.46%) Fireworks
2 5 (0.46%) Photoshop
3 2 (0.18%) Microsoft Excel
4 1 (0.092%) Chatbot
4 1 (0.092%) Microsoft PowerPoint
4 1 (0.092%) SPSS
Business Applications
1 2 (0.18%) Oracle Procure-to-Pay
1 2 (0.18%) SAP R/3
2 1 (0.092%) SAP Oil and Gas
Cloud Services
1 583 (53.49%) AWS
2 347 (31.83%) Azure
3 339 (31.10%) GCP
4 134 (12.29%) Snowflake
5 115 (10.55%) Serverless
6 111 (10.18%) Amazon EMR
7 99 (9.08%) Amazon Kinesis
8 92 (8.44%) Amazon S3
9 70 (6.42%) AWS Lambda
10 64 (5.87%) Power Platform
11 49 (4.50%) OpenShift
12 37 (3.39%) Amazon EC2
12 37 (3.39%) Amazon SageMaker
13 32 (2.94%) IBM Cloud
13 32 (2.94%) PaaS
14 31 (2.84%) Azure Data Factory
15 28 (2.57%) GitHub
16 25 (2.29%) Amazon SQS
17 24 (2.20%) IaaS
18 23 (2.11%) Azure Synapse Analytics
Communications & Networking
1 25 (2.29%) Wireless
2 15 (1.38%) Internet
3 11 (1.01%) HTTP
4 10 (0.92%) DNS
5 9 (0.83%) HTTPS
5 9 (0.83%) SMS
6 8 (0.73%) DHCP
6 8 (0.73%) IPv4
6 8 (0.73%) IPv6
6 8 (0.73%) SMTP
6 8 (0.73%) SNMP
6 8 (0.73%) SSH
6 8 (0.73%) TCP/IP
7 4 (0.37%) Broadband
7 4 (0.37%) Cisco IPT
7 4 (0.37%) Sourcefire
8 2 (0.18%) 5G
8 2 (0.18%) Kerberos
8 2 (0.18%) Network Security
9 1 (0.092%) SSL
Database & Business Intelligence
1 528 (48.44%) Big Data
2 502 (46.06%) Hadoop
3 269 (24.68%) NoSQL
4 231 (21.19%) Data Warehouse
5 180 (16.51%) Amazon Redshift
5 180 (16.51%) Data Lake
6 155 (14.22%) Apache Hive
7 137 (12.57%) PostgreSQL
8 105 (9.63%) MongoDB
9 96 (8.81%) Relational Database
10 91 (8.35%) Apache Cassandra
11 89 (8.17%) BigQuery
11 89 (8.17%) Flink
12 79 (7.25%) HBase
13 70 (6.42%) Amazon DynamoDB
14 64 (5.87%) Tableau
15 63 (5.78%) Power BI
16 57 (5.23%) Electronic Patient Records
17 56 (5.14%) MySQL
18 50 (4.59%) Amazon Athena
Development Applications
1 135 (12.39%) Git (software)
2 124 (11.38%) Jenkins
3 39 (3.58%) JIRA
4 37 (3.39%) Atlassian Bamboo
5 34 (3.12%) Gradle
5 34 (3.12%) Vagrant
6 32 (2.94%) GitLab
7 22 (2.02%) Oracle APEX
8 19 (1.74%) Bitbucket
9 16 (1.47%) CircleCI
9 16 (1.47%) Maven
10 15 (1.38%) TeamCity
11 13 (1.19%) Cypress.io
12 11 (1.01%) MLflow
13 10 (0.92%) Eclipse
14 9 (0.83%) IBM UrbanCode
14 9 (0.83%) IntelliJ
14 9 (0.83%) Postman
14 9 (0.83%) SoapUI
15 8 (0.73%) JUnit
General
1 339 (31.10%) Finance
2 188 (17.25%) Social Skills
3 131 (12.02%) Analytical Skills
4 94 (8.62%) Retail
5 87 (7.98%) Banking
6 67 (6.15%) Front Office
7 45 (4.13%) Marketing
8 33 (3.03%) Games
9 27 (2.48%) Investment Banking
9 27 (2.48%) Telecoms
10 23 (2.11%) Advertising
11 22 (2.02%) Publishing
12 21 (1.93%) Presentation Skills
13 20 (1.83%) Inclusion and Diversity
14 19 (1.74%) Law
15 15 (1.38%) Public Sector
16 12 (1.10%) Retail Banking
17 11 (1.01%) Pharmaceutical
18 9 (0.83%) Czech Language
19 7 (0.64%) Manufacturing
Job Titles
1 406 (37.25%) Data Engineer
2 128 (11.74%) Developer
2 128 (11.74%) Software Engineer
3 124 (11.38%) Data Scientist
4 117 (10.73%) Architect
5 110 (10.09%) Senior Data Engineer
6 101 (9.27%) Data Architect
7 95 (8.72%) Big Data Engineer
8 61 (5.60%) Java Engineer
9 56 (5.14%) Java Developer
10 49 (4.50%) Python Engineer
11 46 (4.22%) AWS Engineer
12 43 (3.94%) Scala Engineer
13 41 (3.76%) Senior Developer
14 40 (3.67%) Big Data Architect
15 36 (3.30%) Platform Engineer
16 31 (2.84%) Azure Engineer
17 29 (2.66%) Machine Learning Engineer
18 27 (2.48%) Scala Developer
19 26 (2.39%) Senior Data Scientist
Libraries, Frameworks & Software Standards
1 453 (41.56%) Kafka
2 105 (9.63%) React
3 90 (8.26%) Spring
4 76 (6.97%) PySpark
5 74 (6.79%) AngularJS
6 72 (6.61%) Apache Beam
7 69 (6.33%) TensorFlow
8 66 (6.06%) Node.js
9 63 (5.78%) scikit-learn
10 60 (5.50%) HTML
11 57 (5.23%) Pandas
12 55 (5.05%) Spark Streaming
13 50 (4.59%) CSS
14 45 (4.13%) Spring Boot
15 44 (4.04%) Akka
15 44 (4.04%) REST
16 43 (3.94%) Keras
17 41 (3.76%) NumPy
18 39 (3.58%) .NET
18 39 (3.58%) RESTful
Miscellaneous
1 169 (15.50%) Algorithms
2 86 (7.89%) User Experience
3 75 (6.88%) Data Structures
4 44 (4.04%) Virtual Team
5 42 (3.85%) Hedge funds
6 41 (3.76%) Equities
7 40 (3.67%) Greenfield Project
8 32 (2.94%) Derivative
9 30 (2.75%) Self-Motivation
10 26 (2.39%) Foreign Exchange (FX)
11 25 (2.29%) Cloud Native
12 22 (2.02%) Hybrid Cloud
13 21 (1.93%) iPhone
13 21 (1.93%) SWIFT
14 18 (1.65%) Social Network
15 15 (1.38%) Management Information System
16 14 (1.28%) Public Cloud
17 12 (1.10%) Enterprise Software
18 10 (0.92%) Fixed Income
19 8 (0.73%) Life Science
Operating Systems
1 92 (8.44%) Unix
2 87 (7.98%) Linux
3 21 (1.93%) Red Hat Enterprise Linux
4 16 (1.47%) Windows
5 6 (0.55%) Apple iOS
6 3 (0.28%) DC/OS
7 1 (0.092%) Windows Server
7 1 (0.092%) Windows Server 2012
Processes & Methodologies
1 515 (47.25%) Analytics
2 423 (38.81%) Data Science
3 389 (35.69%) Agile Software Development
4 343 (31.47%) Software Engineering
5 306 (28.07%) Data Pipeline
6 305 (27.98%) Machine Learning
7 266 (24.40%) ETL
8 249 (22.84%) Data Engineering
9 223 (20.46%) Data Modelling
10 215 (19.72%) DevOps
11 207 (18.99%) Artificial Intelligence
12 167 (15.32%) Microservices
13 148 (13.58%) Business Intelligence
14 137 (12.57%) Visualisation
15 136 (12.48%) Data Analytics
16 128 (11.74%) Computer Science
17 103 (9.45%) Open Source
18 102 (9.36%) Data Management
19 98 (8.99%) Data Architecture
20 97 (8.90%) Fintech
Programming Languages
1 774 (71.01%) Python
2 576 (52.84%) SQL
3 474 (43.49%) Java
4 363 (33.30%) Scala
5 139 (12.75%) JavaScript
6 79 (7.25%) R
7 70 (6.42%) C#
8 63 (5.78%) TypeScript
9 48 (4.40%) Go
10 47 (4.31%) Shell Script
11 42 (3.85%) Bash Shell
12 39 (3.58%) PowerShell
13 36 (3.30%) PL/SQL
14 34 (3.12%) Java 8
15 29 (2.66%) C++
16 23 (2.11%) C
16 23 (2.11%) MATLAB
17 18 (1.65%) F#
18 17 (1.56%) Kotlin
19 15 (1.38%) Clojure
Qualifications
1 142 (13.03%) Degree
2 64 (5.87%) Computer Science Degree
3 32 (2.94%) PhD
4 12 (1.10%) Master's Degree
4 12 (1.10%) Security Cleared
5 9 (0.83%) SC Cleared
6 8 (0.73%) AWS Certification
7 7 (0.64%) Postgraduate
8 4 (0.37%) (ISC)2 CCSP
8 4 (0.37%) Cisco Certification
8 4 (0.37%) CISSP
8 4 (0.37%) TOGAF Certification
9 2 (0.18%) CCA
9 2 (0.18%) CCSP
10 1 (0.092%) CTC Cleared
10 1 (0.092%) DV Cleared
10 1 (0.092%) Microsoft Certification
Quality Assurance & Compliance
1 74 (6.79%) Data Quality
2 69 (6.33%) QA
3 28 (2.57%) GDPR
4 2 (0.18%) MiFID
System Software
1 345 (31.65%) Docker
2 34 (3.12%) VirtualBox
3 30 (2.75%) HDFS
4 8 (0.73%) Apache ZooKeeper
5 6 (0.55%) Apache Flume
6 1 (0.092%) Active Directory
6 1 (0.092%) Hyper-V
6 1 (0.092%) Platform Symphony
6 1 (0.092%) Virtual Machines
6 1 (0.092%) VMware Infrastructure
Systems Management
1 352 (32.29%) Kubernetes
2 85 (7.80%) Terraform
3 78 (7.16%) Ansible
4 34 (3.12%) Docker Swarm
5 33 (3.03%) Istio
6 25 (2.29%) Oozie
7 14 (1.28%) Hadoop YARN
8 10 (0.92%) Prometheus
9 8 (0.73%) Control-M
10 5 (0.46%) Opscode Chef
11 4 (0.37%) Mesos
11 4 (0.37%) Puppet
12 3 (0.28%) Ambari
12 3 (0.28%) Apache Brooklyn
12 3 (0.28%) Kibana
12 3 (0.28%) Knative
12 3 (0.28%) logstash
12 3 (0.28%) Marathon
12 3 (0.28%) Tivoli
13 1 (0.092%) Kubeflow
Vendors
1 167 (15.32%) Google
2 130 (11.93%) Databricks
3 101 (9.27%) Oracle
4 75 (6.88%) Cloudera
5 68 (6.24%) Microsoft
6 64 (5.87%) Hortonworks
7 57 (5.23%) Talend
8 53 (4.86%) Informatica
9 34 (3.12%) IBM
10 21 (1.93%) Red Hat
10 21 (1.93%) Samsung
10 21 (1.93%) Splunk
11 20 (1.83%) Greenplum
12 12 (1.10%) Apple
13 11 (1.01%) StreamSets
14 9 (0.83%) Cisco
15 8 (0.73%) Adobe
15 8 (0.73%) Monetate
15 8 (0.73%) SAP
15 8 (0.73%) Teradata