Period
to 24 July 2021

The following table provides summary statistics for permanent job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 24 July 2021 with a comparison to the same period in the previous 2 years.

Apache Spark
England > London
6 months to
24 Jul 2021
Same period 2020 Same period 2019
Rank 148 169 134
Rank change year-on-year +21 -35 +10
Permanent jobs citing Apache Spark 1,066 628 1,481
As % of all permanent jobs advertised in London 2.37% 2.13% 2.57%
As % of the Application Platforms category 16.08% 13.91% 16.07%
Number of salaries quoted 695 453 1,255
10th Percentile £47,500 £53,750 £55,000
25th Percentile £57,500 £63,750 £63,750
Median annual salary (50th Percentile) £77,500 £80,000 £80,000
Median % change year-on-year -3.13% - +6.67%
75th Percentile £92,500 £97,000 £97,500
90th Percentile £115,000 £124,500 £115,000
England median annual salary £76,650 £76,000 £75,000
% change year-on-year +0.86% +1.33% +7.14%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Permanent vacancies with a requirement for application platform skills 6,630 4,516 9,215
As % of all permanent IT jobs advertised in London 14.72% 15.35% 16.00%
Number of salaries quoted 4,285 3,618 7,573
10th Percentile £41,250 £38,675 £35,000
25th Percentile £50,000 £48,250 £46,250
Median annual salary (50th Percentile) £65,000 £65,000 £62,500
Median % change year-on-year - +4.00% +4.17%
75th Percentile £85,000 £83,000 £80,000
90th Percentile £100,000 £100,000 £95,000
England median annual salary £55,000 £53,000 £52,000
% change year-on-year +3.77% +1.92% +4.00%

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Salary Trend in London

3-month moving average salary quoted in jobs citing Apache Spark in London.

Salary trend for Apache Spark in London

Apache Spark
Salary Histogram in London

Salary distribution for jobs citing Apache Spark in London over the 6 months to 24 July 2021.

Salary histogram for Apache Spark in London

Apache Spark
Job Locations in London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the London region over the 6 months to 24 July 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +23 172 £80,000 +8.11% 37
South London -23 7 £60,000 -20.00% 1
East London +2 1 £77,500 -18.42% 6
West London -38 1 - - 8
Apache Spark
England

For the 6 months to 24 July 2021, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the London region with a requirement for Apache Spark.

1 734 (68.86%) Python
2 589 (55.25%) AWS
3 571 (53.56%) SQL
4 545 (51.13%) Big Data
5 542 (50.84%) Hadoop
6 479 (44.93%) Analytics
7 443 (41.56%) Java
8 441 (41.37%) Kubernetes
9 411 (38.56%) Kafka
10 398 (37.34%) Docker
11 388 (36.40%) Agile Software Development
12 371 (34.80%) Data Science
13 365 (34.24%) Data Pipeline
14 351 (32.93%) GCP
14 351 (32.93%) Azure
15 349 (32.74%) Software Engineering
16 317 (29.74%) Finance
17 295 (27.67%) ETL
17 295 (27.67%) Scala
18 273 (25.61%) NoSQL
18 273 (25.61%) Data Engineering
19 262 (24.58%) Machine Learning
20 256 (24.02%) Artificial Intelligence
21 209 (19.61%) Apache
22 208 (19.51%) Amazon Redshift
23 194 (18.20%) DevOps
24 189 (17.73%) Google
25 183 (17.17%) Data Warehouse
26 182 (17.07%) Microservices
27 176 (16.51%) Data Modelling

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 209 (19.61%) Apache
2 139 (13.04%) Apache Airflow
3 124 (11.63%) Elasticsearch
4 37 (3.47%) Luigi
5 33 (3.10%) Apache Pig
6 24 (2.25%) Jupyter
7 19 (1.78%) Oracle Application Server
8 11 (1.03%) Oracle Workflow
8 11 (1.03%) SAS
9 7 (0.66%) Tomcat
10 6 (0.56%) Confluence
11 4 (0.38%) Oracle SOA Suite
12 3 (0.28%) DataSynapse
12 3 (0.28%) JBoss
12 3 (0.28%) Mule
13 2 (0.19%) GemFire
13 2 (0.19%) WebSphere
14 1 (0.094%) CMS
14 1 (0.094%) IIS
14 1 (0.094%) WebLogic
Applications
1 8 (0.75%) Spreadsheet
2 5 (0.47%) Adobe Illustrator
2 5 (0.47%) Fireworks
2 5 (0.47%) Photoshop
3 3 (0.28%) Microsoft Excel
3 3 (0.28%) SPSS
4 2 (0.19%) Gephi
4 2 (0.19%) Microsoft PowerPoint
5 1 (0.094%) Microsoft Office
Business Applications
1 2 (0.19%) Oracle Procure-to-Pay
1 2 (0.19%) SAP R/3
Cloud Services
1 589 (55.25%) AWS
2 351 (32.93%) Azure
2 351 (32.93%) GCP
3 157 (14.73%) Snowflake
4 122 (11.44%) Amazon EMR
5 115 (10.79%) Serverless
6 74 (6.94%) Amazon Kinesis
7 71 (6.66%) IBM Cloud
8 62 (5.82%) AWS Lambda
9 57 (5.35%) Amazon S3
10 55 (5.16%) Power Platform
11 39 (3.66%) OpenShift
12 38 (3.56%) Cloud Functions
13 36 (3.38%) Amazon SageMaker
14 34 (3.19%) Amazon EC2
15 33 (3.10%) PaaS
16 31 (2.91%) Google Compute Engine
17 30 (2.81%) Google App Engine
17 30 (2.81%) Google Container Engine
18 25 (2.35%) Cloud Computing
Communications & Networking
1 22 (2.06%) Internet
2 19 (1.78%) Wireless
3 9 (0.84%) SMTP
4 8 (0.75%) SMS
5 7 (0.66%) HTTP
6 6 (0.56%) DNS
6 6 (0.56%) IMAP
7 5 (0.47%) Broadband
8 4 (0.38%) HTTPS
8 4 (0.38%) Sourcefire
9 3 (0.28%) 5G
9 3 (0.28%) DHCP
9 3 (0.28%) Firewall
9 3 (0.28%) IPv4
9 3 (0.28%) IPv6
9 3 (0.28%) SNMP
9 3 (0.28%) SSH
9 3 (0.28%) SSL
9 3 (0.28%) TCP/IP
10 2 (0.19%) VPN
Database & Business Intelligence
1 545 (51.13%) Big Data
2 542 (50.84%) Hadoop
3 273 (25.61%) NoSQL
4 208 (19.51%) Amazon Redshift
5 183 (17.17%) Data Warehouse
6 139 (13.04%) Data Lake
7 134 (12.57%) PostgreSQL
8 129 (12.10%) Apache Hive
9 91 (8.54%) Relational Database
10 90 (8.44%) MongoDB
11 89 (8.35%) Amazon Athena
12 88 (8.26%) Neo4j
13 76 (7.13%) Flink
14 73 (6.85%) BigQuery
15 70 (6.57%) Electronic Patient Records
16 68 (6.38%) Apache Cassandra
17 55 (5.16%) Power BI
18 54 (5.07%) HBase
19 52 (4.88%) Tableau
20 49 (4.60%) Amazon DynamoDB
Development Applications
1 155 (14.54%) Jenkins
2 130 (12.20%) Git (software)
3 76 (7.13%) Atlassian Bamboo
3 76 (7.13%) Vagrant
4 43 (4.03%) JIRA
5 34 (3.19%) Gradle
6 29 (2.72%) GitLab
7 23 (2.16%) Oracle APEX
8 19 (1.78%) Maven
9 15 (1.41%) Cypress.io
10 14 (1.31%) TeamCity
11 13 (1.22%) Bitbucket
12 11 (1.03%) MLflow
13 10 (0.94%) Sonatype Nexus
14 9 (0.84%) IBM UrbanCode
15 8 (0.75%) Eclipse
15 8 (0.75%) IntelliJ
15 8 (0.75%) Postman
15 8 (0.75%) SoapUI
15 8 (0.75%) SonarQube
General
1 317 (29.74%) Finance
2 148 (13.88%) Social Skills
3 129 (12.10%) Analytical Skills
4 88 (8.26%) Banking
5 71 (6.66%) Front Office
6 68 (6.38%) Retail
7 41 (3.85%) Marketing
8 32 (3.00%) Games
9 29 (2.72%) Telecoms
10 24 (2.25%) Advertising
11 21 (1.97%) Law
12 19 (1.78%) Investment Banking
13 11 (1.03%) Inclusion and Diversity
14 10 (0.94%) Czech Language
15 9 (0.84%) Retail Banking
16 7 (0.66%) Presentation Skills
16 7 (0.66%) Publishing
17 5 (0.47%) Billing
17 5 (0.47%) Pharmaceutical
18 4 (0.38%) Legal
Job Titles
1 385 (36.12%) Data Engineer
2 170 (15.95%) Developer
3 147 (13.79%) Software Engineer
4 109 (10.23%) Big Data Engineer
5 106 (9.94%) Architect
6 102 (9.57%) Data Scientist
7 95 (8.91%) Senior Data Engineer
8 92 (8.63%) Data Architect
9 81 (7.60%) Java Developer
10 76 (7.13%) AWS Engineer
11 69 (6.47%) Platform Engineer
12 62 (5.82%) Java Engineer
13 55 (5.16%) Python Engineer
14 36 (3.38%) Big Data Architect
15 35 (3.28%) Java Software Engineer
16 34 (3.19%) Platform Architect
16 34 (3.19%) Senior Developer
17 31 (2.91%) Cloud Engineer
18 30 (2.81%) Azure Engineer
19 29 (2.72%) Apache Spark Developer
Libraries, Frameworks & Software Standards
1 411 (38.56%) Kafka
2 111 (10.41%) React
3 110 (10.32%) Spark Streaming
4 96 (9.01%) Spring
5 89 (8.35%) AngularJS
6 78 (7.32%) Node.js
7 73 (6.85%) scikit-learn
8 70 (6.57%) HTML
9 68 (6.38%) Pandas
10 67 (6.29%) PySpark
11 59 (5.53%) NumPy
11 59 (5.53%) Spring Boot
12 58 (5.44%) CSS
13 56 (5.25%) Apache Beam
14 51 (4.78%) RESTful
14 51 (4.78%) TensorFlow
15 46 (4.32%) Web Services
16 45 (4.22%) .NET
17 43 (4.03%) HTML5
18 42 (3.94%) Akka
Miscellaneous
1 143 (13.41%) Algorithms
2 74 (6.94%) User Experience
3 72 (6.75%) Data Structures
4 58 (5.44%) Hedge funds
5 57 (5.35%) Equities
6 41 (3.85%) Hybrid Cloud
7 37 (3.47%) Greenfield Project
8 33 (3.10%) Derivative
9 25 (2.35%) Foreign Exchange (FX)
10 19 (1.78%) Social Network
10 19 (1.78%) Virtual Team
11 17 (1.59%) Cloud Native
12 15 (1.41%) Self-Motivation
13 14 (1.31%) iPhone
13 14 (1.31%) Public Cloud
13 14 (1.31%) SWIFT
14 13 (1.22%) Fixed Income
15 12 (1.13%) Internet of Things
15 12 (1.13%) Management Information System
16 11 (1.03%) Social Media
Operating Systems
1 76 (7.13%) Linux
2 73 (6.85%) Unix
3 14 (1.31%) Red Hat Enterprise Linux
3 14 (1.31%) Windows
4 6 (0.56%) Apple iOS
5 1 (0.094%) Windows Server
5 1 (0.094%) Windows Server 2012
Processes & Methodologies
1 479 (44.93%) Analytics
2 388 (36.40%) Agile Software Development
3 371 (34.80%) Data Science
4 365 (34.24%) Data Pipeline
5 349 (32.74%) Software Engineering
6 295 (27.67%) ETL
7 273 (25.61%) Data Engineering
8 262 (24.58%) Machine Learning
9 256 (24.02%) Artificial Intelligence
10 194 (18.20%) DevOps
11 182 (17.07%) Microservices
12 176 (16.51%) Data Modelling
13 138 (12.95%) Visualisation
14 132 (12.38%) Business Intelligence
15 130 (12.20%) Continuous Integration
16 129 (12.10%) Data Analytics
17 124 (11.63%) Open Source
18 109 (10.23%) Data Architecture
19 108 (10.13%) Computer Science
20 107 (10.04%) Data Munging
Programming Languages
1 734 (68.86%) Python
2 571 (53.56%) SQL
3 443 (41.56%) Java
4 295 (27.67%) Scala
5 147 (13.79%) JavaScript
6 88 (8.26%) C#
7 64 (6.00%) TypeScript
8 58 (5.44%) R
9 39 (3.66%) Go
10 38 (3.56%) Java 8
11 34 (3.19%) PL/SQL
12 29 (2.72%) C++
12 29 (2.72%) PowerShell
12 29 (2.72%) Shell Script
13 28 (2.63%) Bash Shell
14 23 (2.16%) C
14 23 (2.16%) Kotlin
15 18 (1.69%) F#
16 17 (1.59%) ES6
17 15 (1.41%) PHP
Qualifications
1 137 (12.85%) Degree
2 68 (6.38%) Computer Science Degree
3 21 (1.97%) PhD
4 19 (1.78%) Security Cleared
5 7 (0.66%) AWS Certification
5 7 (0.66%) Postgraduate
6 6 (0.56%) SC Cleared
7 5 (0.47%) DV Cleared
7 5 (0.47%) Master's Degree
8 4 (0.38%) Cisco Certification
8 4 (0.38%) CISSP
8 4 (0.38%) TOGAF Certification
9 3 (0.28%) Microsoft Certification
10 2 (0.19%) CCNA
10 2 (0.19%) CEH
10 2 (0.19%) GIAC
10 2 (0.19%) Linux+ Certification
10 2 (0.19%) MCSE
10 2 (0.19%) Network+ Certification
10 2 (0.19%) SSCP
Quality Assurance & Compliance
1 62 (5.82%) QA
2 56 (5.25%) Data Quality
3 25 (2.35%) GDPR
4 2 (0.19%) MiFID
System Software
1 398 (37.34%) Docker
2 76 (7.13%) VirtualBox
3 26 (2.44%) HDFS
4 9 (0.84%) Apache ZooKeeper
5 5 (0.47%) Apache Flume
6 2 (0.19%) Platform Symphony
7 1 (0.094%) Active Directory
7 1 (0.094%) Hyper-V
7 1 (0.094%) Virtual Machines
7 1 (0.094%) VMware Infrastructure
Systems Management
1 441 (41.37%) Kubernetes
2 131 (12.29%) Terraform
3 114 (10.69%) Ansible
4 76 (7.13%) Docker Swarm
5 32 (3.00%) Istio
6 16 (1.50%) Prometheus
7 15 (1.41%) Hadoop YARN
8 13 (1.22%) Oozie
9 7 (0.66%) Ambari
9 7 (0.66%) Kibana
9 7 (0.66%) Puppet
10 5 (0.47%) Kubeflow
11 3 (0.28%) logstash
11 3 (0.28%) Tivoli
12 2 (0.19%) Grafana
12 2 (0.19%) Opscode Chef
13 1 (0.094%) Failover Clustering
13 1 (0.094%) ITRS Geneos
13 1 (0.094%) Mesos
13 1 (0.094%) Zipkin
Vendors
1 189 (17.73%) Google
2 108 (10.13%) Cloudera
3 104 (9.76%) Databricks
4 102 (9.57%) Talend
5 100 (9.38%) Hortonworks
6 94 (8.82%) Oracle
7 74 (6.94%) Informatica
8 72 (6.75%) IBM
9 52 (4.88%) Microsoft
10 20 (1.88%) Greenplum
10 20 (1.88%) Splunk
11 14 (1.31%) Red Hat
11 14 (1.31%) Samsung
12 13 (1.22%) Cisco
12 13 (1.22%) Qubit
13 10 (0.94%) Teradata
14 9 (0.84%) Sybase
15 7 (0.66%) Apple
15 7 (0.66%) Dell
15 7 (0.66%) ServiceNow