Job Vacancy Trend Salary Trend Salary Histogram Job Locations Skill Sets
Period
to 18 May 2021

The following table provides summary statistics for permanent job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 18 May 2021 with a comparison to the same period in the previous 2 years.

Apache Spark
England > London
6 months to
18 May 2021		 Same period 2020 Same period 2019
Rank 147 164 137
Rank change year-on-year +17 -27 +1
Permanent jobs citing Apache Spark 799 869 1,526
As % of all permanent jobs advertised in London 2.38% 2.19% 2.59%
As % of the Application Platforms category 16.53% 14.53% 15.83%
Number of salaries quoted 618 659 1,271
10th Percentile £46,250 £52,500 £55,000
25th Percentile £57,313 £62,500 £62,500
Median annual salary (50th Percentile) £77,000 £77,500 £80,000
Median % change year-on-year -0.65% -3.13% +6.67%
75th Percentile £90,000 £93,750 £96,250
90th Percentile £115,750 £115,000 £111,250
England median annual salary £75,000 £75,000 £75,000

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Permanent vacancies with a requirement for application platform skills 4,835 5,982 9,638
As % of all permanent IT jobs advertised in London 14.38% 15.05% 16.34%
Number of salaries quoted 3,586 4,873 7,837
10th Percentile £41,250 £36,250 £33,750
25th Percentile £50,000 £46,250 £45,000
Median annual salary (50th Percentile) £65,000 £62,500 £62,500
Median % change year-on-year +4.00% - +4.17%
75th Percentile £82,500 £80,000 £80,000
90th Percentile £97,500 £95,000 £95,000
England median annual salary £55,000 £52,500 £50,000
% change year-on-year +4.76% +5.00% -

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Salary Trend in London

3-month moving average salary quoted in jobs citing Apache Spark in London.

Salary trend for Apache Spark in London

Apache Spark
Salary Histogram in London

Salary distribution for jobs citing Apache Spark in London over the 6 months to 18 May 2021.

Salary histogram for Apache Spark in London

Apache Spark
Job Locations in London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the London region over the 6 months to 18 May 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year		 Matching
Permanent
IT Job Ads		 Median Salary
Past 6 Months		 Median Salary
% Change
on Same Period
Last Year		 Live
Job
Vacancies
Central London +39 161 £80,000 +7.38% 43
North London +26 3 £60,000 -27.27%
South London -9 2 £67,500 -14.29% 2
West London +15 1 £87,500 +20.69% 10
East London +6 1 £77,500 -8.82% 5
Apache Spark
England

For the 6 months to 18 May 2021, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the London region with a requirement for Apache Spark.

1 523 (65.46%) Python
2 444 (55.57%) Big Data
3 437 (54.69%) AWS
4 424 (53.07%) Hadoop
5 397 (49.69%) SQL
6 343 (42.93%) Java
6 343 (42.93%) Kafka
7 311 (38.92%) Kubernetes
8 308 (38.55%) Agile Software Development
9 302 (37.80%) Analytics
10 296 (37.05%) Docker
11 288 (36.05%) GCP
12 260 (32.54%) Data Pipeline
13 256 (32.04%) Data Science
13 256 (32.04%) Software Engineering
14 246 (30.79%) Azure
15 239 (29.91%) ETL
16 219 (27.41%) Scala
17 212 (26.53%) Finance
17 212 (26.53%) Data Engineering
18 202 (25.28%) NoSQL
19 188 (23.53%) Artificial Intelligence
20 184 (23.03%) Machine Learning
21 174 (21.78%) Google
22 167 (20.90%) Microservices
23 156 (19.52%) Amazon Redshift
24 136 (17.02%) DevOps
25 126 (15.77%) Continuous Integration
26 121 (15.14%) Jenkins
27 120 (15.02%) Data Warehouse

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 119 (14.89%) Apache
2 104 (13.02%) Apache Airflow
3 61 (7.63%) Elasticsearch
4 37 (4.63%) Luigi
5 34 (4.26%) Apache Pig
6 17 (2.13%) Oracle Application Server
7 15 (1.88%) SAS
8 9 (1.13%) Oracle Workflow
9 6 (0.75%) Jupyter
9 6 (0.75%) Tomcat
10 4 (0.50%) Tornado
11 3 (0.38%) Confluence
11 3 (0.38%) DataSynapse
11 3 (0.38%) Mule
12 2 (0.25%) JBoss
12 2 (0.25%) WebSphere
13 1 (0.13%) Apache Synapse
13 1 (0.13%) GemFire
Applications
1 9 (1.13%) SPSS
2 7 (0.88%) Microsoft Excel
3 2 (0.25%) Gephi
4 1 (0.13%) Microsoft Office
4 1 (0.13%) Microsoft PowerPoint
4 1 (0.13%) SPSS Modeler
Business Applications
1 1 (0.13%) Oracle Procure-to-Pay
1 1 (0.13%) SAP R/3
Cloud Services
1 437 (54.69%) AWS
2 288 (36.05%) GCP
3 246 (30.79%) Azure
4 97 (12.14%) Amazon EMR
5 77 (9.64%) Serverless
6 68 (8.51%) IBM Cloud
7 49 (6.13%) Amazon S3
8 46 (5.76%) Amazon Kinesis
9 42 (5.26%) Cloud Functions
10 30 (3.75%) AWS Lambda
11 28 (3.50%) Google Compute Engine
12 27 (3.38%) Google App Engine
12 27 (3.38%) Google Container Engine
13 26 (3.25%) GitHub
14 25 (3.13%) Amazon EC2
14 25 (3.13%) Amazon SageMaker
15 23 (2.88%) Google Analytics
16 22 (2.75%) PaaS
17 18 (2.25%) Cloud Computing
18 17 (2.13%) OpenShift
Communications & Networking
1 17 (2.13%) Internet
2 15 (1.88%) Firewall
3 14 (1.75%) HTTP
4 12 (1.50%) Wireless
5 9 (1.13%) TCP/IP
6 6 (0.75%) DNS
6 6 (0.75%) IMAP
6 6 (0.75%) SMTP
7 4 (0.50%) Broadband
7 4 (0.50%) VPN
8 3 (0.38%) 802.11
8 3 (0.38%) Fibre Channel
8 3 (0.38%) FTP
8 3 (0.38%) LAN
8 3 (0.38%) SAN
8 3 (0.38%) SMS
8 3 (0.38%) SNMP
8 3 (0.38%) SSH
8 3 (0.38%) SSL
8 3 (0.38%) WAN
Database & Business Intelligence
1 444 (55.57%) Big Data
2 424 (53.07%) Hadoop
3 202 (25.28%) NoSQL
4 156 (19.52%) Amazon Redshift
5 120 (15.02%) Data Warehouse
6 103 (12.89%) Apache Hive
7 88 (11.01%) Data Lake
8 84 (10.51%) PostgreSQL
9 80 (10.01%) Amazon Athena
10 76 (9.51%) Neo4j
11 70 (8.76%) BigQuery
12 66 (8.26%) Electronic Patient Records
13 61 (7.63%) MongoDB
14 57 (7.13%) Relational Database
15 53 (6.63%) Flink
16 42 (5.26%) Snowflake Schema
17 41 (5.13%) Tableau
18 39 (4.88%) Apache Cassandra
19 37 (4.63%) Presto
20 36 (4.51%) HBase
Development Applications
1 121 (15.14%) Jenkins
2 83 (10.39%) Git (software)
3 70 (8.76%) Atlassian Bamboo
3 70 (8.76%) Vagrant
4 36 (4.51%) JIRA
5 21 (2.63%) Oracle APEX
6 17 (2.13%) Maven
6 17 (2.13%) Sonatype Nexus
7 14 (1.75%) TeamCity
8 13 (1.63%) Gradle
9 12 (1.50%) GitLab
10 8 (1.00%) Bitbucket
11 6 (0.75%) Artifactory
11 6 (0.75%) Cypress.io
12 5 (0.63%) MLflow
12 5 (0.63%) SonarQube
13 4 (0.50%) Arsenale Dataplane
13 4 (0.50%) IBM UrbanCode
13 4 (0.50%) Octopus Deploy
13 4 (0.50%) Spinnaker
General
1 212 (26.53%) Finance
2 88 (11.01%) Social Skills
3 78 (9.76%) Analytical Skills
4 71 (8.89%) Front Office
5 63 (7.88%) Banking
6 39 (4.88%) Marketing
7 34 (4.26%) Retail
8 25 (3.13%) Telecoms
9 22 (2.75%) Games
10 15 (1.88%) Law
11 14 (1.75%) Advertising
12 9 (1.13%) Manufacturing
13 7 (0.88%) Investment Banking
13 7 (0.88%) Publishing
14 6 (0.75%) Organisational Skills
15 5 (0.63%) Presentation Skills
16 4 (0.50%) Financial Institution
17 3 (0.38%) Electronics
17 3 (0.38%) Pharmaceutical
17 3 (0.38%) Public Sector
Job Titles
1 294 (36.80%) Data Engineer
2 175 (21.90%) Developer
3 122 (15.27%) Software Engineer
4 80 (10.01%) Architect
5 77 (9.64%) Big Data Engineer
6 72 (9.01%) Senior Data Engineer
7 68 (8.51%) AWS Engineer
8 66 (8.26%) Data Architect
9 63 (7.88%) Platform Engineer
10 62 (7.76%) Java Developer
11 55 (6.88%) Data Scientist
12 45 (5.63%) Python Engineer
13 44 (5.51%) Developer/Software Engineer
14 41 (5.13%) Java Engineer
15 40 (5.01%) .NET Developer
16 37 (4.63%) .NET Software Developer
17 36 (4.51%) .NET Software Engineer
18 34 (4.26%) Java Software Engineer
19 32 (4.01%) Cloud Engineer
20 29 (3.63%) Platform Architect
Libraries, Frameworks & Software Standards
1 343 (42.93%) Kafka
2 95 (11.89%) Spark Streaming
3 87 (10.89%) React
4 64 (8.01%) Node.js
5 57 (7.13%) .NET
6 51 (6.38%) Spring
7 48 (6.01%) AngularJS
8 46 (5.76%) NumPy
9 42 (5.26%) Pandas
9 42 (5.26%) Spring Boot
10 37 (4.63%) RESTful
11 36 (4.51%) HTML
11 36 (4.51%) PySpark
11 36 (4.51%) scikit-learn
12 35 (4.38%) .NET Core
12 35 (4.38%) Web Services
13 31 (3.88%) CSS
13 31 (3.88%) HTML5
14 25 (3.13%) TensorFlow
15 23 (2.88%) Dask
Miscellaneous
1 72 (9.01%) Algorithms
2 49 (6.13%) Equities
3 47 (5.88%) Hedge funds
4 40 (5.01%) Data Structures
5 35 (4.38%) Hybrid Cloud
6 26 (3.25%) Greenfield Project
7 23 (2.88%) User Experience
8 19 (2.38%) Public Cloud
9 16 (2.00%) Foreign Exchange (FX)
10 13 (1.63%) Derivative
11 11 (1.38%) Cloud Native
12 10 (1.25%) Video\Live Streaming
13 9 (1.13%) Market Risk
13 9 (1.13%) Social Media
14 8 (1.00%) Internet of Things
14 8 (1.00%) LIMS
14 8 (1.00%) NHS
15 7 (0.88%) Data Centre
16 6 (0.75%) Self-Motivation
16 6 (0.75%) Virtual Team
Operating Systems
1 46 (5.76%) Linux
2 27 (3.38%) Unix
3 9 (1.13%) Ubuntu
4 7 (0.88%) Windows
5 3 (0.38%) Apple iOS
5 3 (0.38%) CentOS
6 2 (0.25%) Oracle Linux
7 1 (0.13%) Red Hat Enterprise Linux
Processes & Methodologies
1 308 (38.55%) Agile Software Development
2 302 (37.80%) Analytics
3 260 (32.54%) Data Pipeline
4 256 (32.04%) Data Science
4 256 (32.04%) Software Engineering
5 239 (29.91%) ETL
6 212 (26.53%) Data Engineering
7 188 (23.53%) Artificial Intelligence
8 184 (23.03%) Machine Learning
9 167 (20.90%) Microservices
10 136 (17.02%) DevOps
11 126 (15.77%) Continuous Integration
12 92 (11.51%) Data Munging
13 91 (11.39%) Business Intelligence
14 90 (11.26%) Open Source
15 89 (11.14%) Data Modelling
16 85 (10.64%) Unit Testing
17 84 (10.51%) Data Analytics
18 79 (9.89%) Configuration Management
19 78 (9.76%) Computer Science
Programming Languages
1 523 (65.46%) Python
2 397 (49.69%) SQL
3 343 (42.93%) Java
4 219 (27.41%) Scala
5 116 (14.52%) JavaScript
6 92 (11.51%) C#
7 46 (5.76%) R
8 38 (4.76%) Go
9 32 (4.01%) TypeScript
10 25 (3.13%) C++
11 23 (2.88%) PL/SQL
12 22 (2.75%) Bash Shell
12 22 (2.75%) C
12 22 (2.75%) Kotlin
13 18 (2.25%) F#
14 17 (2.13%) ES6
14 17 (2.13%) Java 8
14 17 (2.13%) PHP
15 12 (1.50%) PowerShell
16 11 (1.38%) Perl
Qualifications
1 101 (12.64%) Degree
2 50 (6.26%) Computer Science Degree
3 15 (1.88%) Security Cleared
4 12 (1.50%) PhD
5 4 (0.50%) DV Cleared
5 4 (0.50%) TOGAF Certification
6 3 (0.38%) AWS Certification
6 3 (0.38%) Master's Degree
6 3 (0.38%) Numerate Degree
6 3 (0.38%) Postgraduate
6 3 (0.38%) SC Cleared
7 2 (0.25%) CCNA
7 2 (0.25%) Cisco Certification
7 2 (0.25%) CISSP
7 2 (0.25%) CompTIA Security+
7 2 (0.25%) MCSE
7 2 (0.25%) Microsoft Certification
7 2 (0.25%) Network+ Certification
7 2 (0.25%) SANS
7 2 (0.25%) SSCP
Quality Assurance & Compliance
1 54 (6.76%) QA
2 24 (3.00%) Data Quality
3 6 (0.75%) GDPR
4 1 (0.13%) MiFID
System Software
1 296 (37.05%) Docker
2 70 (8.76%) VirtualBox
3 14 (1.75%) HDFS
4 6 (0.75%) Apache Flume
5 4 (0.50%) Apache ZooKeeper
6 3 (0.38%) VMware Infrastructure
7 1 (0.13%) NFS
7 1 (0.13%) oVirt
7 1 (0.13%) Platform Symphony
7 1 (0.13%) Virtual Machines
Systems Management
1 311 (38.92%) Kubernetes
2 117 (14.64%) Terraform
3 104 (13.02%) Ansible
4 70 (8.76%) Docker Swarm
5 19 (2.38%) Puppet
6 14 (1.75%) Prometheus
7 10 (1.25%) Istio
8 9 (1.13%) Oozie
9 8 (1.00%) Opscode Chef
10 6 (0.75%) Hadoop YARN
11 5 (0.63%) Ambari
11 5 (0.63%) Kibana
11 5 (0.63%) Kubeflow
12 3 (0.38%) Grafana
12 3 (0.38%) Nagios
13 1 (0.13%) ITRS Geneos
13 1 (0.13%) logstash
13 1 (0.13%) Mesos
13 1 (0.13%) SELinux
13 1 (0.13%) Zipkin
Vendors
1 174 (21.78%) Google
2 97 (12.14%) Cloudera
3 95 (11.89%) Talend
4 83 (10.39%) Hortonworks
5 69 (8.64%) IBM
6 65 (8.14%) Informatica
7 63 (7.88%) Oracle
8 62 (7.76%) Databricks
9 24 (3.00%) Microsoft
10 9 (1.13%) Qubit
10 9 (1.13%) Red Hat
10 9 (1.13%) Splunk
11 7 (0.88%) Cisco
11 7 (0.88%) SAP
11 7 (0.88%) Teradata
12 6 (0.75%) Sybase
13 5 (0.63%) Alteryx
13 5 (0.63%) Greenplum
14 4 (0.50%) Solace
14 4 (0.50%) VMware