Period
to 21 October 2020

The following table provides summary statistics for permanent job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 21 October 2020 with a comparison to the same period in the previous 2 years.

Apache Spark
England > London
6 months to
21 Oct 2020
Same period 2019 Same period 2018
Rank 158 143 174
Rank change year-on-year -15 +31 +10
Permanent jobs citing Apache Spark 425 1,299 1,446
As % of all permanent jobs advertised in London 2.15% 2.38% 2.14%
As % of the Application Platforms category 14.24% 15.47% 12.11%
Number of salaries quoted 351 1,122 1,122
Median annual salary £80,000 £80,000 £75,000
Median salary % change year-on-year - +6.67% +7.14%
10th Percentile £53,750 £53,750 £50,000
90th Percentile £121,250 £115,000 £105,000
England median annual salary £72,500 £72,500 £72,500

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Permanent vacancies with a requirement for application platform skills 2,985 8,397 11,945
As % of all permanent IT jobs advertised in London 15.13% 15.37% 17.68%
Number of salaries quoted 2,490 6,859 9,549
Median annual salary £67,500 £62,500 £60,000
Median salary % change year-on-year +8.00% +4.17% +4.35%
10th Percentile £41,250 £32,500 £32,500
90th Percentile £103,750 £95,000 £92,500
England median annual salary £55,000 £50,000 £50,000
% change year-on-year +10.00% - +5.26%

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Salary Trend in London

3-month moving average salary quoted in jobs citing Apache Spark in London.

Salary trend for Apache Spark in London

Apache Spark
Salary Histogram in London

Salary distribution for jobs citing Apache Spark in London over the 6 months to 21 October 2020.

Salary histogram for Apache Spark in London

Apache Spark
Job Locations in London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the London region over the 6 months to 21 October 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +44 111 £75,000 - 22
West London +57 7 £77,500 -3.13% 12
East London +13 5 £65,000 -9.72% 7
North London +33 4 £65,000 -13.33% 1
South London +45 2 £62,500 -26.47% 3
Apache Spark
England

For the 6 months to 21 October 2020, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the London region with a requirement for Apache Spark.

1 282 (66.35%) Python
2 233 (54.82%) Hadoop
3 218 (51.29%) AWS
4 213 (50.12%) Big Data
5 202 (47.53%) Kafka
6 178 (41.88%) Java
7 163 (38.35%) SQL
8 153 (36.00%) Finance
9 150 (35.29%) Scala
10 137 (32.24%) Machine Learning
11 136 (32.00%) Data Science
12 132 (31.06%) Agile Software Development
13 127 (29.88%) Azure
14 125 (29.41%) Analytics
15 110 (25.88%) Kubernetes
16 103 (24.24%) Apache
17 100 (23.53%) Degree
18 99 (23.29%) ETL
18 99 (23.29%) DevOps
19 90 (21.18%) Docker
20 87 (20.47%) GCP
21 86 (20.24%) NoSQL
22 84 (19.76%) Artificial Intelligence
23 80 (18.82%) Mathematics
24 76 (17.88%) Data Lake
25 75 (17.65%) Apache Hive
26 73 (17.18%) Algorithms
27 71 (16.71%) Data Engineering
28 70 (16.47%) R
29 69 (16.24%) Git (software)

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 103 (24.24%) Apache
2 29 (6.82%) Elasticsearch
3 17 (4.00%) Apache Airflow
4 12 (2.82%) Jupyter
5 9 (2.12%) Cloud Foundry
5 9 (2.12%) SAS
5 9 (2.12%) Tomcat
6 7 (1.65%) Apache Synapse
7 6 (1.41%) CMS
7 6 (1.41%) Confluence
8 5 (1.18%) Luigi
8 5 (1.18%) Tornado
9 3 (0.71%) Apache Pig
9 3 (0.71%) WebSphere
10 1 (0.24%) Apache Solr
10 1 (0.24%) GlassFish
Applications
1 7 (1.65%) Microsoft Excel
2 1 (0.24%) SPSS
2 1 (0.24%) SPSS Modeler
Business Applications
1 3 (0.71%) Distributed Ledger
1 3 (0.71%) Oracle Cash Management
Cloud Services
1 218 (51.29%) AWS
2 127 (29.88%) Azure
3 87 (20.47%) GCP
4 47 (11.06%) Serverless
5 38 (8.94%) Amazon S3
6 27 (6.35%) Power Platform
7 25 (5.88%) Amazon EMR
8 23 (5.41%) AWS Lambda
9 22 (5.18%) Amazon CloudWatch
10 19 (4.47%) Amazon Kinesis
10 19 (4.47%) Azure Data Factory
11 18 (4.24%) Amazon GuardDuty
11 18 (4.24%) AWS CloudTrail
12 17 (4.00%) Amazon SQS
12 17 (4.00%) SaaS
13 10 (2.35%) Amazon ECS
13 10 (2.35%) OpenShift
14 9 (2.12%) Amazon SageMaker
14 9 (2.12%) AWS CloudFormation
15 8 (1.88%) GitHub
Communications & Networking
1 12 (2.82%) HTTP
2 9 (2.12%) Firewall
2 9 (2.12%) Kerberos
2 9 (2.12%) TCP/IP
3 7 (1.65%) 5G
4 5 (1.18%) Parlay
5 3 (0.71%) AMQP
5 3 (0.71%) FTP
5 3 (0.71%) Internet
5 3 (0.71%) Wireless
6 2 (0.47%) HTTPS
6 2 (0.47%) SSL
7 1 (0.24%) Network Security
Database & Business Intelligence
1 233 (54.82%) Hadoop
2 213 (50.12%) Big Data
3 86 (20.24%) NoSQL
4 76 (17.88%) Data Lake
5 75 (17.65%) Apache Hive
6 56 (13.18%) Apache Cassandra
7 52 (12.24%) Amazon Redshift
7 52 (12.24%) Data Warehouse
8 47 (11.06%) SQL Server
9 43 (10.12%) PostgreSQL
10 42 (9.88%) Flink
11 36 (8.47%) MongoDB
12 35 (8.24%) Relational Database
13 27 (6.35%) Power BI
14 25 (5.88%) HBase
14 25 (5.88%) RDBMS
15 22 (5.18%) Data Mining
15 22 (5.18%) MySQL
16 20 (4.71%) BigQuery
17 18 (4.24%) Impala
Development Applications
1 69 (16.24%) Git (software)
2 30 (7.06%) Jenkins
3 19 (4.47%) TeamCity
4 17 (4.00%) JIRA
5 12 (2.82%) Atlassian Bamboo
6 8 (1.88%) Octopus Deploy
7 7 (1.65%) Maven
8 6 (1.41%) Bitbucket
8 6 (1.41%) Cucumber
8 6 (1.41%) Selenium
9 5 (1.18%) GoCD
10 4 (0.94%) Team Foundation Server
11 3 (0.71%) Sonatype Nexus
12 2 (0.47%) CA Release Automation
12 2 (0.47%) ER Studio
12 2 (0.47%) ERwin
13 1 (0.24%) CircleCI
13 1 (0.24%) Vagrant
13 1 (0.24%) Visual Studio
13 1 (0.24%) Visual Studio Team System
General
1 153 (36.00%) Finance
2 66 (15.53%) Banking
3 27 (6.35%) Front Office
4 25 (5.88%) Retail
5 22 (5.18%) Marketing
6 18 (4.24%) Telecoms
7 14 (3.29%) Games
8 11 (2.59%) Advertising
8 11 (2.59%) Investment Banking
8 11 (2.59%) Manufacturing
9 7 (1.65%) Law
10 5 (1.18%) Pharmaceutical
11 3 (0.71%) Financial Institution
11 3 (0.71%) Retail Banking
12 1 (0.24%) Legal
12 1 (0.24%) Public Sector
Job Titles
1 150 (35.29%) Data Engineer
2 86 (20.24%) Developer
3 55 (12.94%) Data Scientist
4 50 (11.76%) Senior Data Engineer
5 41 (9.65%) Python Engineer
6 40 (9.41%) Architect
7 39 (9.18%) Java Developer
8 35 (8.24%) Senior Data Scientist
9 30 (7.06%) AWS Engineer
9 30 (7.06%) Data Architect
10 29 (6.82%) Big Data Engineer
11 23 (5.41%) Java Engineer
12 20 (4.71%) Software Engineer
13 17 (4.00%) Full Stack Developer
13 17 (4.00%) Senior Developer
14 16 (3.76%) DevOps Engineer
15 15 (3.53%) Big Data Architect
16 14 (3.29%) Big Data Developer
16 14 (3.29%) Cloud Engineer
17 13 (3.06%) Azure Engineer
Libraries, Frameworks & Software Standards
1 202 (47.53%) Kafka
2 48 (11.29%) Natural Language Processing
3 44 (10.35%) TensorFlow
4 41 (9.65%) scikit-learn
5 35 (8.24%) Node.js
5 35 (8.24%) Spring
6 34 (8.00%) Pandas
7 32 (7.53%) PySpark
7 32 (7.53%) React
8 31 (7.29%) AngularJS
9 29 (6.82%) NumPy
9 29 (6.82%) RESTful
10 24 (5.65%) Keras
11 18 (4.24%) Web Services
12 17 (4.00%) Akka
12 17 (4.00%) HTML
12 17 (4.00%) JSON
12 17 (4.00%) REST
12 17 (4.00%) XGBoost
13 16 (3.76%) PyTorch
Miscellaneous
1 73 (17.18%) Algorithms
2 57 (13.41%) Analytical Skills
3 27 (6.35%) Hedge funds
4 26 (6.12%) Cloud Native
5 23 (5.41%) Equities
6 16 (3.76%) Data Structures
7 15 (3.53%) Hybrid Cloud
8 14 (3.29%) Management Information System
9 13 (3.06%) Greenfield Project
9 13 (3.06%) Market Risk
10 12 (2.82%) FMCG
10 12 (2.82%) Self-Motivation
11 9 (2.12%) Life Science
12 8 (1.88%) Robotics
13 7 (1.65%) Mobile App
14 5 (1.18%) Pricing Data
14 5 (1.18%) Public Cloud
14 5 (1.18%) Social Media
15 4 (0.94%) Renewable Energy
15 4 (0.94%) Virtual Team
Operating Systems
1 44 (10.35%) Linux
2 15 (3.53%) Red Hat Enterprise Linux
2 15 (3.53%) Unix
3 6 (1.41%) Ubuntu
4 1 (0.24%) Android
4 1 (0.24%) Apple iOS
4 1 (0.24%) Windows
Processes & Methodologies
1 137 (32.24%) Machine Learning
2 136 (32.00%) Data Science
3 132 (31.06%) Agile Software Development
4 125 (29.41%) Analytics
5 99 (23.29%) DevOps
5 99 (23.29%) ETL
6 84 (19.76%) Artificial Intelligence
7 80 (18.82%) Mathematics
8 71 (16.71%) Data Engineering
9 66 (15.53%) Visualisation
10 63 (14.82%) Computer Science
11 58 (13.65%) Microservices
12 55 (12.94%) Software Engineering
13 50 (11.76%) Business Intelligence
14 45 (10.59%) Data Modelling
15 42 (9.88%) Statistics
16 41 (9.65%) Data Visualisation
16 41 (9.65%) Mentoring
17 38 (8.94%) Deep Learning
18 35 (8.24%) TDD
Programming Languages
1 282 (66.35%) Python
2 178 (41.88%) Java
3 163 (38.35%) SQL
4 150 (35.29%) Scala
5 70 (16.47%) R
6 57 (13.41%) JavaScript
7 45 (10.59%) C#
8 19 (4.47%) Bash Shell
9 18 (4.24%) Go
10 13 (3.06%) Kotlin
10 13 (3.06%) TypeScript
11 11 (2.59%) PHP
11 11 (2.59%) Ruby
11 11 (2.59%) Shell Script
12 9 (2.12%) Java 8
13 8 (1.88%) MATLAB
14 7 (1.65%) C
14 7 (1.65%) C++
14 7 (1.65%) ES6
14 7 (1.65%) PowerShell
Qualifications
1 100 (23.53%) Degree
2 53 (12.47%) Computer Science Degree
3 19 (4.47%) PhD
4 13 (3.06%) Master's Degree
5 9 (2.12%) AWS Certification
6 6 (1.41%) Numerate Degree
7 5 (1.18%) Postgraduate
8 3 (0.71%) Sun Certification
8 3 (0.71%) TOGAF Certification
9 1 (0.24%) SC Cleared
9 1 (0.24%) Security Cleared
Quality Assurance & Compliance
1 17 (4.00%) GDPR
2 14 (3.29%) QA
3 7 (1.65%) Data Quality
4 3 (0.71%) ISO 20022
4 3 (0.71%) PSD2
4 3 (0.71%) TARGET2
5 1 (0.24%) MiFID
5 1 (0.24%) PMO
System Software
1 90 (21.18%) Docker
2 13 (3.06%) HDFS
3 7 (1.65%) Apache Flume
4 6 (1.41%) Apache ZooKeeper
5 2 (0.47%) Virtual Machines
6 1 (0.24%) Active Directory
6 1 (0.24%) VirtualBox
Systems Management
1 110 (25.88%) Kubernetes
2 43 (10.12%) Terraform
3 29 (6.82%) Ansible
4 9 (2.12%) Oozie
4 9 (2.12%) Puppet
5 7 (1.65%) Hadoop YARN
6 5 (1.18%) Ambari
7 4 (0.94%) Kibana
7 4 (0.94%) Nagios
7 4 (0.94%) Zipkin
8 3 (0.71%) Cobbler
8 3 (0.71%) Red Hat Satellite
9 2 (0.47%) Opscode Chef
9 2 (0.47%) Single Sign-On
10 1 (0.24%) Docker Swarm
10 1 (0.24%) Mesos
10 1 (0.24%) SELinux
Vendors
1 61 (14.35%) Databricks
2 49 (11.53%) Tableau
3 33 (7.76%) Oracle
4 32 (7.53%) Google
5 26 (6.12%) Cloudera
6 15 (3.53%) Red Hat
6 15 (3.53%) Splunk
7 13 (3.06%) Microsoft
8 10 (2.35%) Hortonworks
9 7 (1.65%) StreamSets
10 6 (1.41%) IBM
10 6 (1.41%) Informatica
10 6 (1.41%) Talend
11 5 (1.18%) Palantir
12 4 (0.94%) Foundry
12 4 (0.94%) Sybase
13 3 (0.71%) Guidewire
13 3 (0.71%) MuleSoft
13 3 (0.71%) NVIDIA
13 3 (0.71%) Solace