Period
to 5 August 2020

The following table provides summary statistics for permanent job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 5 August 2020 with a comparison to the same period in the previous 2 years.

Apache Spark
England > London
6 months to
5 Aug 2020
Same period 2019 Same period 2018
Rank 163 131 150
Rank change year-on-year -32 +19 +56
Permanent jobs citing Apache Spark 574 1,460 1,651
As % of all permanent jobs advertised in London 2.10% 2.58% 2.36%
As % of the Application Platforms category 13.50% 16.19% 13.32%
Number of salaries quoted 426 1,292 1,289
Median annual salary £80,000 £80,000 £75,000
Median salary % change year-on-year - +6.67% +7.14%
10th Percentile £50,750 £55,000 £51,250
90th Percentile £122,500 £115,000 £107,500
England median annual salary £75,000 £75,000 £70,000
% change year-on-year - +7.14% +3.70%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Permanent vacancies with a requirement for application platform skills 4,252 9,018 12,394
As % of all permanent IT jobs advertised in London 15.54% 15.91% 17.73%
Number of salaries quoted 3,346 7,329 9,863
Median annual salary £65,000 £62,500 £60,000
Median salary % change year-on-year +4.00% +4.17% +4.35%
10th Percentile £38,750 £33,750 £33,750
90th Percentile £100,000 £95,000 £91,000
England median annual salary £54,000 £51,000 £50,000
% change year-on-year +5.88% +2.00% +5.26%

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Salary Trend in London

3-month moving average salary quoted in jobs citing Apache Spark in London.

Salary trend for Apache Spark in London

Apache Spark
Salary Histogram in London

Salary distribution for jobs citing Apache Spark in London over the 6 months to 5 August 2020.

Salary histogram for Apache Spark in London

Apache Spark
Job Locations in London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the London region over the 6 months to 5 August 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +33 128 £75,000 - 14
South London +40 18 £67,500 -15.63% 3
West London +34 13 £77,500 +10.71% 13
East London +8 3 £95,000 +35.71% 8
North London 0 1 £82,500 +6.45% 2
Apache Spark
England

For the 6 months to 5 August 2020, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the London region with a requirement for Apache Spark.

1 396 (68.99%) Python
2 358 (62.37%) Hadoop
3 317 (55.23%) Big Data
4 282 (49.13%) AWS
5 249 (43.38%) Kafka
6 245 (42.68%) Java
7 228 (39.72%) Analytics
8 227 (39.55%) Data Science
9 216 (37.63%) SQL
10 214 (37.28%) Agile Software Development
11 210 (36.59%) Scala
12 185 (32.23%) Azure
13 183 (31.88%) Machine Learning
14 169 (29.44%) Finance
15 150 (26.13%) NoSQL
16 148 (25.78%) GCP
17 140 (24.39%) DevOps
18 139 (24.22%) Degree
19 132 (23.00%) Kubernetes
20 124 (21.60%) Apache Hive
21 115 (20.03%) Docker
22 114 (19.86%) Data Engineering
23 112 (19.51%) ETL
24 111 (19.34%) Apache
25 109 (18.99%) Artificial Intelligence
26 106 (18.47%) R
27 98 (17.07%) Software Engineering
28 87 (15.16%) Computer Science
29 86 (14.98%) Visualisation
29 86 (14.98%) Git (software)

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 111 (19.34%) Apache
2 32 (5.57%) Jupyter
3 27 (4.70%) Elasticsearch
4 26 (4.53%) Apache Airflow
5 16 (2.79%) Tomcat
6 15 (2.61%) Confluence
7 11 (1.92%) Apache Solr
7 11 (1.92%) SAS
8 8 (1.39%) Cloud Foundry
8 8 (1.39%) Luigi
9 7 (1.22%) Apache Synapse
10 6 (1.05%) Tornado
11 4 (0.70%) Apache Pig
12 3 (0.52%) CMS
12 3 (0.52%) WebSphere
13 1 (0.17%) Oracle Workflow
Applications
1 3 (0.52%) Microsoft Excel
Business Applications
1 5 (0.87%) SAP CRM
1 5 (0.87%) SAP ERP
1 5 (0.87%) SAP IS-U
Cloud Services
1 282 (49.13%) AWS
2 185 (32.23%) Azure
3 148 (25.78%) GCP
4 49 (8.54%) Amazon EMR
5 46 (8.01%) Amazon S3
6 43 (7.49%) Power Platform
7 42 (7.32%) Serverless
8 31 (5.40%) Azure Data Factory
9 25 (4.36%) Amazon CloudWatch
10 23 (4.01%) Cloud Computing
11 20 (3.48%) Amazon Kinesis
12 18 (3.14%) Amazon GuardDuty
12 18 (3.14%) Amazon SQS
12 18 (3.14%) AWS CloudTrail
13 17 (2.96%) Dropbox
13 17 (2.96%) Slack
14 14 (2.44%) Amazon EC2
15 13 (2.26%) AWS Lambda
15 13 (2.26%) SaaS
16 10 (1.74%) IBM Cloud
Communications & Networking
1 10 (1.74%) HTTP
2 6 (1.05%) TCP/IP
3 5 (0.87%) 5G
4 4 (0.70%) Firewall
4 4 (0.70%) Intrusion Detection
5 3 (0.52%) AMQP
5 3 (0.52%) FTP
5 3 (0.52%) Kerberos
6 1 (0.17%) HTTPS
6 1 (0.17%) SSL
Database & Business Intelligence
1 358 (62.37%) Hadoop
2 317 (55.23%) Big Data
3 150 (26.13%) NoSQL
4 124 (21.60%) Apache Hive
5 78 (13.59%) Apache Cassandra
6 72 (12.54%) Data Warehouse
7 69 (12.02%) Data Lake
8 68 (11.85%) Amazon Redshift
9 65 (11.32%) SQL Server
10 58 (10.10%) HBase
11 55 (9.58%) MongoDB
12 51 (8.89%) Flink
13 47 (8.19%) PostgreSQL
14 43 (7.49%) Power BI
15 41 (7.14%) Data Mining
16 37 (6.45%) Relational Database
17 32 (5.57%) BigQuery
18 29 (5.05%) Impala
19 23 (4.01%) Cosmos DB
19 23 (4.01%) Neo4j
Development Applications
1 86 (14.98%) Git (software)
2 34 (5.92%) Atlassian Bamboo
3 31 (5.40%) Jenkins
4 19 (3.31%) JIRA
5 14 (2.44%) Bitbucket
5 14 (2.44%) TeamCity
6 13 (2.26%) Maven
6 13 (2.26%) Vagrant
7 10 (1.74%) JUnit
8 9 (1.57%) GitLab
8 9 (1.57%) Octopus Deploy
9 8 (1.39%) Artifactory
10 7 (1.22%) GoCD
10 7 (1.22%) Visual Studio
11 6 (1.05%) Sonatype Nexus
12 5 (0.87%) Ant
12 5 (0.87%) IBM UrbanCode
12 5 (0.87%) Team Foundation Server
13 3 (0.52%) CircleCI
13 3 (0.52%) Visual Studio Team System
General
1 169 (29.44%) Finance
2 71 (12.37%) Banking
3 42 (7.32%) Retail
4 41 (7.14%) Marketing
5 37 (6.45%) Front Office
6 19 (3.31%) Manufacturing
7 17 (2.96%) Telecoms
8 16 (2.79%) Investment Banking
9 13 (2.26%) Games
10 12 (2.09%) Public Sector
11 10 (1.74%) Advertising
12 4 (0.70%) Automotive
12 4 (0.70%) Retail Banking
13 3 (0.52%) Financial Institution
13 3 (0.52%) Law
13 3 (0.52%) Pharmaceutical
14 2 (0.35%) Legal
15 1 (0.17%) Publishing
Job Titles
1 225 (39.20%) Data Engineer
2 95 (16.55%) Developer
3 71 (12.37%) Data Scientist
4 66 (11.50%) Big Data Engineer
5 59 (10.28%) Architect
6 55 (9.58%) Senior Data Engineer
7 46 (8.01%) Data Architect
8 45 (7.84%) Senior Data Scientist
9 40 (6.97%) Java Developer
10 38 (6.62%) AWS Engineer
10 38 (6.62%) Python Engineer
11 26 (4.53%) Cloud Engineer
12 23 (4.01%) Big Data Architect
12 23 (4.01%) Platform Engineer
12 23 (4.01%) Senior Developer
13 20 (3.48%) Full Stack Developer
13 20 (3.48%) Scala Developer
14 18 (3.14%) Security Engineer
14 18 (3.14%) Software Engineer
15 14 (2.44%) Python Developer
Libraries, Frameworks & Software Standards
1 249 (43.38%) Kafka
2 51 (8.89%) React
3 48 (8.36%) Natural Language Processing
4 45 (7.84%) AngularJS
5 41 (7.14%) Spring
6 40 (6.97%) scikit-learn
6 40 (6.97%) TensorFlow
7 37 (6.45%) Web Services
8 33 (5.75%) Pandas
9 32 (5.57%) NumPy
9 32 (5.57%) PySpark
9 32 (5.57%) REST
10 30 (5.23%) Spark Streaming
11 27 (4.70%) .NET
11 27 (4.70%) Node.js
12 24 (4.18%) Akka
12 24 (4.18%) Hibernate
13 21 (3.66%) Keras
14 19 (3.31%) Flask
14 19 (3.31%) XML
Miscellaneous
1 86 (14.98%) Algorithms
2 74 (12.89%) Analytical Skills
3 44 (7.67%) Hedge funds
4 33 (5.75%) Cloud Native
5 31 (5.40%) Data Structures
6 30 (5.23%) Equities
7 25 (4.36%) Greenfield Project
7 25 (4.36%) User Experience
8 21 (3.66%) Management Information System
9 20 (3.48%) Hybrid Cloud
10 18 (3.14%) Market Risk
11 17 (2.96%) Blog
12 13 (2.26%) Life Science
13 12 (2.09%) Enterprise Software
13 12 (2.09%) Robotics
14 11 (1.92%) Acoustics
14 11 (1.92%) Reference Data
15 10 (1.74%) Mobile App
15 10 (1.74%) Self-Motivation
16 9 (1.57%) Virtual Team
Operating Systems
1 51 (8.89%) Linux
2 31 (5.40%) Unix
3 14 (2.44%) Red Hat Enterprise Linux
4 4 (0.70%) Windows
5 3 (0.52%) Android
5 3 (0.52%) Apple iOS
5 3 (0.52%) Ubuntu
6 1 (0.17%) Windows Server
Processes & Methodologies
1 228 (39.72%) Analytics
2 227 (39.55%) Data Science
3 214 (37.28%) Agile Software Development
4 183 (31.88%) Machine Learning
5 140 (24.39%) DevOps
6 114 (19.86%) Data Engineering
7 112 (19.51%) ETL
8 109 (18.99%) Artificial Intelligence
9 98 (17.07%) Software Engineering
10 87 (15.16%) Computer Science
11 86 (14.98%) Visualisation
12 80 (13.94%) Mathematics
13 76 (13.24%) Data Analytics
14 68 (11.85%) Data Modelling
15 66 (11.50%) Business Intelligence
16 61 (10.63%) Microservices
17 57 (9.93%) Continuous Integration
18 55 (9.58%) Mentoring
18 55 (9.58%) Statistics
19 53 (9.23%) TDD
Programming Languages
1 396 (68.99%) Python
2 245 (42.68%) Java
3 216 (37.63%) SQL
4 210 (36.59%) Scala
5 106 (18.47%) R
6 66 (11.50%) JavaScript
7 46 (8.01%) C#
8 28 (4.88%) C++
9 22 (3.83%) Bash Shell
10 15 (2.61%) Go
10 15 (2.61%) TypeScript
11 14 (2.44%) PHP
12 13 (2.26%) C
13 11 (1.92%) Kotlin
14 9 (1.57%) Java 8
15 8 (1.39%) MATLAB
15 8 (1.39%) Shell Script
16 7 (1.22%) Ruby
17 6 (1.05%) ES6
18 5 (0.87%) PowerShell
Qualifications
1 139 (24.22%) Degree
2 73 (12.72%) Computer Science Degree
3 20 (3.48%) PhD
4 10 (1.74%) AWS Certification
4 10 (1.74%) Security Cleared
5 9 (1.57%) DV Cleared
5 9 (1.57%) Master's Degree
5 9 (1.57%) SC Cleared
6 8 (1.39%) PMI Certification
7 7 (1.22%) Numerate Degree
8 4 (0.70%) Microsoft Certification
9 3 (0.52%) MBA
Quality Assurance & Compliance
1 18 (3.14%) GDPR
1 18 (3.14%) QA
2 15 (2.61%) Data Quality
3 7 (1.22%) NIST
4 1 (0.17%) Actionable Recommendations
System Software
1 115 (20.03%) Docker
2 19 (3.31%) HDFS
3 14 (2.44%) Apache ZooKeeper
4 13 (2.26%) VirtualBox
5 4 (0.70%) Apache Flume
6 1 (0.17%) Virtual Machines
Systems Management
1 132 (23.00%) Kubernetes
2 42 (7.32%) Terraform
3 31 (5.40%) Ansible
4 14 (2.44%) Hadoop YARN
5 13 (2.26%) Docker Swarm
6 11 (1.92%) Puppet
7 7 (1.22%) Oozie
8 6 (1.05%) Kibana
8 6 (1.05%) Opscode Chef
8 6 (1.05%) Salt
9 5 (0.87%) Ambari
10 4 (0.70%) Nagios
10 4 (0.70%) Zipkin
11 3 (0.52%) Cobbler
11 3 (0.52%) Red Hat Satellite
12 2 (0.35%) Prometheus
13 1 (0.17%) Single Sign-On
Vendors
1 75 (13.07%) Google
2 74 (12.89%) Tableau
3 59 (10.28%) Databricks
4 37 (6.45%) Cloudera
5 36 (6.27%) Oracle
6 34 (5.92%) Microsoft
7 24 (4.18%) Splunk
8 23 (4.01%) Hortonworks
9 20 (3.48%) IBM
10 17 (2.96%) CrowdStrike
10 17 (2.96%) Talend
11 11 (1.92%) StreamSets
12 10 (1.74%) Informatica
13 8 (1.39%) Facebook
13 8 (1.39%) Sybase
14 7 (1.22%) Red Hat
15 6 (1.05%) Ab Initio
15 6 (1.05%) EXASOL
16 5 (0.87%) SAP
17 4 (0.70%) Palantir