Job Vacancy Trend Salary Trend Salary Histogram Job Locations Skill Sets
Period
to 23 January 2022

The following table provides summary statistics for permanent job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 23 January 2022 with a comparison to the same period in the previous 2 years.

Apache Spark
England > London
6 months to
23 Jan 2022		 Same period 2021 Same period 2020
Rank 190 160 159
Rank change year-on-year -30 -1 +7
Permanent jobs citing Apache Spark 972 537 1,162
As % of all permanent jobs advertised in London 1.93% 2.24% 2.27%
As % of the Application Platforms category 13.92% 15.69% 15.30%
Number of salaries quoted 539 439 953
10th Percentile £53,750 £49,750 £53,750
25th Percentile £65,000 £57,500 £62,500
Median annual salary (50th Percentile) £80,000 £77,500 £77,500
Median % change year-on-year +3.23% - -
75th Percentile £101,875 £95,000 £93,750
90th Percentile £120,000 £112,500 £112,500
England median annual salary £80,000 £70,000 £72,500
% change year-on-year +14.29% -3.45% -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Permanent vacancies with a requirement for application platform skills 6,983 3,423 7,596
As % of all permanent IT jobs advertised in London 13.83% 14.28% 14.83%
Number of salaries quoted 4,371 2,843 6,282
10th Percentile £37,500 £41,250 £32,500
25th Percentile £49,625 £51,250 £45,000
Median annual salary (50th Percentile) £65,000 £65,000 £60,750
Median % change year-on-year - +7.00% +1.25%
75th Percentile £87,500 £83,750 £80,000
90th Percentile £102,500 £100,000 £95,000
England median annual salary £55,000 £55,000 £50,000
% change year-on-year - +10.00% -

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Salary Trend in London

3-month moving average salary quoted in jobs citing Apache Spark in London.

Salary trend for Apache Spark in London

Apache Spark
Salary Histogram in London

Salary distribution for jobs citing Apache Spark in London over the 6 months to 23 January 2022.

Salary histogram for Apache Spark in London

Apache Spark
Job Locations in London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the London region over the 6 months to 23 January 2022. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year		 Matching
Permanent
IT Job Ads		 Median Salary
Past 6 Months		 Median Salary
% Change
on Same Period
Last Year		 Live
Jobs
Central London -35 135 £78,750 -1.56% 36
West London -55 23 - - 10
East London -33 13 £95,000 +46.15% 4
South London -13 9 £70,000 +3.70% 1
Apache Spark
England

For the 6 months to 23 January 2022, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the London region with a requirement for Apache Spark.

1 725 (74.59%) Python
2 502 (51.65%) AWS
3 478 (49.18%) Big Data
4 466 (47.94%) Data Science
5 465 (47.84%) Java
6 462 (47.53%) Analytics
7 441 (45.37%) Kafka
8 439 (45.16%) Hadoop
9 433 (44.55%) SQL
10 377 (38.79%) Scala
11 358 (36.83%) Agile Software Development
12 346 (35.60%) GCP
13 315 (32.41%) Azure
14 307 (31.58%) Machine Learning
15 278 (28.60%) Data Pipeline
16 268 (27.57%) Kubernetes
17 267 (27.47%) Data Engineering
17 267 (27.47%) DevOps
18 253 (26.03%) NoSQL
19 241 (24.79%) ETL
19 241 (24.79%) CI/CD
20 236 (24.28%) Docker
20 236 (24.28%) Finance
21 219 (22.53%) Software Engineering
22 217 (22.33%) Data Warehouse
22 217 (22.33%) Social Skills
23 212 (21.81%) Data Modelling
23 212 (21.81%) Artificial Intelligence
24 196 (20.16%) Data Lake
25 167 (17.18%) Apache Airflow

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 167 (17.18%) Apache Airflow
2 159 (16.36%) Apache
3 99 (10.19%) Elasticsearch
4 30 (3.09%) Confluence
5 27 (2.78%) Luigi
6 25 (2.57%) Apache Pig
7 23 (2.37%) Jupyter
8 13 (1.34%) Apache Pulsar
9 12 (1.23%) SAS
10 5 (0.51%) Apache Solr
10 5 (0.51%) Tornado
11 4 (0.41%) Oracle Workflow
12 2 (0.21%) Oracle SOA Suite
13 1 (0.10%) Apache Synapse
13 1 (0.10%) CMS
13 1 (0.10%) nginx
Applications
1 3 (0.31%) Microsoft PowerPoint
2 1 (0.10%) Chatbot
2 1 (0.10%) Microsoft Excel
2 1 (0.10%) Microsoft Office
2 1 (0.10%) Photoshop
2 1 (0.10%) SPSS
Business Applications
1 1 (0.10%) Oracle Procure-to-Pay
1 1 (0.10%) SAP Oil and Gas
1 1 (0.10%) SAP R/3
Cloud Services
1 502 (51.65%) AWS
2 346 (35.60%) GCP
3 315 (32.41%) Azure
4 139 (14.30%) Amazon S3
5 114 (11.73%) Amazon EMR
6 98 (10.08%) Snowflake
7 96 (9.88%) Serverless
8 95 (9.77%) Amazon Kinesis
9 47 (4.84%) Power Platform
10 46 (4.73%) GitHub
11 42 (4.32%) Matillion
12 41 (4.22%) AWS Lambda
13 31 (3.19%) Amazon EC2
13 31 (3.19%) Azure Synapse Analytics
14 29 (2.98%) SaaS
15 23 (2.37%) AWS CloudFormation
16 22 (2.26%) Azure Data Factory
16 22 (2.26%) PaaS
17 21 (2.16%) Azure Machine Learning
18 20 (2.06%) OpenShift
Communications & Networking
1 28 (2.88%) Wireless
2 24 (2.47%) HTTP
3 21 (2.16%) Internet
4 6 (0.62%) DNS
5 5 (0.51%) Broadband
5 5 (0.51%) DHCP
5 5 (0.51%) HTTPS
5 5 (0.51%) IPv4
5 5 (0.51%) IPv6
5 5 (0.51%) SMTP
5 5 (0.51%) SNMP
5 5 (0.51%) SSH
5 5 (0.51%) TCP/IP
6 4 (0.41%) Kerberos
7 3 (0.31%) Reverse Proxy
7 3 (0.31%) SMS
8 2 (0.21%) Cisco IPT
9 1 (0.10%) LAN
9 1 (0.10%) WAN
Database & Business Intelligence
1 478 (49.18%) Big Data
2 439 (45.16%) Hadoop
3 253 (26.03%) NoSQL
4 217 (22.33%) Data Warehouse
5 196 (20.16%) Data Lake
6 166 (17.08%) Apache Hive
7 119 (12.24%) MongoDB
8 114 (11.73%) Amazon Redshift
8 114 (11.73%) Flink
9 113 (11.63%) BigQuery
10 97 (9.98%) Apache Cassandra
11 91 (9.36%) MySQL
12 90 (9.26%) HBase
13 85 (8.74%) PostgreSQL
14 66 (6.79%) Relational Database
15 64 (6.58%) Amazon DynamoDB
15 64 (6.58%) Tableau
16 60 (6.17%) Amazon Athena
17 58 (5.97%) Metadata
18 54 (5.56%) Electronic Patient Records
Development Applications
1 113 (11.63%) Git (software)
2 96 (9.88%) Jenkins
3 35 (3.60%) JIRA
4 31 (3.19%) CircleCI
5 22 (2.26%) Bitbucket
6 15 (1.54%) GitLab
7 14 (1.44%) Atlassian Bamboo
8 13 (1.34%) Maven
8 13 (1.34%) Spinnaker
9 10 (1.03%) JUnit
10 9 (0.93%) Burp Suite
11 8 (0.82%) Vagrant
12 6 (0.62%) Gradle
12 6 (0.62%) MLflow
12 6 (0.62%) RStudio
12 6 (0.62%) Sonatype Nexus
13 5 (0.51%) kappa
13 5 (0.51%) Octopus Deploy
13 5 (0.51%) TeamCity
14 4 (0.41%) SonarQube
General
1 236 (24.28%) Finance
2 217 (22.33%) Social Skills
3 94 (9.67%) Analytical Skills
4 87 (8.95%) Marketing
5 81 (8.33%) Banking
6 74 (7.61%) Retail
7 66 (6.79%) Inclusion and Diversity
8 63 (6.48%) Games
9 39 (4.01%) Telecoms
10 34 (3.50%) Presentation Skills
11 28 (2.88%) Front Office
12 27 (2.78%) Pharmaceutical
13 26 (2.67%) Advertising
13 26 (2.67%) Publishing
14 23 (2.37%) Investment Banking
15 20 (2.06%) Public Sector
16 18 (1.85%) Law
16 18 (1.85%) Manufacturing
17 13 (1.34%) Legal
18 11 (1.13%) Electronics
Job Titles
1 429 (44.14%) Data Engineer
2 102 (10.49%) Senior Data Engineer
3 91 (9.36%) Architect
4 82 (8.44%) Data Architect
4 82 (8.44%) Developer
5 78 (8.02%) Data Scientist
5 78 (8.02%) Software Engineer
6 66 (6.79%) Python Engineer
7 57 (5.86%) AWS Engineer
8 56 (5.76%) Big Data Engineer
9 40 (4.12%) Cloud Engineer
9 40 (4.12%) Senior Software Engineer
10 38 (3.91%) Java Engineer
11 35 (3.60%) Principal Engineer
12 34 (3.50%) Senior Developer
13 33 (3.40%) Java Developer
13 33 (3.40%) Lead Architect
13 33 (3.40%) Lead Data Architect
14 28 (2.88%) Consultant
15 27 (2.78%) Scala Engineer
Libraries, Frameworks & Software Standards
1 441 (45.37%) Kafka
2 88 (9.05%) Apache Beam
3 75 (7.72%) PySpark
4 52 (5.35%) React
4 52 (5.35%) TensorFlow
5 50 (5.14%) REST
6 40 (4.12%) Node.js
7 36 (3.70%) Spring
8 30 (3.09%) Pandas
9 28 (2.88%) Keras
10 27 (2.78%) RabbitMQ
11 26 (2.67%) AngularJS
11 26 (2.67%) scikit-learn
12 24 (2.47%) Spark Streaming
13 23 (2.37%) Apache NiFi
14 21 (2.16%) Spring Boot
15 19 (1.95%) Akka
16 18 (1.85%) Flask
16 18 (1.85%) Web Services
17 17 (1.75%) Dask
Miscellaneous
1 129 (13.27%) Algorithms
2 48 (4.94%) Cloud Native
3 45 (4.63%) Data Structures
4 42 (4.32%) Virtual Team
5 39 (4.01%) Self-Motivation
6 36 (3.70%) User Experience
7 34 (3.50%) Greenfield Project
8 30 (3.09%) Hybrid Cloud
9 21 (2.16%) Enterprise Software
10 19 (1.95%) Blog
10 19 (1.95%) Equities
11 15 (1.54%) Internet of Things
12 14 (1.44%) Public Cloud
13 11 (1.13%) Hedge funds
14 10 (1.03%) Management Information System
15 9 (0.93%) Derivative
16 8 (0.82%) Foreign Exchange (FX)
16 8 (0.82%) Life Science
16 8 (0.82%) Social Media
17 7 (0.72%) SWIFT
Operating Systems
1 66 (6.79%) Unix
2 59 (6.07%) Linux
3 16 (1.65%) Windows
4 9 (0.93%) Kali Linux
4 9 (0.93%) Red Hat Enterprise Linux
5 3 (0.31%) DC/OS
6 1 (0.10%) Android
6 1 (0.10%) Apple iOS
6 1 (0.10%) Windows Server
Processes & Methodologies
1 466 (47.94%) Data Science
2 462 (47.53%) Analytics
3 358 (36.83%) Agile Software Development
4 307 (31.58%) Machine Learning
5 278 (28.60%) Data Pipeline
6 267 (27.47%) Data Engineering
6 267 (27.47%) DevOps
7 241 (24.79%) CI/CD
7 241 (24.79%) ETL
8 219 (22.53%) Software Engineering
9 212 (21.81%) Artificial Intelligence
9 212 (21.81%) Data Modelling
10 165 (16.98%) Data Management
11 161 (16.56%) Computer Science
12 141 (14.51%) Microservices
13 136 (13.99%) Business Intelligence
14 129 (13.27%) Visualisation
15 127 (13.07%) Problem-Solving
16 106 (10.91%) Data Analytics
17 99 (10.19%) Data Architecture
Programming Languages
1 725 (74.59%) Python
2 465 (47.84%) Java
3 433 (44.55%) SQL
4 377 (38.79%) Scala
5 99 (10.19%) R
6 75 (7.72%) Go
7 67 (6.89%) JavaScript
8 50 (5.14%) C#
9 41 (4.22%) Bash
9 41 (4.22%) C++
10 27 (2.78%) PowerShell
10 27 (2.78%) Shell Script
11 24 (2.47%) TypeScript
12 18 (1.85%) Kotlin
13 15 (1.54%) C
13 15 (1.54%) MATLAB
14 13 (1.34%) Ruby
15 11 (1.13%) T-SQL
16 9 (0.93%) Clojure
16 9 (0.93%) Java 8
Qualifications
1 142 (14.61%) Degree
2 74 (7.61%) Computer Science Degree
3 23 (2.37%) PhD
4 20 (2.06%) Master's Degree
5 16 (1.65%) AWS Certification
5 16 (1.65%) Security Cleared
6 7 (0.72%) SC Cleared
7 4 (0.41%) Microsoft Certification
7 4 (0.41%) TOGAF Certification
8 3 (0.31%) CISSP
8 3 (0.31%) MCSA
9 2 (0.21%) (ISC)2 CCSP
9 2 (0.21%) CCA
9 2 (0.21%) CCSP
9 2 (0.21%) Cisco Certification
9 2 (0.21%) DV Cleared
10 1 (0.10%) CRISC
10 1 (0.10%) CTC Cleared
10 1 (0.10%) ISO 27001 Lead Auditor
10 1 (0.10%) ISO 27001 Lead Implementer
Quality Assurance & Compliance
1 73 (7.51%) Data Quality
2 39 (4.01%) QA
3 30 (3.09%) GDPR
4 9 (0.93%) PCI DSS
5 7 (0.72%) GxP
6 2 (0.21%) CRD IV
6 2 (0.21%) IFRS
6 2 (0.21%) IFRS 9
7 1 (0.10%) Dodd-Frank
7 1 (0.10%) EMIR
7 1 (0.10%) FATCA
7 1 (0.10%) ISO/IEC 27001
7 1 (0.10%) MiFID
System Software
1 236 (24.28%) Docker
2 25 (2.57%) HDFS
3 23 (2.37%) Apache Flume
4 8 (0.82%) VirtualBox
5 5 (0.51%) Apache ZooKeeper
6 2 (0.21%) Samba
7 1 (0.10%) Ceph
7 1 (0.10%) etcd
7 1 (0.10%) NFS
Systems Management
1 268 (27.57%) Kubernetes
2 68 (7.00%) Terraform
3 51 (5.25%) Ansible
4 34 (3.50%) Oozie
5 25 (2.57%) Opscode Chef
6 21 (2.16%) Puppet
7 14 (1.44%) Prometheus
8 12 (1.23%) Grafana
9 10 (1.03%) Hadoop YARN
9 10 (1.03%) Istio
10 8 (0.82%) Docker Swarm
11 7 (0.72%) Control-M
12 6 (0.62%) Consul
13 4 (0.41%) Mesos
14 3 (0.31%) Apache Brooklyn
14 3 (0.31%) Knative
14 3 (0.31%) Marathon
15 2 (0.21%) Ambari
15 2 (0.21%) Kubeflow
15 2 (0.21%) vRealize Orchestrator
Vendors
1 160 (16.46%) Google
2 117 (12.04%) Databricks
3 66 (6.79%) Cloudera
4 57 (5.86%) Microsoft
5 52 (5.35%) Oracle
6 40 (4.12%) Hortonworks
7 36 (3.70%) Talend
8 34 (3.50%) Informatica
9 23 (2.37%) Apple
10 19 (1.95%) BEA
10 19 (1.95%) CrowdStrike
10 19 (1.95%) Spotify
11 16 (1.65%) Splunk
12 15 (1.54%) IBM
13 14 (1.44%) SAP
14 12 (1.23%) Cognizant
14 12 (1.23%) Salesforce.com
15 11 (1.13%) StreamSets
15 11 (1.13%) Teradata
16 9 (0.93%) Alteryx