Apache Spark
UK

The following table provides summary statistics for permanent job vacancies with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 10 December 2023 with a comparison to the same period in the previous 2 years.

6 months to
10 Dec 2023
Same period 2022 Same period 2021
Rank 348 277 267
Rank change year-on-year -71 -10 -12
Permanent jobs citing Apache Spark 450 1,615 1,760
As % of all permanent jobs advertised in the UK 0.86% 1.32% 1.38%
As % of the Application Platforms category 8.63% 11.33% 11.12%
Number of salaries quoted 241 723 917
10th Percentile £50,172 £52,500 £49,150
25th Percentile £67,500 £61,250 £60,000
Median annual salary (50th Percentile) £79,632 £80,000 £77,500
Median % change year-on-year -0.46% +3.23% +14.81%
75th Percentile £97,290 £100,000 £96,250
90th Percentile £115,000 £116,000 £112,500
UK excluding London median annual salary £75,000 £65,000 £67,500
% change year-on-year +15.38% -3.70% +22.73%

All Application Platform Skills
UK

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for application platform skills.

Permanent vacancies with a requirement for application platform skills 5,213 14,258 15,833
As % of all permanent jobs advertised in the UK 9.97% 11.67% 12.46%
Number of salaries quoted 3,883 8,496 10,303
10th Percentile £31,500 £31,250 £30,000
25th Percentile £40,500 £41,250 £37,750
Median annual salary (50th Percentile) £55,000 £57,500 £55,000
Median % change year-on-year -4.35% +4.55% +4.76%
75th Percentile £77,500 £78,750 £72,500
90th Percentile £95,000 £97,270 £90,000
UK excluding London median annual salary £49,944 £50,000 £45,369
% change year-on-year -0.11% +10.21% +0.82%

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a proportion of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Salary Trend

3-month moving average salary quoted in jobs citing Apache Spark.

Salary trend for Apache Spark in the UK

Apache Spark
Salary Histogram

Salary distribution for jobs citing Apache Spark over the 6 months to 10 December 2023.

Salary histogram for Apache Spark in the UK

Apache Spark
Top 15 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the UK over the 6 months to 10 December 2023. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
England -106 339 £79,632 -0.46% 155
UK excluding London +31 221 £75,000 +15.38% 69
London -72 209 £90,000 +2.86% 109
Work from Home -30 192 £79,632 -0.46% 70
Scotland +85 99 £79,632 +32.72% 4
South West +79 46 £75,000 +22.59% 4
North of England +13 36 £69,447 +6.84% 18
South East +81 22 £75,000 +25.00% 26
Yorkshire +4 19 £51,528 -15.87% 6
North West +86 17 £69,447 +1.01% 11
Midlands +76 10 £70,000 -9.68% 5
West Midlands +92 7 £70,000 -6.67% 1
East of England +54 4 - - 6
Wales +6 4 £51,528 +3.06% 5
East Midlands +31 3 £65,000 -18.75% 4

Apache Spark
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 151 (33.56%) Apache
2 53 (11.78%) Apache Airflow
3 15 (3.33%) SAS
4 13 (2.89%) Jupyter
5 10 (2.22%) nginx
6 3 (0.67%) Apache Solr
6 3 (0.67%) Confluence
7 2 (0.44%) JBoss
7 2 (0.44%) Luigi
7 2 (0.44%) Tomcat
7 2 (0.44%) WebSphere
8 1 (0.22%) Apache Pulsar
8 1 (0.22%) Apache Synapse
Applications
1 4 (0.89%) MS Visio
Cloud Services
1 233 (51.78%) AWS
2 158 (35.11%) Azure
3 157 (34.89%) GCP
4 68 (15.11%) Power Platform
5 67 (14.89%) GitHub
6 50 (11.11%) GitHub Actions
7 49 (10.89%) Amazon S3
8 47 (10.44%) Vertex AI
9 31 (6.89%) Serverless
10 23 (5.11%) AWS Lambda
11 19 (4.22%) Amazon EKS
11 19 (4.22%) Amazon EMR
11 19 (4.22%) Amazon SQS
12 18 (4.00%) Azure Data Factory
12 18 (4.00%) Google Kubernetes Engine
13 16 (3.56%) Azure Synapse Analytics
13 16 (3.56%) OpenShift
13 16 (3.56%) SaaS
13 16 (3.56%) Snowflake
14 14 (3.11%) Amazon Kinesis
Communications & Networking
1 30 (6.67%) Wireless
2 11 (2.44%) Firewall
3 6 (1.33%) DSL
3 6 (1.33%) SSH
4 5 (1.11%) AMQP
4 5 (1.11%) Network Security
4 5 (1.11%) WAN
5 4 (0.89%) Internet
6 1 (0.22%) HTTP
Database & Business Intelligence
1 303 (67.33%) Big Data
2 196 (43.56%) Hadoop
3 139 (30.89%) Data Lake
4 134 (29.78%) NoSQL
5 83 (18.44%) Data Warehouse
6 75 (16.67%) Apache Hive
7 73 (16.22%) Relational Database
8 72 (16.00%) BigQuery
9 68 (15.11%) Power BI
10 59 (13.11%) PostgreSQL
11 58 (12.89%) Tableau
12 50 (11.11%) Looker
13 49 (10.89%) Bigtable
13 49 (10.89%) MySQL
14 43 (9.56%) Elasticsearch
15 35 (7.78%) Amazon Redshift
16 34 (7.56%) Presto
17 27 (6.00%) Flink
18 23 (5.11%) Data Mining
19 20 (4.44%) Apache Ignite
Development Applications
1 87 (19.33%) Jenkins
2 23 (5.11%) Git
3 11 (2.44%) Spinnaker
4 10 (2.22%) Atlassian Bamboo
5 8 (1.78%) JUnit
6 6 (1.33%) Cucumber
6 6 (1.33%) ScalaTest
6 6 (1.33%) WebDriver
7 5 (1.11%) Bitbucket Server
7 5 (1.11%) Oracle Forms
8 3 (0.67%) JIRA
8 3 (0.67%) Maven
9 2 (0.44%) CircleCI
10 1 (0.22%) Ant
10 1 (0.22%) Gradle
10 1 (0.22%) Postman
10 1 (0.22%) TeamCity
General
1 190 (42.22%) Finance
2 106 (23.56%) Banking
3 96 (21.33%) Inclusion and Diversity
4 92 (20.44%) Social Skills
5 79 (17.56%) Analytical Skills
6 78 (17.33%) Law
7 50 (11.11%) Marketing
8 49 (10.89%) Retail
9 32 (7.11%) Influencing Skills
10 31 (6.89%) Legal
11 24 (5.33%) Presentation Skills
12 12 (2.67%) Financial Institution
13 9 (2.00%) Retail Finance
14 8 (1.78%) Games
15 7 (1.56%) Investment Banking
15 7 (1.56%) Manufacturing
16 6 (1.33%) Private Banking
16 6 (1.33%) Retail Banking
17 2 (0.44%) German Language
17 2 (0.44%) Local Government
Job Titles
1 148 (32.89%) Data Engineer
2 99 (22.00%) Senior
3 94 (20.89%) Software Engineer
4 74 (16.44%) Lead
5 66 (14.67%) Architect
6 64 (14.22%) Data Architect
7 51 (11.33%) Big Data Engineer
8 40 (8.89%) Cloud Architect
9 38 (8.44%) Big Data Software Engineer
10 35 (7.78%) Developer
11 33 (7.33%) Java Engineer
12 32 (7.11%) Java Software Engineer
13 31 (6.89%) AWS Engineer
14 29 (6.44%) Lead Data Engineer
14 29 (6.44%) Senior Data Engineer
15 28 (6.22%) Scala Engineer
15 28 (6.22%) Scala Software Engineer
16 26 (5.78%) AWS Software Engineer
17 24 (5.33%) GCP Engineer
17 24 (5.33%) Java Developer
Libraries, Frameworks & Software Standards
1 167 (37.11%) Kafka
2 40 (8.89%) Spring
3 39 (8.67%) PySpark
4 25 (5.56%) REST
5 24 (5.33%) Pandas
5 24 (5.33%) React
6 23 (5.11%) Spring Boot
7 21 (4.67%) JSON
8 20 (4.44%) .NET
9 18 (4.00%) Apache NiFi
9 18 (4.00%) RabbitMQ
10 16 (3.56%) AngularJS
11 15 (3.33%) ActiveMQ
12 14 (3.11%) Apache Beam
12 14 (3.11%) Node.js
12 14 (3.11%) XML
13 11 (2.44%) Elastic Stack
13 11 (2.44%) GraphQL
13 11 (2.44%) scikit-learn
14 10 (2.22%) NumPy
Miscellaneous
1 77 (17.11%) Algorithms
2 42 (9.33%) Cloud Native
3 35 (7.78%) Public Cloud
4 30 (6.67%) Data Structures
5 26 (5.78%) Distributed Systems
6 15 (3.33%) Credit Risk
7 14 (3.11%) Greenfield Project
8 13 (2.89%) Management Information System
8 13 (2.89%) Onboarding
9 12 (2.67%) Hybrid Cloud
10 11 (2.44%) Distributed Computing
10 11 (2.44%) Product Ownership
11 10 (2.22%) IoT
12 8 (1.78%) Hedge funds
13 7 (1.56%) Legacy Code
14 6 (1.33%) Self-Motivation
15 5 (1.11%) Cyber Defence
15 5 (1.11%) Legacy Systems
15 5 (1.11%) Smart Meter
16 4 (0.89%) Enterprise Software
Operating Systems
1 44 (9.78%) Linux
2 16 (3.56%) Unix
3 11 (2.44%) CentOS
4 10 (2.22%) Red Hat Enterprise Linux
5 5 (1.11%) Debian
5 5 (1.11%) Ubuntu
Processes & Methodologies
1 208 (46.22%) Agile
2 175 (38.89%) CI/CD
3 163 (36.22%) Machine Learning
4 160 (35.56%) Analytics
5 137 (30.44%) ETL
6 134 (29.78%) Software Engineering
7 130 (28.89%) Data Pipeline
8 129 (28.67%) Visualisation
9 128 (28.44%) Artificial Intelligence
10 107 (23.78%) Data Engineering
11 99 (22.00%) Data Science
12 98 (21.78%) Business Intelligence
13 93 (20.67%) Data Visualisation
14 91 (20.22%) Data Analytics
15 87 (19.33%) Data Integration
15 87 (19.33%) Data Modelling
16 81 (18.00%) SDLC
17 68 (15.11%) Infrastructure as Code
18 65 (14.44%) Operational Stability
19 63 (14.00%) Problem-Solving
Programming Languages
1 243 (54.00%) Python
2 186 (41.33%) SQL
3 180 (40.00%) Java
4 112 (24.89%) Scala
5 46 (10.22%) R
6 31 (6.89%) Bash
7 25 (5.56%) Go
8 24 (5.33%) C#
9 19 (4.22%) C++
10 12 (2.67%) JavaScript
10 12 (2.67%) Shell Script
11 11 (2.44%) PL/SQL
12 10 (2.22%) VB
13 6 (1.33%) TypeScript
14 5 (1.11%) C
14 5 (1.11%) Data Analysis Expressions
15 4 (0.89%) PowerShell
16 2 (0.44%) Perl
16 2 (0.44%) XPath
17 1 (0.22%) MATLAB
Qualifications
1 96 (21.33%) Degree
2 35 (7.78%) Security Cleared
3 27 (6.00%) Computer Science Degree
4 26 (5.78%) Master's Degree
5 16 (3.56%) SC Cleared
6 13 (2.89%) DV Cleared
7 7 (1.56%) Azure Certification
8 4 (0.89%) Microsoft Certification
9 3 (0.67%) AWS Certification
10 1 (0.22%) CCA
10 1 (0.22%) Numerate Degree
10 1 (0.22%) PhD
Quality Assurance & Compliance
1 43 (9.56%) Data Quality
2 15 (3.33%) CRD IV
2 15 (3.33%) IFRS
2 15 (3.33%) IFRS 9
3 10 (2.22%) SLA
4 4 (0.89%) Accessibility
5 2 (0.44%) QA
6 1 (0.22%) GDPR
6 1 (0.22%) HIPAA
System Software
1 42 (9.33%) Docker
2 23 (5.11%) HDFS
3 4 (0.89%) Apache Flume
4 1 (0.22%) Slurm Workload Manager
Systems Management
1 86 (19.11%) Terraform
2 80 (17.78%) Kubernetes
3 21 (4.67%) Ansible
4 12 (2.67%) Grafana
5 11 (2.44%) Prometheus
6 10 (2.22%) HAProxy
7 8 (1.78%) Oozie
8 6 (1.33%) Puppet
9 2 (0.44%) Apache Ranger
9 2 (0.44%) Hadoop YARN
9 2 (0.44%) Kibana
10 1 (0.22%) Control-M
10 1 (0.22%) logstash
10 1 (0.22%) Progress Chef
10 1 (0.22%) Rancher
Vendors
1 106 (23.56%) Databricks
2 98 (21.78%) Google
3 39 (8.67%) Microsoft
4 18 (4.00%) Cloudera
5 14 (3.11%) Confluent
6 12 (2.67%) Dell
7 11 (2.44%) Oracle
8 10 (2.22%) Atlassian
8 10 (2.22%) Red Hat
9 6 (1.33%) Alteryx
9 6 (1.33%) Cognos
9 6 (1.33%) Informatica
9 6 (1.33%) Sybase
10 5 (1.11%) SaltStack
10 5 (1.11%) Splunk
10 5 (1.11%) Teradata
11 4 (0.89%) IBM
11 4 (0.89%) Talend
12 2 (0.44%) Adobe
12 2 (0.44%) Peregrine