Apache Spark
UK

The following table provides summary statistics for permanent job vacancies with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 28 May 2024 with a comparison to the same period in the previous 2 years.

6 months to
28 May 2024
Same period 2023 Same period 2022
Rank 293 253 247
Rank change year-on-year -40 -6 +13
Permanent jobs citing Apache Spark 1,003 1,410 2,773
As % of all permanent jobs advertised in the UK 0.98% 1.45% 1.66%
As % of the Application Platforms category 9.42% 12.70% 11.79%
Number of salaries quoted 280 577 903
10th Percentile £48,850 £59,900 £50,000
25th Percentile £63,750 £67,500 £58,750
Median annual salary (50th Percentile) £80,000 £87,500 £77,500
Median % change year-on-year -8.57% +12.90% +3.33%
75th Percentile £105,000 £110,000 £92,500
90th Percentile £118,750 £150,000 £108,750
UK excluding London median annual salary £69,724 £70,000 £62,500
% change year-on-year -0.40% +12.00% -

All Application Platform Skills
UK

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for application platform skills.

Permanent vacancies with a requirement for application platform skills 10,651 11,104 23,524
As % of all permanent jobs advertised in the UK 10.38% 11.41% 14.09%
Number of salaries quoted 7,087 7,484 11,651
10th Percentile £27,500 £31,250 £28,750
25th Percentile £34,125 £40,000 £37,885
Median annual salary (50th Percentile) £50,000 £55,022 £55,000
Median % change year-on-year -9.13% +0.04% +4.76%
75th Percentile £70,000 £77,500 £73,750
90th Percentile £97,500 £95,000 £92,500
UK excluding London median annual salary £42,500 £47,672 £45,000
% change year-on-year -10.85% +5.94% -

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a proportion of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Salary Trend

3-month moving average salary quoted in jobs citing Apache Spark.

Salary trend for Apache Spark in the UK

Apache Spark
Salary Histogram

Salary distribution for jobs citing Apache Spark over the 6 months to 28 May 2024.

Salary histogram for Apache Spark in the UK

Apache Spark
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the UK over the 6 months to 28 May 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
UK excluding London +110 733 £69,724 -0.40% 60
Scotland +156 601 £47,500 -20.83% 8
England -253 383 £80,000 -10.61% 139
London -130 248 £90,000 -5.26% 94
Work from Home -218 167 £85,000 -15.00% 70
South West -49 51 £80,000 +6.67% 15
North of England -102 42 £60,000 -14.29% 18
North West -14 35 £62,500 -14.68% 12
Midlands -16 16 £72,750 +3.93% 4
South East -54 14 £52,000 -25.71% 8
East Midlands -20 10 £60,000 - 2
West Midlands +4 6 £73,000 +4.29% 2
Yorkshire -43 6 £51,528 -25.13% 4
East of England -47 5 £101,250 +102.50% 5
Wales +2 3 £72,500 +38.10% 1
Northern Ireland +18 1 £70,000 -30.00% 3
North East -24 1 £75,000 - 2

Apache Spark
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 289 (28.81%) Apache
2 169 (16.85%) Apache Airflow
3 27 (2.69%) Jupyter
4 17 (1.69%) nginx
5 6 (0.60%) Confluence
6 5 (0.50%) Apache Pulsar
6 5 (0.50%) SAS
7 2 (0.20%) Luigi
8 1 (0.100%) Apache Pig
8 1 (0.100%) Apache Solr
Applications
1 7 (0.70%) Microsoft Excel
2 4 (0.40%) Spreadsheet
3 1 (0.100%) Microsoft Office
Business Applications
1 4 (0.40%) Primavera
Cloud Services
1 688 (68.59%) AWS
2 213 (21.24%) Azure
3 175 (17.45%) GCP
4 104 (10.37%) Serverless
5 99 (9.87%) Amazon EKS
6 96 (9.57%) AWS Lambda
7 74 (7.38%) Amazon SQS
8 73 (7.28%) Amazon EMR
9 64 (6.38%) Amazon S3
10 57 (5.68%) Amazon ECS
11 55 (5.48%) Amazon EC2
12 47 (4.69%) OpenShift
12 47 (4.69%) Snowflake
13 43 (4.29%) AWS CloudFormation
14 39 (3.89%) Power Platform
15 37 (3.69%) Azure Synapse Analytics
16 36 (3.59%) Azure Data Factory
17 26 (2.59%) Amazon Kinesis
17 26 (2.59%) SaaS
18 25 (2.49%) GitHub
Communications & Networking
1 28 (2.79%) AMQP
2 17 (1.69%) DSL
3 16 (1.60%) Wireless
4 5 (0.50%) Firewall
4 5 (0.50%) Network Security
5 2 (0.20%) HTTP
Database & Business Intelligence
1 701 (69.89%) Big Data
2 411 (40.98%) Hadoop
3 410 (40.88%) NoSQL
4 161 (16.05%) Apache Hive
5 103 (10.27%) HBase
6 102 (10.17%) Data Lake
7 87 (8.67%) Data Warehouse
8 55 (5.48%) Electronic Patient Records
9 46 (4.59%) Elasticsearch
10 45 (4.49%) Flink
11 44 (4.39%) Amazon Redshift
11 44 (4.39%) Data Mining
12 42 (4.19%) SQL Server
13 38 (3.79%) Tableau
14 37 (3.69%) Power BI
15 35 (3.49%) BigQuery
16 31 (3.09%) Presto
17 28 (2.79%) Cloudera CDH
18 27 (2.69%) Relational Database
19 23 (2.29%) MongoDB
Development Applications
1 138 (13.76%) Jenkins
2 128 (12.76%) Git
3 85 (8.47%) Bitbucket Server
4 17 (1.69%) Atlassian Bamboo
5 13 (1.30%) Spinnaker
6 8 (0.80%) Oracle Forms
6 8 (0.80%) Visual Studio
7 7 (0.70%) CircleCI
7 7 (0.70%) JIRA
8 5 (0.50%) MLflow
9 4 (0.40%) Bitbucket
10 3 (0.30%) Deequ
10 3 (0.30%) Octopus Deploy
11 2 (0.20%) Team Foundation Server
12 1 (0.100%) Ant
12 1 (0.100%) ERwin
12 1 (0.100%) GitLab
12 1 (0.100%) Gradle
12 1 (0.100%) JUnit
12 1 (0.100%) Maven
General
1 645 (64.31%) Finance
2 619 (61.71%) Inclusion and Diversity
3 587 (58.52%) Law
4 318 (31.70%) Marketing
5 276 (27.52%) Legal
6 162 (16.15%) Influencing Skills
7 113 (11.27%) Social Skills
8 91 (9.07%) Analytical Skills
9 70 (6.98%) Financial Institution
10 60 (5.98%) Banking
11 28 (2.79%) Aerospace
12 21 (2.09%) Investment Banking
12 21 (2.09%) Retail
13 19 (1.89%) Presentation Skills
14 18 (1.79%) Games
15 14 (1.40%) Electronics
16 12 (1.20%) Private Banking
16 12 (1.20%) Public Sector
17 7 (0.70%) Retail Banking
18 4 (0.40%) Music Industry
Job Titles
1 589 (58.72%) Software Engineer
2 390 (38.88%) Big Data Engineer
3 372 (37.09%) Big Data Software Engineer
4 330 (32.90%) Data Engineer
5 227 (22.63%) Lead
6 219 (21.83%) Senior
7 216 (21.54%) Scala Engineer
7 216 (21.54%) Scala Software Engineer
8 191 (19.04%) Java Engineer
9 190 (18.94%) Java Software Engineer
10 189 (18.84%) AWS Engineer
11 186 (18.54%) Lead Software Engineer
12 177 (17.65%) AWS Software Engineer
13 146 (14.56%) Lead Big Data Engineer
14 135 (13.46%) Python Engineer
15 119 (11.86%) Java Scala Software Engineer
16 114 (11.37%) Python Software Engineer
17 104 (10.37%) Senior Software Engineer
18 97 (9.67%) Lead Java Engineer
19 88 (8.77%) Developer
Libraries, Frameworks & Software Standards
1 208 (20.74%) Kafka
2 168 (16.75%) PySpark
3 114 (11.37%) Spring
4 87 (8.67%) Spring Boot
5 65 (6.48%) React
6 52 (5.18%) RabbitMQ
7 51 (5.08%) AngularJS
8 47 (4.69%) REST
9 32 (3.19%) Node.js
10 31 (3.09%) Apache Beam
11 29 (2.89%) gRPC
11 29 (2.89%) OpenAPI
11 29 (2.89%) Swagger
12 28 (2.79%) Material-UI
13 23 (2.29%) Pandas
14 22 (2.19%) ActiveMQ
14 22 (2.19%) Apache NiFi
15 20 (1.99%) Elastic Stack
16 17 (1.69%) GraphQL
16 17 (1.69%) TensorFlow
Miscellaneous
1 247 (24.63%) Algorithms
2 228 (22.73%) Cloud Native
3 61 (6.08%) Data Structures
4 42 (4.19%) Legacy Code
5 40 (3.99%) Management Information System
6 34 (3.39%) Public Cloud
7 30 (2.99%) Distributed Systems
8 26 (2.59%) Product Ownership
9 19 (1.89%) Hybrid Cloud
10 15 (1.50%) Greenfield Project
11 14 (1.40%) Legacy Systems
12 11 (1.10%) Enterprise Software
13 9 (0.90%) Distributed Applications
14 8 (0.80%) BBC
15 7 (0.70%) Distributed Computing
16 6 (0.60%) Mainframe
16 6 (0.60%) Onboarding
17 4 (0.40%) IoT
17 4 (0.40%) Renewable Energy
18 3 (0.30%) Analytical Mindset
Operating Systems
1 103 (10.27%) Unix
2 68 (6.78%) Linux
3 28 (2.79%) CentOS
3 28 (2.79%) Debian
3 28 (2.79%) Red Hat Enterprise Linux
3 28 (2.79%) Ubuntu
Processes & Methodologies
1 713 (71.09%) Software Engineering
2 707 (70.49%) Agile
3 611 (60.92%) CI/CD
4 547 (54.54%) SDLC
5 488 (48.65%) Operational Stability
6 385 (38.38%) Machine Learning
7 325 (32.40%) Artificial Intelligence
8 293 (29.21%) Risk Management
9 268 (26.72%) Visualisation
10 259 (25.82%) Community of Practice
11 228 (22.73%) Performance Tuning
12 211 (21.04%) Continuous Improvement
13 207 (20.64%) Problem-Solving
14 198 (19.74%) Microservices
15 178 (17.75%) Decision-Making
16 176 (17.55%) Data Engineering
17 161 (16.05%) Data Modelling
18 152 (15.15%) Data Pipeline
19 138 (13.76%) Open Source
20 136 (13.56%) ETL
Programming Languages
1 607 (60.52%) SQL
2 600 (59.82%) Java
3 576 (57.43%) Python
4 392 (39.08%) Scala
5 103 (10.27%) PL/SQL
6 44 (4.39%) Go
7 35 (3.49%) Bash
8 34 (3.39%) JavaScript
9 31 (3.09%) Kotlin
10 25 (2.49%) C
11 24 (2.39%) R
12 22 (2.19%) C#
13 21 (2.09%) Shell Script
14 18 (1.79%) C++
15 12 (1.20%) TypeScript
16 8 (0.80%) Clojure
16 8 (0.80%) Data Analysis Expressions
16 8 (0.80%) MATLAB
17 6 (0.60%) Rust
18 4 (0.40%) VB
Qualifications
1 124 (12.36%) Degree
2 58 (5.78%) Security Cleared
3 51 (5.08%) DV Cleared
4 37 (3.69%) SC Cleared
5 34 (3.39%) Computer Science Degree
6 21 (2.09%) Master's Degree
7 11 (1.10%) Microsoft Certification
7 11 (1.10%) PhD
8 6 (0.60%) Azure Certification
8 6 (0.60%) BPSS Clearance
8 6 (0.60%) DBS Check
9 4 (0.40%) CIMA
9 4 (0.40%) Numerate Degree
10 3 (0.30%) ISTQB
11 1 (0.100%) AWS Certification
11 1 (0.100%) AWS Certified Solutions Architect
Quality Assurance & Compliance
1 70 (6.98%) Data Quality
2 36 (3.59%) SLA
3 21 (2.09%) QA
4 4 (0.40%) Accessibility
4 4 (0.40%) Solvency II
5 3 (0.30%) CRD IV
5 3 (0.30%) IFRS
5 3 (0.30%) IFRS 9
6 2 (0.20%) GDPR
System Software
1 114 (11.37%) HDFS
2 67 (6.68%) Docker
3 17 (1.69%) Virtual Machines
4 6 (0.60%) Ceph
5 3 (0.30%) Virtual Servers
6 2 (0.20%) IAG
7 1 (0.100%) Apache Flume
7 1 (0.100%) Slurm Workload Manager
Systems Management
1 201 (20.04%) Kubernetes
2 186 (18.54%) Terraform
3 66 (6.58%) Ansible
4 32 (3.19%) Puppet
5 18 (1.79%) Prometheus
6 17 (1.69%) Grafana
6 17 (1.69%) HAProxy
7 6 (0.60%) Knative
8 4 (0.40%) Kibana
9 3 (0.30%) Progress Chef
10 2 (0.20%) Argo
10 2 (0.20%) Kubeflow
11 1 (0.100%) logstash
11 1 (0.100%) Podman
Vendors
1 256 (25.52%) Databricks
2 112 (11.17%) Google
3 91 (9.07%) Informatica
4 88 (8.77%) Cloudera
5 64 (6.38%) Microsoft
6 33 (3.29%) Oracle
7 28 (2.79%) Red Hat
7 28 (2.79%) SaltStack
8 19 (1.89%) Cognos
9 18 (1.79%) Alteryx
10 17 (1.69%) Dell
10 17 (1.69%) Sybase
11 16 (1.60%) Teradata
12 6 (0.60%) YouTube
13 5 (0.50%) Confluent
14 4 (0.40%) Apple
14 4 (0.40%) Atlassian
14 4 (0.40%) Hortonworks
14 4 (0.40%) SAP
14 4 (0.40%) Splunk