Apache Spark
UK

The following table provides summary statistics for permanent job vacancies with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 23 May 2024 with a comparison to the same period in the previous 2 years.

6 months to
23 May 2024
Same period 2023 Same period 2022
Rank 283 255 249
Rank change year-on-year -28 -6 +15
Permanent jobs citing Apache Spark 1,025 1,424 2,690
As % of all permanent jobs advertised in the UK 1.02% 1.45% 1.64%
As % of the Application Platforms category 9.61% 12.68% 11.64%
Number of salaries quoted 289 583 893
10th Percentile £49,800 £57,750 £50,000
25th Percentile £62,500 £67,500 £58,000
Median annual salary (50th Percentile) £80,000 £87,500 £77,500
Median % change year-on-year -8.57% +12.90% +3.33%
75th Percentile £102,500 £110,000 £92,500
90th Percentile £118,750 £150,000 £109,750
UK excluding London median annual salary £69,447 £70,000 £62,500
% change year-on-year -0.79% +12.00% -

All Application Platform Skills
UK

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for application platform skills.

Permanent vacancies with a requirement for application platform skills 10,663 11,229 23,116
As % of all permanent jobs advertised in the UK 10.57% 11.40% 14.11%
Number of salaries quoted 7,085 7,564 11,607
10th Percentile £27,500 £31,250 £28,750
25th Percentile £34,000 £40,000 £37,750
Median annual salary (50th Percentile) £50,000 £55,043 £55,000
Median % change year-on-year -9.16% +0.08% +4.76%
75th Percentile £70,000 £77,500 £73,750
90th Percentile £96,250 £95,000 £92,500
UK excluding London median annual salary £42,500 £48,000 £45,000
% change year-on-year -11.46% +6.67% -

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a proportion of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Salary Trend

3-month moving average salary quoted in jobs citing Apache Spark.

Salary trend for Apache Spark in the UK

Apache Spark
Salary Histogram

Salary distribution for jobs citing Apache Spark over the 6 months to 23 May 2024.

Salary histogram for Apache Spark in the UK

Apache Spark
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the UK over the 6 months to 23 May 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
UK excluding London +121 754 £69,447 -0.79% 113
Scotland +150 616 £47,500 -20.83% 11
England -236 389 £80,000 -8.83% 181
London -133 249 £90,000 -5.26% 134
Work from Home -222 177 £80,000 -20.00% 83
South West -35 52 £80,000 +6.67% 16
North of England -105 46 £60,000 -14.29% 16
North West -17 37 £62,500 -14.68% 9
Midlands -21 16 £72,750 +3.93% 8
South East -47 14 £52,000 -25.71% 54
East Midlands -11 10 £60,000 - 4
Yorkshire -38 8 £51,528 -20.73% 4
West Midlands +1 6 £73,000 +4.29% 4
East of England -35 5 £101,250 +102.50% 3
Wales +9 4 £72,500 +38.10% 1
Northern Ireland +25 1 £70,000 -30.00% 4
North East -24 1 £75,000 - 3

Apache Spark
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 295 (28.78%) Apache
2 172 (16.78%) Apache Airflow
3 27 (2.63%) Jupyter
4 17 (1.66%) nginx
5 7 (0.68%) SAS
6 6 (0.59%) Confluence
7 3 (0.29%) Apache Pulsar
8 2 (0.20%) Apache Solr
8 2 (0.20%) Luigi
9 1 (0.098%) Apache Pig
Applications
1 7 (0.68%) Microsoft Excel
2 4 (0.39%) Spreadsheet
3 1 (0.098%) Microsoft Office
Business Applications
1 4 (0.39%) Primavera
Cloud Services
1 702 (68.49%) AWS
2 213 (20.78%) Azure
3 179 (17.46%) GCP
4 105 (10.24%) Serverless
5 101 (9.85%) Amazon EKS
6 97 (9.46%) AWS Lambda
7 78 (7.61%) Amazon SQS
8 73 (7.12%) Amazon EMR
9 67 (6.54%) Amazon S3
10 58 (5.66%) Amazon ECS
11 56 (5.46%) Amazon EC2
12 46 (4.49%) OpenShift
12 46 (4.49%) Snowflake
13 42 (4.10%) AWS CloudFormation
14 39 (3.80%) Power Platform
15 37 (3.61%) Azure Synapse Analytics
16 36 (3.51%) Azure Data Factory
17 29 (2.83%) SaaS
18 28 (2.73%) GitHub
19 25 (2.44%) Amazon Kinesis
Communications & Networking
1 28 (2.73%) AMQP
2 18 (1.76%) DSL
3 16 (1.56%) Wireless
4 7 (0.68%) Firewall
5 5 (0.49%) Network Security
6 2 (0.20%) HTTP
Database & Business Intelligence
1 713 (69.56%) Big Data
2 421 (41.07%) Hadoop
3 419 (40.88%) NoSQL
4 166 (16.20%) Apache Hive
5 104 (10.15%) Data Lake
5 104 (10.15%) HBase
6 86 (8.39%) Data Warehouse
7 55 (5.37%) Electronic Patient Records
8 50 (4.88%) Elasticsearch
9 44 (4.29%) Data Mining
9 44 (4.29%) Flink
10 43 (4.20%) Amazon Redshift
11 42 (4.10%) SQL Server
12 39 (3.80%) Tableau
13 37 (3.61%) Power BI
14 33 (3.22%) BigQuery
15 31 (3.02%) Presto
16 28 (2.73%) Cloudera CDH
17 27 (2.63%) Relational Database
18 24 (2.34%) Redis
Development Applications
1 141 (13.76%) Jenkins
2 129 (12.59%) Git
3 86 (8.39%) Bitbucket Server
4 17 (1.66%) Atlassian Bamboo
5 14 (1.37%) Spinnaker
6 8 (0.78%) CircleCI
6 8 (0.78%) Oracle Forms
6 8 (0.78%) Visual Studio
7 7 (0.68%) JIRA
8 5 (0.49%) MLflow
9 4 (0.39%) Bitbucket
10 3 (0.29%) Deequ
10 3 (0.29%) Octopus Deploy
11 2 (0.20%) Team Foundation Server
12 1 (0.098%) Ant
12 1 (0.098%) ERwin
12 1 (0.098%) Gradle
12 1 (0.098%) JUnit
12 1 (0.098%) Maven
General
1 664 (64.78%) Finance
2 633 (61.76%) Inclusion and Diversity
3 597 (58.24%) Law
4 323 (31.51%) Marketing
5 280 (27.32%) Legal
6 165 (16.10%) Influencing Skills
7 118 (11.51%) Social Skills
8 97 (9.46%) Analytical Skills
9 72 (7.02%) Financial Institution
10 68 (6.63%) Banking
11 28 (2.73%) Aerospace
12 24 (2.34%) Retail
13 21 (2.05%) Investment Banking
13 21 (2.05%) Presentation Skills
14 20 (1.95%) Games
15 14 (1.37%) Electronics
16 13 (1.27%) Private Banking
17 12 (1.17%) Public Sector
18 8 (0.78%) Retail Banking
19 6 (0.59%) Retail Finance
Job Titles
1 603 (58.83%) Software Engineer
2 396 (38.63%) Big Data Engineer
3 378 (36.88%) Big Data Software Engineer
4 337 (32.88%) Data Engineer
5 229 (22.34%) Lead
6 226 (22.05%) Senior
7 220 (21.46%) Scala Engineer
7 220 (21.46%) Scala Software Engineer
8 194 (18.93%) Java Engineer
9 193 (18.83%) AWS Engineer
9 193 (18.83%) Java Software Engineer
10 188 (18.34%) Lead Software Engineer
11 181 (17.66%) AWS Software Engineer
12 148 (14.44%) Lead Big Data Engineer
13 138 (13.46%) Python Engineer
14 119 (11.61%) Java Scala Software Engineer
15 116 (11.32%) Python Software Engineer
16 106 (10.34%) Senior Software Engineer
17 97 (9.46%) Lead Java Engineer
18 91 (8.88%) Developer
Libraries, Frameworks & Software Standards
1 214 (20.88%) Kafka
2 171 (16.68%) PySpark
3 117 (11.41%) Spring
4 88 (8.59%) Spring Boot
5 66 (6.44%) React
6 52 (5.07%) AngularJS
7 50 (4.88%) RabbitMQ
8 49 (4.78%) REST
9 33 (3.22%) Apache Beam
10 32 (3.12%) Node.js
11 29 (2.83%) gRPC
12 28 (2.73%) Material-UI
12 28 (2.73%) OpenAPI
12 28 (2.73%) Swagger
13 24 (2.34%) Pandas
14 23 (2.24%) ActiveMQ
15 22 (2.15%) Apache NiFi
16 20 (1.95%) Elastic Stack
16 20 (1.95%) GraphQL
17 17 (1.66%) YAML
Miscellaneous
1 256 (24.98%) Algorithms
2 230 (22.44%) Cloud Native
3 65 (6.34%) Data Structures
4 43 (4.20%) Legacy Code
5 42 (4.10%) Management Information System
6 35 (3.41%) Public Cloud
7 31 (3.02%) Distributed Systems
8 27 (2.63%) Product Ownership
9 19 (1.85%) Hybrid Cloud
10 16 (1.56%) Greenfield Project
11 15 (1.46%) Legacy Systems
12 11 (1.07%) Enterprise Software
13 10 (0.98%) Distributed Applications
14 8 (0.78%) BBC
15 7 (0.68%) Distributed Computing
16 6 (0.59%) Credit Risk
16 6 (0.59%) Mainframe
16 6 (0.59%) Onboarding
17 4 (0.39%) IoT
17 4 (0.39%) Renewable Energy
Operating Systems
1 105 (10.24%) Unix
2 69 (6.73%) Linux
3 28 (2.73%) CentOS
3 28 (2.73%) Debian
3 28 (2.73%) Red Hat Enterprise Linux
3 28 (2.73%) Ubuntu
Processes & Methodologies
1 729 (71.12%) Software Engineering
2 724 (70.63%) Agile
3 621 (60.59%) CI/CD
4 560 (54.63%) SDLC
5 498 (48.59%) Operational Stability
6 392 (38.24%) Machine Learning
7 329 (32.10%) Artificial Intelligence
8 301 (29.37%) Risk Management
9 271 (26.44%) Visualisation
10 264 (25.76%) Community of Practice
11 233 (22.73%) Performance Tuning
12 215 (20.98%) Continuous Improvement
13 213 (20.78%) Problem-Solving
14 200 (19.51%) Microservices
15 182 (17.76%) Decision-Making
16 176 (17.17%) Data Engineering
17 163 (15.90%) Data Modelling
18 153 (14.93%) Data Pipeline
19 138 (13.46%) Analytics
19 138 (13.46%) Open Source
Programming Languages
1 621 (60.59%) SQL
2 615 (60.00%) Java
3 589 (57.46%) Python
4 402 (39.22%) Scala
5 105 (10.24%) PL/SQL
6 47 (4.59%) Go
7 38 (3.71%) Bash
8 34 (3.32%) JavaScript
9 30 (2.93%) Kotlin
10 27 (2.63%) R
11 25 (2.44%) C
12 23 (2.24%) C#
13 21 (2.05%) Shell Script
14 18 (1.76%) C++
15 12 (1.17%) TypeScript
16 8 (0.78%) Clojure
16 8 (0.78%) Data Analysis Expressions
16 8 (0.78%) MATLAB
17 6 (0.59%) Rust
18 5 (0.49%) VB
Qualifications
1 127 (12.39%) Degree
2 58 (5.66%) Security Cleared
3 51 (4.98%) DV Cleared
4 37 (3.61%) SC Cleared
5 34 (3.32%) Computer Science Degree
6 20 (1.95%) Master's Degree
7 11 (1.07%) Microsoft Certification
8 10 (0.98%) PhD
9 6 (0.59%) Azure Certification
9 6 (0.59%) BPSS Clearance
9 6 (0.59%) DBS Check
10 4 (0.39%) CIMA
10 4 (0.39%) Numerate Degree
11 3 (0.29%) ISTQB
12 1 (0.098%) AWS Certification
12 1 (0.098%) AWS Certified Solutions Architect
Quality Assurance & Compliance
1 71 (6.93%) Data Quality
2 37 (3.61%) SLA
3 21 (2.05%) QA
4 6 (0.59%) CRD IV
4 6 (0.59%) IFRS
4 6 (0.59%) IFRS 9
5 4 (0.39%) Accessibility
5 4 (0.39%) Solvency II
6 2 (0.20%) GDPR
System Software
1 116 (11.32%) HDFS
2 64 (6.24%) Docker
3 17 (1.66%) Virtual Machines
4 6 (0.59%) Ceph
5 3 (0.29%) Virtual Servers
6 2 (0.20%) Apache Flume
6 2 (0.20%) IAG
7 1 (0.098%) Slurm Workload Manager
Systems Management
1 206 (20.10%) Kubernetes
2 188 (18.34%) Terraform
3 65 (6.34%) Ansible
4 32 (3.12%) Puppet
5 18 (1.76%) Prometheus
6 17 (1.66%) Grafana
6 17 (1.66%) HAProxy
7 6 (0.59%) Knative
8 4 (0.39%) Kibana
9 3 (0.29%) Progress Chef
10 2 (0.20%) Argo
10 2 (0.20%) Kubeflow
11 1 (0.098%) logstash
11 1 (0.098%) Podman
Vendors
1 262 (25.56%) Databricks
2 114 (11.12%) Google
3 91 (8.88%) Informatica
4 90 (8.78%) Cloudera
5 63 (6.15%) Microsoft
6 34 (3.32%) Oracle
7 28 (2.73%) Red Hat
7 28 (2.73%) SaltStack
8 20 (1.95%) Cognos
8 20 (1.95%) Dell
9 19 (1.85%) Alteryx
10 18 (1.76%) Sybase
11 16 (1.56%) Teradata
12 8 (0.78%) YouTube
13 7 (0.68%) Confluent
14 5 (0.49%) Atlassian
15 4 (0.39%) Apple
15 4 (0.39%) SAP
15 4 (0.39%) Splunk
16 3 (0.29%) Facebook