Apache Spark Jobs

Apache Spark
UK

The table below provides summary statistics for permanent job vacancies requiring Apache Spark skills. It includes a benchmarking guide to the annual salaries offered in vacancies that cited Apache Spark over the 6 months leading up to 26 July 2024, comparing them to the same period in the previous two years.

6 months to
26 Jul 2024
Same period 2023 Same period 2022
Rank 358 375 264
Rank change year-on-year +17 -111 -17
Permanent jobs citing Apache Spark 649 690 2,799
As % of all permanent jobs advertised in the UK 0.72% 0.86% 1.60%
As % of the Application Platforms category 7.45% 7.83% 11.46%
Number of salaries quoted 205 512 930
10th Percentile £46,250 £60,000 £50,000
25th Percentile £56,250 £67,500 £57,500
Median annual salary (50th Percentile) £80,000 £85,000 £77,500
Median % change year-on-year -5.88% +9.68% +3.33%
75th Percentile £98,750 £105,000 £92,500
90th Percentile £118,750 £150,000 £108,750
UK excluding London median annual salary £62,500 £73,250 £60,000
% change year-on-year -14.68% +22.08% -5.88%

All Application Platform Skills
UK

Apache Spark falls under the Application Platforms category. For comparison with the information above, the following table provides summary statistics for all permanent job vacancies requiring application platform skills.

Permanent vacancies with a requirement for application platform skills 8,710 8,813 24,432
As % of all permanent jobs advertised in the UK 9.64% 10.92% 13.94%
Number of salaries quoted 5,787 6,758 11,737
10th Percentile £29,000 £31,250 £30,000
25th Percentile £36,250 £40,000 £39,250
Median annual salary (50th Percentile) £52,500 £55,834 £55,000
Median % change year-on-year -5.97% +1.52% +4.76%
75th Percentile £77,500 £76,250 £75,000
90th Percentile £117,500 £95,000 £92,500
UK excluding London median annual salary £45,000 £48,050 £47,500
% change year-on-year -6.35% +1.16% +5.56%

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a proportion of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Salary Trend

3-month moving average salary quoted in jobs citing Apache Spark.

Salary trend for Apache Spark in the UK

Apache Spark
Salary Histogram

Salary distribution for jobs citing Apache Spark over the 6 months to 26 July 2024.

Salary histogram for Apache Spark in the UK

Apache Spark
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the UK over the 6 months to 26 July 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
UK excluding London +185 481 £62,500 -14.68% 25
Scotland +146 336 £47,500 -24.00% 2
England -167 302 £80,000 -5.88% 22
London -112 154 £92,500 - 8
Work from Home -125 109 £73,000 -18.89% 25
North of England +6 51 £55,000 -26.67% 7
South West -5 50 £80,000 +6.67% 2
North West -8 39 £55,000 -26.67% 4
South East -23 17 £55,000 -24.14% 4
Midlands -19 16 £72,750 +3.93% 2
East Midlands -21 10 £55,000 -12.00% 1
Yorkshire +53 9 £49,637 -34.40% 2
East of England +1 8 £110,000 +4.76% 2
West Midlands -8 6 £73,000 +4.29% 1
North East -9 3 £49,274 -10.41% 1
Wales - 2 £66,199 - 3
Northern Ireland +20 1 £70,000 - 4

Apache Spark
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 207 (31.90%) Apache
2 128 (19.72%) Apache Airflow
3 18 (2.77%) Jupyter
4 16 (2.47%) nginx
5 11 (1.69%) Confluence
6 5 (0.77%) Apache Pulsar
7 2 (0.31%) Apache Pig
7 2 (0.31%) SAS
8 1 (0.15%) Luigi
Applications
1 5 (0.77%) Microsoft Excel
2 2 (0.31%) Microsoft Office
Business Applications
1 2 (0.31%) Primavera
Cloud Services
1 441 (67.95%) AWS
2 168 (25.89%) Azure
3 129 (19.88%) GCP
4 75 (11.56%) Serverless
5 72 (11.09%) AWS Lambda
6 64 (9.86%) Amazon EKS
7 48 (7.40%) Amazon S3
8 47 (7.24%) Amazon EMR
9 46 (7.09%) OpenShift
10 43 (6.63%) Snowflake
11 42 (6.47%) AWS CloudFormation
12 39 (6.01%) Amazon SQS
13 37 (5.70%) Power Platform
14 35 (5.39%) Amazon EC2
15 34 (5.24%) Amazon ECS
16 33 (5.08%) Azure Synapse Analytics
17 32 (4.93%) Azure Data Factory
18 30 (4.62%) Amazon Kinesis
19 28 (4.31%) GitHub
20 27 (4.16%) Azure DevOps
Communications & Networking
1 29 (4.47%) AMQP
2 6 (0.92%) HTTP
3 3 (0.46%) Wireless
4 2 (0.31%) Network Security
Database & Business Intelligence
1 450 (69.34%) Big Data
2 263 (40.52%) NoSQL
3 236 (36.36%) Hadoop
4 99 (15.25%) Apache Hive
5 64 (9.86%) HBase
6 63 (9.71%) Data Warehouse
7 62 (9.55%) Data Lake
8 35 (5.39%) Power BI
9 34 (5.24%) Amazon Redshift
9 34 (5.24%) Electronic Patient Records
10 33 (5.08%) Data Mining
11 30 (4.62%) BigQuery
12 29 (4.47%) Cloudera CDH
12 29 (4.47%) Presto
13 28 (4.31%) Flink
14 22 (3.39%) PostgreSQL
15 19 (2.93%) DynamoDB
16 18 (2.77%) Impala
17 17 (2.62%) Elasticsearch
17 17 (2.62%) Spark SQL
Development Applications
1 108 (16.64%) Jenkins
2 93 (14.33%) Git
3 60 (9.24%) Bitbucket Server
4 16 (2.47%) Atlassian Bamboo
5 11 (1.69%) JIRA
6 8 (1.23%) MLflow
6 8 (1.23%) RStudio
7 7 (1.08%) Spinnaker
8 6 (0.92%) Gatling
9 4 (0.62%) Bitbucket
10 3 (0.46%) Octopus Deploy
11 2 (0.31%) CircleCI
11 2 (0.31%) Deequ
11 2 (0.31%) Flyway
11 2 (0.31%) GitLab
11 2 (0.31%) Liquibase
11 2 (0.31%) Oracle Forms
11 2 (0.31%) Team Foundation Server
12 1 (0.15%) ERwin
12 1 (0.15%) JMeter
General
1 389 (59.94%) Finance
2 353 (54.39%) Inclusion and Diversity
3 337 (51.93%) Law
4 173 (26.66%) Legal
5 165 (25.42%) Marketing
6 93 (14.33%) Influencing Skills
7 88 (13.56%) Social Skills
8 44 (6.78%) Analytical Skills
9 33 (5.08%) Financial Institution
10 29 (4.47%) Aerospace
11 18 (2.77%) Retail
12 14 (2.16%) Electronics
13 13 (2.00%) Banking
13 13 (2.00%) Games
14 7 (1.08%) Presentation Skills
15 6 (0.92%) Advertising
16 4 (0.62%) Music Industry
16 4 (0.62%) Public Sector
17 3 (0.46%) Billing
17 3 (0.46%) Manufacturing
Job Titles
1 376 (57.94%) Software Engineer
2 259 (39.91%) Big Data Engineer
3 249 (38.37%) Data Engineer
4 242 (37.29%) Big Data Software Engineer
5 140 (21.57%) Lead
6 136 (20.96%) Senior
7 126 (19.41%) AWS Engineer
7 126 (19.41%) Scala Engineer
7 126 (19.41%) Scala Software Engineer
8 116 (17.87%) AWS Software Engineer
9 112 (17.26%) Lead Software Engineer
10 95 (14.64%) Lead Big Data Engineer
11 88 (13.56%) Python Engineer
12 85 (13.10%) Java Engineer
13 84 (12.94%) Java Software Engineer
14 82 (12.63%) Python Software Engineer
15 65 (10.02%) Senior Software Engineer
16 62 (9.55%) Lead Java Engineer
17 61 (9.40%) Java Scala Software Engineer
18 46 (7.09%) Lead Data Engineer
Libraries, Frameworks & Software Standards
1 142 (21.88%) Kafka
2 128 (19.72%) PySpark
3 56 (8.63%) Spring
3 56 (8.63%) Spring Boot
4 52 (8.01%) RabbitMQ
5 44 (6.78%) React
6 38 (5.86%) REST
7 32 (4.93%) Node.js
8 30 (4.62%) OpenAPI
8 30 (4.62%) Swagger
9 29 (4.47%) AngularJS
9 29 (4.47%) gRPC
9 29 (4.47%) Material-UI
10 22 (3.39%) Pandas
11 19 (2.93%) ActiveMQ
11 19 (2.93%) Apache Beam
11 19 (2.93%) Apache NiFi
12 16 (2.47%) Elastic Stack
12 16 (2.47%) Lambda Expressions
12 16 (2.47%) YAML
Miscellaneous
1 144 (22.19%) Algorithms
2 141 (21.73%) Cloud Native
3 31 (4.78%) Data Structures
4 27 (4.16%) Distributed Systems
5 21 (3.24%) Legacy Code
6 18 (2.77%) Hybrid Cloud
7 16 (2.47%) Public Cloud
8 13 (2.00%) Legacy Systems
9 12 (1.85%) Management Information System
10 7 (1.08%) Greenfield Project
11 6 (0.92%) Blog
11 6 (0.92%) Distributed Computing
11 6 (0.92%) Media Streaming
12 5 (0.77%) Product Ownership
13 4 (0.62%) Renewable Energy
13 4 (0.62%) Virtual Team
14 3 (0.46%) Analytical Mindset
14 3 (0.46%) Distributed Applications
14 3 (0.46%) IoT
14 3 (0.46%) Onboarding
Operating Systems
1 64 (9.86%) Linux
2 63 (9.71%) Unix
3 29 (4.47%) CentOS
3 29 (4.47%) Debian
3 29 (4.47%) Red Hat Enterprise Linux
3 29 (4.47%) Ubuntu
Processes & Methodologies
1 439 (67.64%) Agile
2 432 (66.56%) Software Engineering
3 405 (62.40%) CI/CD
4 322 (49.61%) SDLC
5 271 (41.76%) Operational Stability
6 237 (36.52%) Machine Learning
7 180 (27.73%) Artificial Intelligence
8 176 (27.12%) Risk Management
9 165 (25.42%) Visualisation
10 153 (23.57%) Community of Practice
11 128 (19.72%) Microservices
12 123 (18.95%) Performance Tuning
13 120 (18.49%) Continuous Improvement
14 118 (18.18%) Open Source
15 114 (17.57%) Data Engineering
15 114 (17.57%) Problem-Solving
16 100 (15.41%) Data Pipeline
17 99 (15.25%) ETL
18 96 (14.79%) Analytics
19 94 (14.48%) Mentoring
Programming Languages
1 397 (61.17%) Python
2 375 (57.78%) SQL
3 329 (50.69%) Java
4 237 (36.52%) Scala
5 63 (9.71%) PL/SQL
6 39 (6.01%) JavaScript
7 34 (5.24%) Kotlin
8 29 (4.47%) Bash
9 21 (3.24%) R
10 20 (3.08%) Shell Script
11 19 (2.93%) Go
12 18 (2.77%) C++
13 15 (2.31%) TypeScript
14 14 (2.16%) C
15 12 (1.85%) C#
16 10 (1.54%) Clojure
17 6 (0.92%) Rust
18 5 (0.77%) Data Analysis Expressions
19 4 (0.62%) T-SQL
20 2 (0.31%) VB
Qualifications
1 76 (11.71%) Degree
2 65 (10.02%) Security Cleared
3 48 (7.40%) DV Cleared
4 45 (6.93%) SC Cleared
5 34 (5.24%) Computer Science Degree
6 21 (3.24%) Master's Degree
7 11 (1.69%) PhD
8 6 (0.92%) DBS Check
9 4 (0.62%) Azure Certification
9 4 (0.62%) BPSS Clearance
10 3 (0.46%) Microsoft Certification
11 2 (0.31%) CIMA
11 2 (0.31%) ISTQB
12 1 (0.15%) AWS Certification
12 1 (0.15%) AWS Certified Solutions Architect
Quality Assurance & Compliance
1 66 (10.17%) Data Quality
2 14 (2.16%) QA
2 14 (2.16%) SLA
3 4 (0.62%) Accessibility
4 2 (0.31%) GDPR
5 1 (0.15%) ISO 20022
System Software
1 68 (10.48%) HDFS
2 42 (6.47%) Docker
3 16 (2.47%) Virtual Machines
4 6 (0.92%) Ceph
5 3 (0.46%) Virtual Servers
6 2 (0.31%) IAG
7 1 (0.15%) Apache Flume
Systems Management
1 169 (26.04%) Terraform
2 139 (21.42%) Kubernetes
3 64 (9.86%) Ansible
4 32 (4.93%) Puppet
5 16 (2.47%) Grafana
5 16 (2.47%) HAProxy
5 16 (2.47%) Prometheus
6 6 (0.92%) Knative
7 3 (0.46%) Progress Chef
8 2 (0.31%) Argo
8 2 (0.31%) Kubeflow
9 1 (0.15%) Apache Ranger
9 1 (0.15%) Hadoop YARN
9 1 (0.15%) Kibana
Vendors
1 184 (28.35%) Databricks
2 77 (11.86%) Google
3 73 (11.25%) Informatica
4 49 (7.55%) Microsoft
5 40 (6.16%) Cloudera
6 29 (4.47%) Red Hat
6 29 (4.47%) SaltStack
7 9 (1.39%) Oracle
8 6 (0.92%) Apple
8 6 (0.92%) Splunk
9 5 (0.77%) Teradata
10 4 (0.62%) Dell
11 3 (0.46%) Alteryx
11 3 (0.46%) Atlassian
11 3 (0.46%) Foundry
11 3 (0.46%) Hortonworks
11 3 (0.46%) Palantir
11 3 (0.46%) SAP
11 3 (0.46%) SnapLogic
11 3 (0.46%) StreamSets