Apache Spark Jobs

Apache Spark
UK

The table below provides summary statistics for permanent job vacancies requiring Apache Spark skills. It includes a benchmarking guide to the annual salaries offered in vacancies that cited Apache Spark over the 6 months leading up to 23 June 2024, comparing them to the same period in the previous two years.

6 months to
23 Jun 2024
Same period 2023 Same period 2022
Rank 320 301 248
Rank change year-on-year -19 -53 +7
Permanent jobs citing Apache Spark 901 1,099 2,841
As % of all permanent jobs advertised in the UK 0.82% 1.19% 1.64%
As % of the Application Platforms category 8.45% 10.67% 11.76%
Number of salaries quoted 275 563 907
10th Percentile £47,500 £60,000 £50,000
25th Percentile £63,750 £67,500 £57,500
Median annual salary (50th Percentile) £80,000 £87,500 £75,000
Median % change year-on-year -8.57% +16.67% -
75th Percentile £102,500 £110,000 £92,500
90th Percentile £118,750 £150,000 £108,750
UK excluding London median annual salary £69,447 £70,000 £62,500
% change year-on-year -0.79% +12.00% -

All Application Platform Skills
UK

Apache Spark falls under the Application Platforms category. For comparison with the information above, the following table provides summary statistics for all permanent job vacancies requiring application platform skills.

Permanent vacancies with a requirement for application platform skills 10,667 10,303 24,168
As % of all permanent jobs advertised in the UK 9.71% 11.14% 13.99%
Number of salaries quoted 7,124 7,261 11,556
10th Percentile £27,500 £31,250 £29,000
25th Percentile £35,000 £40,000 £38,750
Median annual salary (50th Percentile) £50,000 £55,043 £55,000
Median % change year-on-year -9.16% +0.08% +4.76%
75th Percentile £70,500 £77,500 £73,750
90th Percentile £100,000 £95,000 £92,500
UK excluding London median annual salary £42,500 £48,759 £45,500
% change year-on-year -12.84% +7.16% +1.11%

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a proportion of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Salary Trend

3-month moving average salary quoted in jobs citing Apache Spark.

Salary trend for Apache Spark in the UK

Apache Spark
Salary Histogram

Salary distribution for jobs citing Apache Spark over the 6 months to 23 June 2024.

Salary histogram for Apache Spark in the UK

Apache Spark
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the UK over the 6 months to 23 June 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
UK excluding London +188 642 £69,447 -0.79% 27
Scotland +154 507 £47,500 -20.83% 1
England -228 375 £80,000 -8.57% 88
London -143 237 £90,000 -5.26% 66
Work from Home -164 162 £85,000 -15.00% 150
South West -40 51 £80,000 +6.67% 5
North of England -63 44 £55,000 -26.67% 7
North West -23 39 £55,000 -26.67% 4
Midlands -31 16 £72,750 +3.93% 7
South East -54 15 £55,000 -21.43% 5
East Midlands -20 10 £60,000 - 2
West Midlands -13 6 £73,000 +4.29% 5
East of England -12 5 £101,250 +102.50% 2
Yorkshire -12 4 £55,000 -23.52% 3
Wales - 3 £72,500 - 2
Northern Ireland +19 1 £70,000 -30.00%
North East -17 1 £75,000 +36.36%

Apache Spark
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 268 (29.74%) Apache
2 159 (17.65%) Apache Airflow
3 26 (2.89%) Jupyter
4 16 (1.78%) nginx
5 10 (1.11%) Confluence
6 5 (0.55%) Apache Pulsar
7 2 (0.22%) Luigi
7 2 (0.22%) SAS
8 1 (0.11%) Apache Pig
Applications
1 7 (0.78%) Microsoft Excel
2 4 (0.44%) Spreadsheet
3 1 (0.11%) Microsoft Office
Business Applications
1 4 (0.44%) Primavera
Cloud Services
1 614 (68.15%) AWS
2 212 (23.53%) Azure
3 166 (18.42%) GCP
4 94 (10.43%) Serverless
5 86 (9.54%) AWS Lambda
6 84 (9.32%) Amazon EKS
7 62 (6.88%) Amazon EMR
8 60 (6.66%) Amazon SQS
9 55 (6.10%) Amazon S3
10 48 (5.33%) Amazon ECS
10 48 (5.33%) Snowflake
11 47 (5.22%) Amazon EC2
11 47 (5.22%) OpenShift
12 44 (4.88%) AWS CloudFormation
13 43 (4.77%) Power Platform
14 41 (4.55%) Azure Data Factory
15 39 (4.33%) Azure Synapse Analytics
16 27 (3.00%) Amazon Kinesis
16 27 (3.00%) Azure DevOps
17 26 (2.89%) GitHub
Communications & Networking
1 29 (3.22%) AMQP
2 12 (1.33%) Wireless
3 10 (1.11%) DSL
4 5 (0.55%) Network Security
5 3 (0.33%) Firewall
6 2 (0.22%) HTTP
Database & Business Intelligence
1 621 (68.92%) Big Data
2 366 (40.62%) NoSQL
3 359 (39.84%) Hadoop
4 135 (14.98%) Apache Hive
5 92 (10.21%) Data Lake
6 90 (9.99%) HBase
7 82 (9.10%) Data Warehouse
8 47 (5.22%) Data Mining
8 47 (5.22%) Electronic Patient Records
9 43 (4.77%) Amazon Redshift
10 42 (4.66%) SQL Server
11 41 (4.55%) Power BI
12 39 (4.33%) Flink
13 38 (4.22%) Elasticsearch
14 37 (4.11%) BigQuery
15 31 (3.44%) Tableau
16 30 (3.33%) Presto
17 29 (3.22%) Cloudera CDH
18 24 (2.66%) PostgreSQL
19 22 (2.44%) Relational Database
Development Applications
1 130 (14.43%) Jenkins
2 121 (13.43%) Git
3 79 (8.77%) Bitbucket Server
4 16 (1.78%) Atlassian Bamboo
5 11 (1.22%) JIRA
5 11 (1.22%) Spinnaker
6 8 (0.89%) Visual Studio
7 7 (0.78%) Oracle Forms
8 6 (0.67%) MLflow
9 5 (0.55%) CircleCI
10 4 (0.44%) Bitbucket
11 3 (0.33%) Deequ
11 3 (0.33%) Octopus Deploy
12 2 (0.22%) Gatling
12 2 (0.22%) Team Foundation Server
13 1 (0.11%) ERwin
13 1 (0.11%) GitLab
13 1 (0.11%) Subversion
General
1 550 (61.04%) Finance
2 523 (58.05%) Inclusion and Diversity
3 498 (55.27%) Law
4 271 (30.08%) Marketing
5 242 (26.86%) Legal
6 140 (15.54%) Influencing Skills
7 109 (12.10%) Social Skills
8 77 (8.55%) Analytical Skills
9 57 (6.33%) Financial Institution
10 38 (4.22%) Banking
11 29 (3.22%) Aerospace
12 18 (2.00%) Games
12 18 (2.00%) Retail
13 16 (1.78%) Electronics
13 16 (1.78%) Presentation Skills
14 14 (1.55%) Investment Banking
15 12 (1.33%) Public Sector
16 5 (0.55%) Private Banking
16 5 (0.55%) Retail Banking
17 4 (0.44%) Music Industry
Job Titles
1 517 (57.38%) Software Engineer
2 347 (38.51%) Big Data Engineer
3 330 (36.63%) Big Data Software Engineer
4 309 (34.30%) Data Engineer
5 203 (22.53%) Lead
5 203 (22.53%) Senior
6 184 (20.42%) Scala Engineer
6 184 (20.42%) Scala Software Engineer
7 163 (18.09%) AWS Engineer
7 163 (18.09%) Lead Software Engineer
8 151 (16.76%) AWS Software Engineer
9 148 (16.43%) Java Engineer
10 147 (16.32%) Java Software Engineer
11 130 (14.43%) Lead Big Data Engineer
12 124 (13.76%) Python Engineer
13 104 (11.54%) Python Software Engineer
14 99 (10.99%) Java Scala Software Engineer
15 91 (10.10%) Lead Java Engineer
15 91 (10.10%) Senior Software Engineer
16 76 (8.44%) Developer
Libraries, Frameworks & Software Standards
1 194 (21.53%) Kafka
2 165 (18.31%) PySpark
3 87 (9.66%) Spring
4 75 (8.32%) Spring Boot
5 55 (6.10%) React
6 52 (5.77%) RabbitMQ
7 41 (4.55%) REST
8 39 (4.33%) AngularJS
9 31 (3.44%) Node.js
10 30 (3.33%) Apache Beam
10 30 (3.33%) gRPC
10 30 (3.33%) OpenAPI
10 30 (3.33%) Swagger
11 29 (3.22%) Material-UI
12 21 (2.33%) Apache NiFi
12 21 (2.33%) Pandas
13 20 (2.22%) ActiveMQ
14 18 (2.00%) Elastic Stack
14 18 (2.00%) TensorFlow
15 16 (1.78%) YAML
Miscellaneous
1 213 (23.64%) Algorithms
2 198 (21.98%) Cloud Native
3 50 (5.55%) Data Structures
4 34 (3.77%) Legacy Code
5 27 (3.00%) Distributed Systems
5 27 (3.00%) Management Information System
6 25 (2.77%) Public Cloud
7 20 (2.22%) Product Ownership
8 19 (2.11%) Hybrid Cloud
9 13 (1.44%) Greenfield Project
10 10 (1.11%) Legacy Systems
11 8 (0.89%) BBC
12 7 (0.78%) Distributed Applications
12 7 (0.78%) Distributed Computing
13 6 (0.67%) Mainframe
14 5 (0.55%) Enterprise Software
14 5 (0.55%) IoT
15 4 (0.44%) Renewable Energy
16 3 (0.33%) Analytical Mindset
16 3 (0.33%) Onboarding
Operating Systems
1 89 (9.88%) Unix
2 67 (7.44%) Linux
3 29 (3.22%) CentOS
3 29 (3.22%) Debian
3 29 (3.22%) Red Hat Enterprise Linux
3 29 (3.22%) Ubuntu
Processes & Methodologies
1 625 (69.37%) Agile
2 618 (68.59%) Software Engineering
3 547 (60.71%) CI/CD
4 470 (52.16%) SDLC
5 413 (45.84%) Operational Stability
6 331 (36.74%) Machine Learning
7 272 (30.19%) Artificial Intelligence
8 250 (27.75%) Risk Management
9 236 (26.19%) Visualisation
10 225 (24.97%) Community of Practice
11 191 (21.20%) Performance Tuning
12 181 (20.09%) Microservices
13 174 (19.31%) Continuous Improvement
14 173 (19.20%) Problem-Solving
15 166 (18.42%) Data Engineering
16 148 (16.43%) Data Pipeline
17 137 (15.21%) Decision-Making
17 137 (15.21%) Open Source
18 133 (14.76%) Data Modelling
19 128 (14.21%) ETL
Programming Languages
1 538 (59.71%) SQL
2 529 (58.71%) Python
3 505 (56.05%) Java
4 338 (37.51%) Scala
5 90 (9.99%) PL/SQL
6 38 (4.22%) Go
6 38 (4.22%) JavaScript
7 35 (3.88%) Kotlin
8 29 (3.22%) Bash
9 24 (2.66%) C
10 21 (2.33%) C#
11 19 (2.11%) R
11 19 (2.11%) Shell Script
12 17 (1.89%) C++
13 14 (1.55%) TypeScript
14 10 (1.11%) Clojure
15 8 (0.89%) Data Analysis Expressions
15 8 (0.89%) MATLAB
16 6 (0.67%) Rust
17 3 (0.33%) VB
Qualifications
1 110 (12.21%) Degree
2 57 (6.33%) Security Cleared
3 51 (5.66%) DV Cleared
4 37 (4.11%) SC Cleared
5 33 (3.66%) Computer Science Degree
6 21 (2.33%) Master's Degree
7 11 (1.22%) Microsoft Certification
7 11 (1.22%) PhD
8 6 (0.67%) Azure Certification
8 6 (0.67%) BPSS Clearance
8 6 (0.67%) DBS Check
9 4 (0.44%) CIMA
9 4 (0.44%) Numerate Degree
10 3 (0.33%) ISTQB
11 1 (0.11%) AWS Certification
11 1 (0.11%) AWS Certified Solutions Architect
Quality Assurance & Compliance
1 66 (7.33%) Data Quality
2 24 (2.66%) SLA
3 19 (2.11%) QA
4 4 (0.44%) Accessibility
4 4 (0.44%) Solvency II
5 2 (0.22%) GDPR
System Software
1 99 (10.99%) HDFS
2 66 (7.33%) Docker
3 17 (1.89%) Virtual Machines
4 6 (0.67%) Ceph
5 3 (0.33%) Virtual Servers
6 2 (0.22%) IAG
7 1 (0.11%) Apache Flume
7 1 (0.11%) Slurm Workload Manager
Systems Management
1 191 (21.20%) Kubernetes
2 185 (20.53%) Terraform
3 66 (7.33%) Ansible
4 33 (3.66%) Puppet
5 17 (1.89%) Prometheus
6 16 (1.78%) Grafana
6 16 (1.78%) HAProxy
7 6 (0.67%) Knative
8 3 (0.33%) Kibana
8 3 (0.33%) Progress Chef
9 2 (0.22%) Argo
9 2 (0.22%) Kubeflow
10 1 (0.11%) Apache Ranger
10 1 (0.11%) Hadoop YARN
10 1 (0.11%) Podman
Vendors
1 234 (25.97%) Databricks
2 106 (11.76%) Google
3 89 (9.88%) Informatica
4 69 (7.66%) Cloudera
5 63 (6.99%) Microsoft
6 29 (3.22%) Red Hat
6 29 (3.22%) SaltStack
7 25 (2.77%) Oracle
8 15 (1.66%) Teradata
9 12 (1.33%) Alteryx
9 12 (1.33%) Cognos
9 12 (1.33%) Dell
10 10 (1.11%) Sybase
11 6 (0.67%) YouTube
12 5 (0.55%) SAP
13 4 (0.44%) Apple
13 4 (0.44%) Hortonworks
13 4 (0.44%) Splunk
14 3 (0.33%) Facebook
14 3 (0.33%) SnapLogic