Apache Spark
UK

The following table provides summary statistics for permanent job vacancies with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 23 April 2024 with a comparison to the same period in the previous 2 years.

6 months to
23 Apr 2024
Same period 2023 Same period 2022
Rank 277 249 254
Rank change year-on-year -28 +5 +10
Permanent jobs citing Apache Spark 1,030 1,531 2,418
As % of all permanent jobs advertised in the UK 1.05% 1.45% 1.57%
As % of the Application Platforms category 9.64% 12.50% 11.41%
Number of salaries quoted 284 621 889
10th Percentile £50,000 £57,500 £50,000
25th Percentile £61,180 £66,250 £58,750
Median annual salary (50th Percentile) £80,000 £87,500 £80,000
Median % change year-on-year -8.57% +9.38% +6.67%
75th Percentile £105,625 £110,000 £93,750
90th Percentile £118,750 £150,000 £112,500
UK excluding London median annual salary £69,447 £68,823 £62,500
% change year-on-year +0.91% +10.12% +0.46%

All Application Platform Skills
UK

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for application platform skills.

Permanent vacancies with a requirement for application platform skills 10,684 12,250 21,197
As % of all permanent jobs advertised in the UK 10.93% 11.59% 13.77%
Number of salaries quoted 7,089 8,043 11,814
10th Percentile £27,500 £31,250 £28,500
25th Percentile £34,000 £40,000 £37,500
Median annual salary (50th Percentile) £50,000 £55,000 £54,192
Median % change year-on-year -9.09% +1.49% +3.22%
75th Percentile £67,500 £77,500 £73,750
90th Percentile £91,250 £97,182 £92,125
UK excluding London median annual salary £42,500 £47,500 £45,000
% change year-on-year -10.53% +5.56% -

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a proportion of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Salary Trend

3-month moving average salary quoted in jobs citing Apache Spark.

Salary trend for Apache Spark in the UK

Apache Spark
Salary Histogram

Salary distribution for jobs citing Apache Spark over the 6 months to 23 April 2024.

Salary histogram for Apache Spark in the UK

Apache Spark
Top 16 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the UK over the 6 months to 23 April 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
UK excluding London +98 751 £69,447 +0.91% 51
Scotland +152 608 £47,500 -20.83% 10
England -242 399 £80,000 -10.61% 123
London -133 258 £90,000 -5.26% 80
Work from Home -191 199 £80,000 -20.00% 108
South West -19 54 £80,000 +6.67% 6
North of England -113 51 £55,000 -20.08% 11
North West -4 39 £62,500 -12.28% 5
South East -54 14 £49,000 -26.04% 7
Yorkshire -53 12 £51,528 -20.73% 3
Midlands -2 10 £72,750 +3.93% 6
Wales +4 7 £51,528 -1.85%
East of England -8 6 £101,250 +102.50% 9
East Midlands -20 6 £60,000 - 3
West Midlands +17 4 £73,000 +4.29% 3
Northern Ireland +21 1 £70,000 -30.00% 3

Apache Spark
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 303 (29.42%) Apache
2 163 (15.83%) Apache Airflow
3 28 (2.72%) Jupyter
4 18 (1.75%) nginx
5 16 (1.55%) SAS
6 6 (0.58%) Confluence
7 2 (0.19%) Apache Solr
7 2 (0.19%) Luigi
8 1 (0.097%) Apache Pig
Applications
1 7 (0.68%) Microsoft Excel
2 4 (0.39%) Spreadsheet
Business Applications
1 4 (0.39%) Primavera
Cloud Services
1 696 (67.57%) AWS
2 218 (21.17%) Azure
3 194 (18.83%) GCP
4 99 (9.61%) Serverless
5 93 (9.03%) Amazon EKS
6 92 (8.93%) AWS Lambda
7 80 (7.77%) Amazon SQS
8 71 (6.89%) Amazon EMR
8 71 (6.89%) Amazon S3
9 57 (5.53%) Amazon ECS
10 55 (5.34%) Amazon EC2
11 53 (5.15%) Power Platform
12 46 (4.47%) Snowflake
13 44 (4.27%) OpenShift
14 43 (4.17%) GitHub
15 37 (3.59%) AWS CloudFormation
16 36 (3.50%) Azure Synapse Analytics
17 35 (3.40%) Azure Data Factory
18 33 (3.20%) SaaS
19 26 (2.52%) Amazon Kinesis
Communications & Networking
1 24 (2.33%) AMQP
1 24 (2.33%) Wireless
2 18 (1.75%) DSL
3 9 (0.87%) Firewall
4 5 (0.49%) Network Security
5 2 (0.19%) HTTP
Database & Business Intelligence
1 720 (69.90%) Big Data
2 432 (41.94%) Hadoop
3 425 (41.26%) NoSQL
4 165 (16.02%) Apache Hive
5 128 (12.43%) Data Lake
6 103 (10.00%) HBase
7 89 (8.64%) Data Warehouse
8 66 (6.41%) Elasticsearch
9 55 (5.34%) Data Mining
10 51 (4.95%) Electronic Patient Records
10 51 (4.95%) Power BI
10 51 (4.95%) Tableau
11 43 (4.17%) Amazon Redshift
12 41 (3.98%) BigQuery
12 41 (3.98%) Flink
13 38 (3.69%) Relational Database
13 38 (3.69%) SQL Server
14 31 (3.01%) PostgreSQL
15 29 (2.82%) Apache Ignite
15 29 (2.82%) Presto
Development Applications
1 151 (14.66%) Jenkins
2 127 (12.33%) Git
3 84 (8.16%) Bitbucket Server
4 18 (1.75%) Atlassian Bamboo
5 15 (1.46%) Spinnaker
6 8 (0.78%) CircleCI
6 8 (0.78%) JIRA
6 8 (0.78%) Oracle Forms
6 8 (0.78%) Visual Studio
7 4 (0.39%) Bitbucket
7 4 (0.39%) MLflow
8 3 (0.29%) Deequ
8 3 (0.29%) Octopus Deploy
9 2 (0.19%) JUnit
9 2 (0.19%) Team Foundation Server
10 1 (0.097%) Ant
10 1 (0.097%) ERwin
10 1 (0.097%) Gradle
10 1 (0.097%) Maven
10 1 (0.097%) Postman
General
1 674 (65.44%) Finance
2 622 (60.39%) Inclusion and Diversity
3 580 (56.31%) Law
4 315 (30.58%) Marketing
5 271 (26.31%) Legal
6 168 (16.31%) Influencing Skills
7 129 (12.52%) Social Skills
8 121 (11.75%) Analytical Skills
9 92 (8.93%) Banking
10 70 (6.80%) Financial Institution
11 45 (4.37%) Retail
12 29 (2.82%) Presentation Skills
13 24 (2.33%) Aerospace
14 20 (1.94%) Investment Banking
15 19 (1.84%) Games
16 13 (1.26%) Private Banking
17 12 (1.17%) Public Sector
18 10 (0.97%) Electronics
19 9 (0.87%) Retail Finance
20 8 (0.78%) Retail Banking
Job Titles
1 573 (55.63%) Software Engineer
2 388 (37.67%) Big Data Engineer
3 369 (35.83%) Big Data Software Engineer
4 331 (32.14%) Data Engineer
5 235 (22.82%) Senior
6 226 (21.94%) Lead
7 212 (20.58%) Scala Engineer
7 212 (20.58%) Scala Software Engineer
8 189 (18.35%) Java Engineer
9 188 (18.25%) AWS Engineer
9 188 (18.25%) Java Software Engineer
10 178 (17.28%) Lead Software Engineer
11 176 (17.09%) AWS Software Engineer
12 144 (13.98%) Lead Big Data Engineer
13 134 (13.01%) Python Engineer
14 117 (11.36%) Java Scala Software Engineer
15 112 (10.87%) Python Software Engineer
16 96 (9.32%) Senior Software Engineer
17 95 (9.22%) Lead Java Engineer
18 89 (8.64%) Developer
Libraries, Frameworks & Software Standards
1 229 (22.23%) Kafka
2 167 (16.21%) PySpark
3 110 (10.68%) Spring
4 81 (7.86%) Spring Boot
5 57 (5.53%) React
6 49 (4.76%) AngularJS
7 44 (4.27%) REST
8 42 (4.08%) RabbitMQ
9 34 (3.30%) Apache Beam
10 29 (2.82%) Node.js
11 25 (2.43%) gRPC
12 24 (2.33%) Material-UI
12 24 (2.33%) OpenAPI
12 24 (2.33%) Pandas
12 24 (2.33%) Swagger
13 23 (2.23%) Apache NiFi
13 23 (2.23%) GraphQL
14 22 (2.14%) ActiveMQ
15 21 (2.04%) Elastic Stack
16 18 (1.75%) YAML
Miscellaneous
1 253 (24.56%) Algorithms
2 222 (21.55%) Cloud Native
3 61 (5.92%) Data Structures
4 42 (4.08%) Legacy Code
5 39 (3.79%) Management Information System
6 34 (3.30%) Public Cloud
7 30 (2.91%) Distributed Systems
8 28 (2.72%) Product Ownership
9 19 (1.84%) Hybrid Cloud
10 16 (1.55%) Greenfield Project
11 15 (1.46%) Credit Risk
11 15 (1.46%) Legacy Systems
12 11 (1.07%) Enterprise Software
13 9 (0.87%) Distributed Applications
14 8 (0.78%) BBC
14 8 (0.78%) Onboarding
15 6 (0.58%) Mainframe
16 4 (0.39%) Distributed Computing
16 4 (0.39%) IoT
17 3 (0.29%) Analytical Mindset
Operating Systems
1 102 (9.90%) Unix
2 59 (5.73%) Linux
3 24 (2.33%) CentOS
3 24 (2.33%) Debian
3 24 (2.33%) Red Hat Enterprise Linux
3 24 (2.33%) Ubuntu
Processes & Methodologies
1 743 (72.14%) Agile
2 696 (67.57%) Software Engineering
3 612 (59.42%) CI/CD
4 551 (53.50%) SDLC
5 487 (47.28%) Operational Stability
6 387 (37.57%) Machine Learning
7 331 (32.14%) Artificial Intelligence
8 301 (29.22%) Risk Management
9 293 (28.45%) Visualisation
10 258 (25.05%) Community of Practice
11 231 (22.43%) Performance Tuning
12 214 (20.78%) Problem-Solving
13 211 (20.49%) Continuous Improvement
14 186 (18.06%) Decision-Making
14 186 (18.06%) Microservices
15 183 (17.77%) Data Engineering
16 159 (15.44%) Data Modelling
17 153 (14.85%) Data Pipeline
18 152 (14.76%) Analytics
19 149 (14.47%) ETL
Programming Languages
1 611 (59.32%) SQL
2 594 (57.67%) Java
3 583 (56.60%) Python
4 397 (38.54%) Scala
5 103 (10.00%) PL/SQL
6 45 (4.37%) Go
7 43 (4.17%) Bash
8 41 (3.98%) R
9 29 (2.82%) JavaScript
10 24 (2.33%) C#
11 21 (2.04%) Shell Script
12 17 (1.65%) C
13 16 (1.55%) Kotlin
14 11 (1.07%) Data Analysis Expressions
15 10 (0.97%) C++
16 8 (0.78%) MATLAB
16 8 (0.78%) VB
17 7 (0.68%) TypeScript
18 3 (0.29%) Clojure
19 2 (0.19%) XPath
Qualifications
1 144 (13.98%) Degree
2 54 (5.24%) Security Cleared
3 47 (4.56%) DV Cleared
4 33 (3.20%) SC Cleared
5 31 (3.01%) Computer Science Degree
6 19 (1.84%) Master's Degree
7 11 (1.07%) Microsoft Certification
8 9 (0.87%) PhD
9 6 (0.58%) Azure Certification
9 6 (0.58%) BPSS Clearance
10 4 (0.39%) CIMA
10 4 (0.39%) Numerate Degree
11 3 (0.29%) ISTQB
12 2 (0.19%) DBS Check
13 1 (0.097%) AWS Certification
13 1 (0.097%) AWS Certified Solutions Architect
Quality Assurance & Compliance
1 63 (6.12%) Data Quality
2 37 (3.59%) SLA
3 17 (1.65%) QA
4 15 (1.46%) CRD IV
4 15 (1.46%) IFRS
4 15 (1.46%) IFRS 9
5 4 (0.39%) Solvency II
6 3 (0.29%) Accessibility
6 3 (0.29%) GDPR
7 1 (0.097%) HIPAA
System Software
1 114 (11.07%) HDFS
2 63 (6.12%) Docker
3 7 (0.68%) Virtual Machines
4 3 (0.29%) Apache Flume
5 2 (0.19%) IAG
6 1 (0.097%) Slurm Workload Manager
Systems Management
1 189 (18.35%) Kubernetes
2 183 (17.77%) Terraform
3 54 (5.24%) Ansible
4 28 (2.72%) Puppet
5 19 (1.84%) Prometheus
6 18 (1.75%) Grafana
6 18 (1.75%) HAProxy
7 4 (0.39%) Kibana
8 3 (0.29%) Progress Chef
9 2 (0.19%) Argo
9 2 (0.19%) Kubeflow
10 1 (0.097%) Control-M
10 1 (0.097%) logstash
10 1 (0.097%) Podman
10 1 (0.097%) Rancher
Vendors
1 254 (24.66%) Databricks
2 134 (13.01%) Google
3 90 (8.74%) Informatica
4 88 (8.54%) Cloudera
5 73 (7.09%) Microsoft
6 32 (3.11%) Oracle
7 24 (2.33%) Dell
7 24 (2.33%) Red Hat
7 24 (2.33%) SaltStack
8 20 (1.94%) Cognos
9 18 (1.75%) Alteryx
9 18 (1.75%) Sybase
10 16 (1.55%) Teradata
11 9 (0.87%) Confluent
12 8 (0.78%) Atlassian
12 8 (0.78%) YouTube
13 5 (0.49%) Apple
14 4 (0.39%) Facebook
14 4 (0.39%) SAP
14 4 (0.39%) Splunk