Period
to 27 October 2021

The following table provides summary statistics for permanent job vacancies with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 27 October 2021 with a comparison to the same period in the previous 2 years.

Apache Spark
UK
6 months to
27 Oct 2021
Same period 2020 Same period 2019
Rank 235 244 239
Rank change year-on-year +9 -5 +16
Permanent jobs citing Apache Spark 1,893 691 1,963
As % of all permanent jobs advertised in the UK 1.49% 1.40% 1.44%
As % of the Application Platforms category 10.86% 9.97% 9.81%
Number of salaries quoted 993 550 1,623
10th Percentile £47,500 £44,875 £47,500
25th Percentile £57,500 £52,500 £56,250
Median annual salary (50th Percentile) £75,000 £70,000 £72,500
Median % change year-on-year +7.14% -3.45% +3.57%
75th Percentile £95,000 £92,188 £91,250
90th Percentile £110,000 £105,000 £110,000
UK excluding London median annual salary £67,500 £55,000 £60,000
% change year-on-year +22.73% -8.33% -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for application platform skills.

All Application Platform Skills
UK
Permanent vacancies with a requirement for application platform skills 17,432 6,930 20,016
As % of all permanent IT jobs advertised in the UK 13.73% 13.99% 14.71%
Number of salaries quoted 10,837 5,761 16,141
10th Percentile £31,250 £31,250 £27,500
25th Percentile £39,000 £40,000 £36,250
Median annual salary (50th Percentile) £55,000 £55,000 £50,000
Median % change year-on-year - +10.00% -
75th Percentile £73,750 £72,500 £68,750
90th Percentile £90,000 £90,000 £85,000
UK excluding London median annual salary £46,500 £45,000 £43,000
% change year-on-year +3.33% +4.65% -1.98%

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a proportion of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Salary Trend

3-month moving average salary quoted in jobs citing Apache Spark.

Salary trend for Apache Spark in the UK

Apache Spark
Salary Histogram

Salary distribution for jobs citing Apache Spark over the 6 months to 27 October 2021.

Salary histogram for Apache Spark in the UK

Apache Spark
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the UK over the 6 months to 27 October 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +12 1,704 £80,000 +9.22% 169
London -22 1,089 £80,000 - 123
UK excluding London +18 685 £67,500 +22.73% 56
South East +37 342 £85,000 +54.55% 13
Work from Home -96 264 £82,500 +32.00% 40
North of England -82 79 £57,500 -4.17% 11
Scotland -52 75 £62,500 - 6
East of England -61 62 £80,000 +14.29% 4
Midlands -97 54 £62,500 -16.67% 10
Yorkshire -71 41 £50,000 -16.67% 1
West Midlands -63 36 £70,000 +24.44% 2
North West -86 35 £65,500 +11.49% 7
South West -79 32 £61,250 +21.29% 7
Northern Ireland - 23 £75,000 - 2
East Midlands -60 18 £53,000 -29.33% 8
Wales -76 18 £45,000 - 3
North East - 3 £56,914 - 3

For the 6 months to 27 October 2021, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Apache Spark.

1 1,307 (69.04%) Python
2 1,060 (56.00%) AWS
3 975 (51.51%) Java
4 957 (50.55%) Kafka
5 947 (50.03%) Big Data
6 885 (46.75%) Hadoop
7 860 (45.43%) SQL
8 784 (41.42%) Agile Software Development
9 779 (41.15%) Analytics
10 682 (36.03%) Software Engineering
11 661 (34.92%) DevOps
12 652 (34.44%) Azure
13 634 (33.49%) Finance
14 584 (30.85%) Data Science
15 566 (29.90%) NoSQL
16 563 (29.74%) GCP
17 552 (29.16%) Apache
18 506 (26.73%) Microservices
19 485 (25.62%) Data Pipeline
20 484 (25.57%) Degree
21 478 (25.25%) Scala
22 466 (24.62%) Kubernetes
23 464 (24.51%) Computer Science
24 453 (23.93%) Jenkins
25 452 (23.88%) Machine Learning
26 422 (22.29%) Docker
27 390 (20.60%) Data Engineering
27 390 (20.60%) PostgreSQL
28 386 (20.39%) ETL
29 377 (19.92%) Spring

Apache Spark
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 552 (29.16%) Apache
2 193 (10.20%) Apache Airflow
3 185 (9.77%) GemFire
4 173 (9.14%) Elasticsearch
5 57 (3.01%) Jupyter
6 51 (2.69%) Cloud Foundry
7 40 (2.11%) Apache Pig
8 38 (2.01%) Luigi
9 31 (1.64%) Confluence
10 23 (1.22%) Apache Solr
11 22 (1.16%) Tomcat
12 20 (1.06%) Oracle Application Server
13 17 (0.90%) SAS
14 10 (0.53%) Oracle SOA Suite
15 9 (0.48%) Apache Synapse
16 8 (0.42%) Oracle Workflow
17 4 (0.21%) Ethereum
17 4 (0.21%) SharePoint
18 3 (0.16%) Mule
18 3 (0.16%) Tornado
Applications
1 10 (0.53%) Spreadsheet
2 6 (0.32%) Photoshop
3 5 (0.26%) Adobe Illustrator
3 5 (0.26%) Fireworks
3 5 (0.26%) Microsoft Excel
4 2 (0.11%) Microsoft PowerPoint
4 2 (0.11%) SPSS
5 1 (0.053%) Chatbot
5 1 (0.053%) Figma
Business Applications
1 2 (0.11%) SAP S/4HANA
2 1 (0.053%) Oracle Procure-to-Pay
2 1 (0.053%) SAP Oil and Gas
2 1 (0.053%) SAP R/3
Cloud Services
1 1,060 (56.00%) AWS
2 652 (34.44%) Azure
3 563 (29.74%) GCP
4 336 (17.75%) Amazon Kinesis
5 229 (12.10%) GitHub
6 206 (10.88%) Serverless
7 167 (8.82%) Snowflake
8 162 (8.56%) Amazon EMR
9 160 (8.45%) Amazon S3
10 130 (6.87%) Power Platform
11 125 (6.60%) AWS Lambda
12 115 (6.08%) Amazon EC2
13 100 (5.28%) OpenShift
14 84 (4.44%) Azure Data Factory
15 68 (3.59%) Azure Synapse Analytics
16 66 (3.49%) Amazon SageMaker
17 60 (3.17%) Azure Cosmos DB
18 59 (3.12%) SaaS
19 55 (2.91%) AWS CloudFormation
20 53 (2.80%) Azure Functions
Communications & Networking
1 25 (1.32%) Wireless
2 22 (1.16%) Internet
3 17 (0.90%) Cisco IPT
4 14 (0.74%) NetScaler
5 12 (0.63%) DNS
6 11 (0.58%) HTTP
7 9 (0.48%) DHCP
7 9 (0.48%) HTTPS
7 9 (0.48%) IPv4
7 9 (0.48%) IPv6
7 9 (0.48%) SMS
7 9 (0.48%) SMTP
7 9 (0.48%) SNMP
7 9 (0.48%) SSH
7 9 (0.48%) TCP/IP
8 5 (0.26%) Cisco ASA
8 5 (0.26%) Firewall
9 4 (0.21%) Sourcefire
10 3 (0.16%) Broadband
10 3 (0.16%) Network Security
Database & Business Intelligence
1 947 (50.03%) Big Data
2 885 (46.75%) Hadoop
3 566 (29.90%) NoSQL
4 390 (20.60%) PostgreSQL
5 355 (18.75%) HBase
6 344 (18.17%) Relational Database
7 338 (17.86%) Data Warehouse
8 276 (14.58%) Apache Hive
9 268 (14.16%) Data Lake
10 257 (13.58%) Amazon Redshift
11 250 (13.21%) Redis
12 234 (12.36%) MariaDB
13 165 (8.72%) Apache Cassandra
14 163 (8.61%) MongoDB
15 148 (7.82%) Amazon DynamoDB
16 135 (7.13%) MySQL
17 132 (6.97%) Tableau
18 129 (6.81%) BigQuery
19 123 (6.50%) Flink
20 118 (6.23%) Power BI
Development Applications
1 453 (23.93%) Jenkins
2 368 (19.44%) Git (software)
3 216 (11.41%) Gradle
4 196 (10.35%) Cypress.io
4 196 (10.35%) Sonatype Nexus
5 194 (10.25%) Eclipse
6 193 (10.20%) IntelliJ
7 192 (10.14%) JUnit
8 184 (9.72%) Cucumber
9 183 (9.67%) Mockito
9 183 (9.67%) Protractor
9 183 (9.67%) Storybook
10 129 (6.81%) Artifactory
11 89 (4.70%) SonarQube
12 57 (3.01%) JIRA
13 36 (1.90%) Atlassian Bamboo
14 32 (1.69%) GitLab
15 30 (1.58%) Bitbucket
16 27 (1.43%) Vagrant
17 22 (1.16%) kappa
General
1 634 (33.49%) Finance
2 343 (18.12%) Banking
3 328 (17.33%) Social Skills
4 277 (14.63%) Inclusion and Diversity
5 272 (14.37%) Law
6 191 (10.09%) Analytical Skills
7 134 (7.08%) Retail
8 87 (4.60%) Presentation Skills
9 61 (3.22%) Front Office
9 61 (3.22%) Games
10 57 (3.01%) Advertising
11 49 (2.59%) Marketing
12 35 (1.85%) Manufacturing
12 35 (1.85%) Pharmaceutical
13 31 (1.64%) Investment Banking
14 28 (1.48%) Telecoms
15 27 (1.43%) Legal
16 24 (1.27%) Publishing
17 22 (1.16%) Public Sector
18 19 (1.00%) Influencing Skills
Job Titles
1 613 (32.38%) Data Engineer
2 322 (17.01%) Software Engineer
3 267 (14.10%) Developer
4 175 (9.24%) Architect
5 173 (9.14%) Senior Data Engineer
6 163 (8.61%) Data Scientist
7 139 (7.34%) Principal Engineer
8 130 (6.87%) Big Data Engineer
9 122 (6.44%) Data Architect
10 111 (5.86%) Principal Software Engineer
11 103 (5.44%) Java Developer
12 72 (3.80%) Lead Software Engineer
13 69 (3.65%) Python Engineer
14 68 (3.59%) Senior Developer
15 66 (3.49%) Java Engineer
16 65 (3.43%) Senior Software Engineer
17 61 (3.22%) Software Developer
18 58 (3.06%) Lead Developer
19 52 (2.75%) AWS Engineer
20 48 (2.54%) Consultant
Libraries, Frameworks & Software Standards
1 957 (50.55%) Kafka
2 377 (19.92%) Spring
3 322 (17.01%) React
4 272 (14.37%) Node.js
5 268 (14.16%) AngularJS
6 261 (13.79%) RabbitMQ
7 247 (13.05%) Spark Streaming
8 236 (12.47%) CSS
9 231 (12.20%) RESTful
10 230 (12.15%) JSON
11 194 (10.25%) Redux
12 188 (9.93%) Twitter Bootstrap
13 184 (9.72%) Play Framework
14 183 (9.67%) Express
14 183 (9.67%) Tailwind CSS
15 177 (9.35%) WireMock
16 142 (7.50%) LAPP Stack
17 140 (7.40%) Spring Boot
18 135 (7.13%) LAMP
19 116 (6.13%) Web Services
Miscellaneous
1 222 (11.73%) Algorithms
2 207 (10.94%) User Experience
3 204 (10.78%) Linux Command Line
4 107 (5.65%) Data Structures
5 104 (5.49%) Self-Motivation
6 67 (3.54%) Virtual Team
7 65 (3.43%) Cloud Native
8 51 (2.69%) Greenfield Project
9 44 (2.32%) Life Science
10 39 (2.06%) Hedge funds
11 37 (1.95%) Equities
12 32 (1.69%) Derivative
13 31 (1.64%) Public Cloud
14 29 (1.53%) Management Information System
15 28 (1.48%) Hybrid Cloud
16 27 (1.43%) Social Media
17 26 (1.37%) Foreign Exchange (FX)
18 22 (1.16%) Social Network
19 21 (1.11%) iPhone
19 21 (1.11%) SWIFT
Operating Systems
1 376 (19.86%) Linux
2 335 (17.70%) Unix
3 32 (1.69%) Windows
4 21 (1.11%) Red Hat Enterprise Linux
5 15 (0.79%) Windows Server
6 14 (0.74%) CentOS
7 7 (0.37%) Apple iOS
8 4 (0.21%) DC/OS
9 1 (0.053%) Android
9 1 (0.053%) Windows Server 2012
Processes & Methodologies
1 784 (41.42%) Agile Software Development
2 779 (41.15%) Analytics
3 682 (36.03%) Software Engineering
4 661 (34.92%) DevOps
5 584 (30.85%) Data Science
6 506 (26.73%) Microservices
7 485 (25.62%) Data Pipeline
8 464 (24.51%) Computer Science
9 452 (23.88%) Machine Learning
10 390 (20.60%) Data Engineering
11 386 (20.39%) ETL
12 351 (18.54%) Data Modelling
13 289 (15.27%) Full Stack Development
14 286 (15.11%) Open Source
15 282 (14.90%) Artificial Intelligence
16 270 (14.26%) Scrum
17 258 (13.63%) OO
18 256 (13.52%) Continuous Integration
19 247 (13.05%) Business Intelligence
20 230 (12.15%) Digital Banking
Programming Languages
1 1,307 (69.04%) Python
2 975 (51.51%) Java
3 860 (45.43%) SQL
4 478 (25.25%) Scala
5 246 (13.00%) Shell Script
6 199 (10.51%) JavaScript
7 191 (10.09%) Groovy
8 169 (8.93%) R
9 116 (6.13%) C#
10 87 (4.60%) TypeScript
11 74 (3.91%) Bash Shell
12 73 (3.86%) Go
12 73 (3.86%) PowerShell
13 49 (2.59%) C++
14 47 (2.48%) C
15 37 (1.95%) PL/SQL
16 35 (1.85%) Java 8
17 26 (1.37%) MATLAB
18 21 (1.11%) F#
19 18 (0.95%) Perl
Qualifications
1 484 (25.57%) Degree
2 177 (9.35%) OCP
2 177 (9.35%) Oracle Certification
3 137 (7.24%) Computer Science Degree
4 53 (2.80%) PhD
5 35 (1.85%) Master's Degree
6 29 (1.53%) Security Cleared
7 24 (1.27%) AWS Certification
7 24 (1.27%) SC Cleared
8 17 (0.90%) (ISC)2 CCSP
8 17 (0.90%) Cisco Certification
8 17 (0.90%) CISSP
9 15 (0.79%) CCSP
10 14 (0.74%) RHCE
11 8 (0.42%) Postgraduate
12 5 (0.26%) Numerate Degree
12 5 (0.26%) TOGAF Certification
13 4 (0.21%) DV Cleared
14 3 (0.16%) Microsoft Certification
15 2 (0.11%) MCDBA
Quality Assurance & Compliance
1 116 (6.13%) Data Quality
2 80 (4.23%) QA
3 34 (1.80%) GDPR
4 16 (0.85%) GxP
5 4 (0.21%) PMO
6 3 (0.16%) PCI DSS
6 3 (0.16%) Sarbanes-Oxley
7 2 (0.11%) ISO 8583
7 2 (0.11%) MiFID
8 1 (0.053%) COBIT
8 1 (0.053%) IFRS
8 1 (0.053%) IFRS 9
8 1 (0.053%) ISO/IEC 27001
8 1 (0.053%) NIST
8 1 (0.053%) SLA
System Software
1 422 (22.29%) Docker
2 220 (11.62%) HDFS
3 142 (7.50%) VMware Infrastructure
4 28 (1.48%) Apache ZooKeeper
5 27 (1.43%) VirtualBox
6 19 (1.00%) Active Directory
7 14 (0.74%) vSphere
7 14 (0.74%) XenApp
7 14 (0.74%) XenDesktop
8 10 (0.53%) Apache Flume
9 1 (0.053%) Hyper-V
9 1 (0.053%) Platform Symphony
9 1 (0.053%) Virtual Machines
9 1 (0.053%) Virtual Servers
Systems Management
1 466 (24.62%) Kubernetes
2 374 (19.76%) Ansible
3 233 (12.31%) Terraform
4 69 (3.65%) Opscode Chef
5 35 (1.85%) Istio
6 31 (1.64%) Oozie
7 28 (1.48%) Puppet
8 27 (1.43%) Docker Swarm
9 25 (1.32%) Hadoop YARN
10 15 (0.79%) Mesos
11 11 (0.58%) Prometheus
12 8 (0.42%) Consul
12 8 (0.42%) Control-M
13 7 (0.37%) Kubeflow
14 5 (0.26%) Tivoli
15 4 (0.21%) Apache Brooklyn
15 4 (0.21%) Grafana
15 4 (0.21%) Kibana
15 4 (0.21%) Marathon
16 3 (0.16%) FortiGate
Vendors
1 234 (12.36%) Databricks
2 206 (10.88%) Google
3 200 (10.57%) VMware
4 183 (9.67%) Okta
5 162 (8.56%) Oracle
6 132 (6.97%) Microsoft
7 81 (4.28%) Cloudera
8 67 (3.54%) Talend
9 63 (3.33%) Hortonworks
10 52 (2.75%) Informatica
11 37 (1.95%) Red Hat
12 36 (1.90%) IBM
12 36 (1.90%) Splunk
13 27 (1.43%) Facebook
13 27 (1.43%) Teradata
14 24 (1.27%) StreamSets
15 21 (1.11%) Samsung
15 21 (1.11%) ServiceNow
16 19 (1.00%) Greenplum
17 18 (0.95%) Dataiku