Period
to 29 July 2021

The following table provides summary statistics for permanent job vacancies with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 29 July 2021 with a comparison to the same period in the previous 2 years.

Apache Spark
UK
6 months to
29 Jul 2021
Same period 2020 Same period 2019
Rank 224 240 231
Rank change year-on-year +16 -9 +12
Permanent jobs citing Apache Spark 1,718 932 2,194
As % of all permanent jobs advertised in the UK 1.52% 1.35% 1.52%
As % of the Application Platforms category 11.04% 9.49% 10.22%
Number of salaries quoted 1,024 644 1,804
10th Percentile £46,250 £47,500 £47,500
25th Percentile £56,250 £56,250 £56,250
Median annual salary (50th Percentile) £75,000 £75,000 £74,218
Median % change year-on-year - +1.05% +6.03%
75th Percentile £90,263 £93,000 £95,000
90th Percentile £107,500 £115,000 £110,000
UK excluding London median annual salary £65,000 £61,000 £60,000
% change year-on-year +6.56% +1.67% +3.90%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for application platform skills.

All Application Platform Skills
UK
Permanent vacancies with a requirement for application platform skills 15,561 9,817 21,470
As % of all permanent IT jobs advertised in the UK 13.78% 14.23% 14.91%
Number of salaries quoted 10,254 7,821 17,112
10th Percentile £31,250 £31,250 £28,000
25th Percentile £40,000 £40,000 £36,250
Median annual salary (50th Percentile) £55,000 £52,500 £50,000
Median % change year-on-year +4.76% +5.00% -
75th Percentile £73,750 £72,500 £68,750
90th Percentile £88,750 £88,750 £85,000
UK excluding London median annual salary £45,000 £45,000 £43,500
% change year-on-year - +3.45% +1.16%

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a proportion of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Salary Trend

3-month moving average salary quoted in jobs citing Apache Spark.

Salary trend for Apache Spark in the UK

Apache Spark
Salary Histogram

Salary distribution for jobs citing Apache Spark over the 6 months to 29 July 2021.

Salary histogram for Apache Spark in the UK

Apache Spark
Top 18 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the UK over the 6 months to 29 July 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +20 1,572 £76,650 -1.10% 214
London +11 1,060 £77,500 -3.13% 141
UK excluding London +19 589 £65,000 +6.56% 89
South East +49 293 £85,000 +16.44% 20
Work from Home -80 248 £80,000 +6.67% 37
North of England -20 79 £60,000 +6.67% 24
Scotland -59 61 £62,500 +13.64% 9
South West -60 48 £62,216 +24.43% 8
Yorkshire -32 45 £60,000 +1.27% 11
East of England -25 41 £80,000 +10.34% 7
North West -45 32 £62,750 +14.09% 12
Midlands -104 28 £60,000 -20.00% 13
Wales -26 22 £45,000 -10.00% 6
West Midlands -60 20 £65,000 +18.18% 10
Northern Ireland - 16 £75,000 - 2
East Midlands -74 8 £50,000 -33.33% 3
North East -23 2 £56,914 -1.02% 1
Isle of Man - 1 £37,500 -

For the 6 months to 29 July 2021, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Apache Spark.

1 1,150 (66.94%) Python
2 942 (54.83%) AWS
3 878 (51.11%) Big Data
4 848 (49.36%) Java
5 841 (48.95%) Kafka
6 831 (48.37%) Hadoop
7 793 (46.16%) SQL
8 731 (42.55%) Agile Software Development
9 654 (38.07%) Analytics
10 630 (36.67%) Azure
11 629 (36.61%) Software Engineering
12 575 (33.47%) Finance
13 563 (32.77%) Kubernetes
14 552 (32.13%) DevOps
15 535 (31.14%) NoSQL
16 519 (30.21%) GCP
17 515 (29.98%) Data Pipeline
18 485 (28.23%) Apache
19 482 (28.06%) Docker
19 482 (28.06%) Data Science
20 447 (26.02%) Microservices
21 435 (25.32%) Jenkins
22 420 (24.45%) Degree
23 408 (23.75%) Scala
24 391 (22.76%) ETL
25 384 (22.35%) Computer Science
25 384 (22.35%) Data Engineering
26 382 (22.24%) Machine Learning
27 372 (21.65%) Ansible
28 347 (20.20%) Git (software)

Apache Spark
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 485 (28.23%) Apache
2 171 (9.95%) Apache Airflow
3 168 (9.78%) GemFire
4 151 (8.79%) Elasticsearch
5 44 (2.56%) Cloud Foundry
6 42 (2.44%) Luigi
7 36 (2.10%) Jupyter
8 35 (2.04%) Apache Pig
9 21 (1.22%) SAS
10 20 (1.16%) Oracle Application Server
11 19 (1.11%) Oracle Workflow
11 19 (1.11%) Tomcat
12 14 (0.81%) Confluence
13 12 (0.70%) Apache Synapse
14 9 (0.52%) Apache Solr
15 4 (0.23%) Oracle SOA Suite
16 3 (0.17%) DataSynapse
16 3 (0.17%) JBoss
16 3 (0.17%) Mule
16 3 (0.17%) SharePoint
Applications
1 8 (0.47%) Spreadsheet
2 6 (0.35%) Gephi
3 5 (0.29%) Adobe Illustrator
3 5 (0.29%) Fireworks
3 5 (0.29%) Microsoft Excel
3 5 (0.29%) Photoshop
4 3 (0.17%) Microsoft PowerPoint
4 3 (0.17%) SPSS
5 2 (0.12%) Microsoft Office
6 1 (0.058%) Microsoft Project
6 1 (0.058%) MS Visio
Business Applications
1 2 (0.12%) Oracle Procure-to-Pay
1 2 (0.12%) SAP R/3
2 1 (0.058%) Dynamics CRM
Cloud Services
1 942 (54.83%) AWS
2 630 (36.67%) Azure
3 519 (30.21%) GCP
4 258 (15.02%) Amazon Kinesis
5 197 (11.47%) GitHub
6 176 (10.24%) Snowflake
7 165 (9.60%) Serverless
8 163 (9.49%) Amazon EMR
9 122 (7.10%) Power Platform
10 96 (5.59%) AWS Lambda
11 86 (5.01%) OpenShift
12 81 (4.71%) Amazon S3
13 77 (4.48%) Amazon EC2
14 69 (4.02%) IBM Cloud
15 65 (3.78%) Azure Data Factory
16 56 (3.26%) Amazon SageMaker
17 50 (2.91%) SaaS
18 44 (2.56%) PaaS
19 43 (2.50%) Azure Synapse Analytics
20 35 (2.04%) Cloud Functions
Communications & Networking
1 27 (1.57%) Internet
2 19 (1.11%) Wireless
3 14 (0.81%) DNS
4 11 (0.64%) Firewall
4 11 (0.64%) HTTP
4 11 (0.64%) SMTP
5 10 (0.58%) NetScaler
6 8 (0.47%) SMS
7 7 (0.41%) SSL
8 6 (0.35%) Ethernet
8 6 (0.35%) HTTPS
8 6 (0.35%) IMAP
8 6 (0.35%) Intrusion Detection
8 6 (0.35%) MPLS
8 6 (0.35%) VPN
9 5 (0.29%) Broadband
9 5 (0.29%) DHCP
9 5 (0.29%) SNMP
9 5 (0.29%) SSH
9 5 (0.29%) TCP/IP
Database & Business Intelligence
1 878 (51.11%) Big Data
2 831 (48.37%) Hadoop
3 535 (31.14%) NoSQL
4 341 (19.85%) PostgreSQL
5 301 (17.52%) Relational Database
6 285 (16.59%) HBase
7 274 (15.95%) Data Warehouse
8 252 (14.67%) Amazon Redshift
9 229 (13.33%) Apache Hive
10 225 (13.10%) Redis
11 220 (12.81%) Data Lake
12 212 (12.34%) MariaDB
13 143 (8.32%) MongoDB
14 115 (6.69%) Apache Cassandra
15 114 (6.64%) Amazon Athena
16 110 (6.40%) Power BI
17 103 (6.00%) Amazon DynamoDB
18 102 (5.94%) Neo4j
19 98 (5.70%) Flink
20 96 (5.59%) Tableau
Development Applications
1 435 (25.32%) Jenkins
2 347 (20.20%) Git (software)
3 202 (11.76%) Gradle
4 181 (10.54%) Cypress.io
5 177 (10.30%) Sonatype Nexus
6 176 (10.24%) IntelliJ
7 174 (10.13%) Eclipse
8 171 (9.95%) JUnit
9 168 (9.78%) Cucumber
10 166 (9.66%) Mockito
10 166 (9.66%) Protractor
10 166 (9.66%) Storybook
11 113 (6.58%) Artifactory
12 76 (4.42%) SonarQube
13 75 (4.37%) Atlassian Bamboo
14 74 (4.31%) Vagrant
15 58 (3.38%) JIRA
16 31 (1.80%) GitLab
17 29 (1.69%) Oracle APEX
18 25 (1.46%) Maven
General
1 575 (33.47%) Finance
2 321 (18.68%) Banking
3 265 (15.42%) Social Skills
4 250 (14.55%) Law
5 230 (13.39%) Inclusion and Diversity
6 193 (11.23%) Analytical Skills
7 105 (6.11%) Retail
8 80 (4.66%) Presentation Skills
9 70 (4.07%) Front Office
10 48 (2.79%) Games
11 45 (2.62%) Marketing
12 37 (2.15%) Advertising
13 30 (1.75%) Telecoms
14 22 (1.28%) Influencing Skills
14 22 (1.28%) Retail Banking
15 20 (1.16%) Investment Banking
16 15 (0.87%) Manufacturing
17 12 (0.70%) Czech Language
18 10 (0.58%) Legal
18 10 (0.58%) Local Government
Job Titles
1 562 (32.71%) Data Engineer
2 312 (18.16%) Software Engineer
3 281 (16.36%) Developer
4 185 (10.77%) Architect
5 151 (8.79%) Senior Data Engineer
6 138 (8.03%) Big Data Engineer
7 128 (7.45%) Data Scientist
8 121 (7.04%) Data Architect
9 118 (6.87%) Java Developer
10 103 (6.00%) Principal Engineer
11 96 (5.59%) Principal Software Engineer
12 90 (5.24%) AWS Engineer
13 71 (4.13%) Python Engineer
14 70 (4.07%) Java Engineer
15 68 (3.96%) Platform Engineer
16 56 (3.26%) Lead Software Engineer
17 55 (3.20%) Senior Software Engineer
18 52 (3.03%) Cloud Engineer
19 50 (2.91%) Azure Engineer
20 43 (2.50%) Python Developer
Libraries, Frameworks & Software Standards
1 841 (48.95%) Kafka
2 342 (19.91%) Spring
3 323 (18.80%) React
4 295 (17.17%) Spark Streaming
5 283 (16.47%) AngularJS
6 275 (16.01%) Node.js
7 228 (13.27%) CSS
8 221 (12.86%) RESTful
9 209 (12.17%) RabbitMQ
10 205 (11.93%) JSON
11 173 (10.07%) Redux
12 171 (9.95%) Twitter Bootstrap
13 166 (9.66%) Express
13 166 (9.66%) Tailwind CSS
14 161 (9.37%) WireMock
15 155 (9.02%) Play Framework
16 142 (8.27%) Spring Boot
17 118 (6.87%) Web Services
18 115 (6.69%) LAMP
18 115 (6.69%) LAPP Stack
Miscellaneous
1 183 (10.65%) Algorithms
1 183 (10.65%) Linux Command Line
2 174 (10.13%) User Experience
3 91 (5.30%) Data Structures
4 88 (5.12%) Self-Motivation
5 57 (3.32%) Equities
5 57 (3.32%) Hedge funds
6 43 (2.50%) Hybrid Cloud
7 38 (2.21%) Greenfield Project
8 37 (2.15%) Cloud Native
9 33 (1.92%) Derivative
10 27 (1.57%) Virtual Team
11 25 (1.46%) Foreign Exchange (FX)
12 20 (1.16%) Social Media
13 19 (1.11%) Management Information System
13 19 (1.11%) Public Cloud
13 19 (1.11%) Social Network
14 16 (0.93%) NHS
15 15 (0.87%) Enterprise Cloud
16 14 (0.81%) SWIFT
Operating Systems
1 326 (18.98%) Linux
2 282 (16.41%) Unix
3 25 (1.46%) Windows
4 14 (0.81%) Red Hat Enterprise Linux
5 11 (0.64%) Windows Server
6 10 (0.58%) CentOS
7 9 (0.52%) Apple iOS
8 3 (0.17%) Android
9 1 (0.058%) Windows Server 2012
Processes & Methodologies
1 731 (42.55%) Agile Software Development
2 654 (38.07%) Analytics
3 629 (36.61%) Software Engineering
4 552 (32.13%) DevOps
5 515 (29.98%) Data Pipeline
6 482 (28.06%) Data Science
7 447 (26.02%) Microservices
8 391 (22.76%) ETL
9 384 (22.35%) Computer Science
9 384 (22.35%) Data Engineering
10 382 (22.24%) Machine Learning
11 303 (17.64%) Artificial Intelligence
12 274 (15.95%) Full Stack Development
13 271 (15.77%) Data Modelling
14 258 (15.02%) Open Source
15 255 (14.84%) Continuous Integration
16 219 (12.75%) OO
17 216 (12.57%) Business Intelligence
18 208 (12.11%) Digital Banking
19 204 (11.87%) Validation
Programming Languages
1 1,150 (66.94%) Python
2 848 (49.36%) Java
3 793 (46.16%) SQL
4 408 (23.75%) Scala
5 201 (11.70%) Shell Script
6 199 (11.58%) JavaScript
7 170 (9.90%) Groovy
8 132 (7.68%) C#
9 113 (6.58%) R
10 77 (4.48%) TypeScript
11 50 (2.91%) C++
11 50 (2.91%) Go
12 48 (2.79%) Bash Shell
13 46 (2.68%) C
13 46 (2.68%) PowerShell
14 45 (2.62%) PL/SQL
15 42 (2.44%) Java 8
16 26 (1.51%) T-SQL
17 25 (1.46%) Kotlin
18 23 (1.34%) Perl
Qualifications
1 420 (24.45%) Degree
2 162 (9.43%) OCP
2 162 (9.43%) Oracle Certification
3 121 (7.04%) Computer Science Degree
4 38 (2.21%) Security Cleared
5 34 (1.98%) PhD
6 19 (1.11%) SC Cleared
7 18 (1.05%) Master's Degree
8 11 (0.64%) AWS Certification
8 11 (0.64%) DV Cleared
9 10 (0.58%) RHCE
10 9 (0.52%) Microsoft Certification
11 8 (0.47%) Cisco Certification
11 8 (0.47%) CISSP
11 8 (0.47%) TOGAF Certification
12 7 (0.41%) Postgraduate
13 6 (0.35%) CCNA
13 6 (0.35%) MCSE
13 6 (0.35%) Network+ Certification
13 6 (0.35%) SSCP
Quality Assurance & Compliance
1 94 (5.47%) Data Quality
2 67 (3.90%) QA
3 27 (1.57%) GDPR
4 6 (0.35%) GxP
5 3 (0.17%) PCI DSS
6 2 (0.12%) COBIT
6 2 (0.12%) ISO/IEC 27001
6 2 (0.12%) MiFID
6 2 (0.12%) NIST
System Software
1 482 (28.06%) Docker
2 202 (11.76%) HDFS
3 122 (7.10%) VMware Infrastructure
4 74 (4.31%) VirtualBox
5 23 (1.34%) Apache ZooKeeper
6 22 (1.28%) Apache Flume
7 12 (0.70%) Active Directory
8 10 (0.58%) vSphere
8 10 (0.58%) XenApp
8 10 (0.58%) XenDesktop
9 4 (0.23%) Virtual Servers
10 2 (0.12%) Firmware
10 2 (0.12%) Platform Symphony
11 1 (0.058%) Hyper-V
11 1 (0.058%) Virtual Machines
Systems Management
1 563 (32.77%) Kubernetes
2 372 (21.65%) Ansible
3 252 (14.67%) Terraform
4 74 (4.31%) Docker Swarm
5 49 (2.85%) Opscode Chef
6 36 (2.10%) Istio
7 27 (1.57%) Hadoop YARN
8 22 (1.28%) Oozie
9 21 (1.22%) Prometheus
9 21 (1.22%) Puppet
10 11 (0.64%) Kibana
11 9 (0.52%) Kubeflow
12 7 (0.41%) Ambari
12 7 (0.41%) Mesos
13 4 (0.23%) Grafana
13 4 (0.23%) Tivoli
14 3 (0.17%) logstash
15 2 (0.12%) Control-M
15 2 (0.12%) FortiGate
16 1 (0.058%) ITRS Geneos
Vendors
1 235 (13.68%) Google
2 200 (11.64%) Databricks
3 179 (10.42%) VMware
4 166 (9.66%) Okta
5 134 (7.80%) Oracle
6 119 (6.93%) Cloudera
7 112 (6.52%) Talend
8 103 (6.00%) Hortonworks
8 103 (6.00%) Microsoft
9 85 (4.95%) IBM
10 82 (4.77%) Informatica
11 31 (1.80%) Splunk
12 25 (1.46%) Teradata
13 24 (1.40%) Red Hat
14 20 (1.16%) Greenplum
15 17 (0.99%) Cisco
16 14 (0.81%) Samsung
17 13 (0.76%) Qubit
18 12 (0.70%) Instagram
19 11 (0.64%) SAP