Period
to 15 April 2021

The following table provides summary statistics for permanent job vacancies with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 15 April 2021 with a comparison to the same period in the previous 2 years.

Apache Spark
UK
6 months to
15 Apr 2021
Same period 2020 Same period 2019
Rank 248 242 240
Rank change year-on-year -6 -2 -10
Permanent jobs citing Apache Spark 1,079 1,496 2,212
As % of all permanent jobs advertised in the UK 1.29% 1.33% 1.49%
As % of the Application Platforms category 9.52% 9.21% 9.60%
Number of salaries quoted 858 1,141 1,797
10th Percentile £46,250 £47,500 £47,500
25th Percentile £55,000 £56,250 £56,489
Median annual salary (50th Percentile) £75,000 £72,500 £75,000
Median % change year-on-year +3.45% -3.33% +7.14%
75th Percentile £92,500 £87,500 £92,500
90th Percentile £110,000 £105,000 £106,250
UK excluding London median annual salary £62,216 £60,000 £60,000
% change year-on-year +3.69% - +9.09%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for application platform skills.

All Application Platform Skills
UK
Permanent vacancies with a requirement for application platform skills 11,337 16,240 23,052
As % of all permanent IT jobs advertised in the UK 13.55% 14.45% 15.49%
Number of salaries quoted 8,701 13,025 18,382
10th Percentile £31,250 £29,000 £28,000
25th Percentile £40,000 £37,500 £36,250
Median annual salary (50th Percentile) £55,000 £50,000 £50,000
Median % change year-on-year +10.00% - -
75th Percentile £72,500 £70,000 £68,750
90th Percentile £87,500 £85,000 £85,000
UK excluding London median annual salary £47,500 £45,000 £43,250
% change year-on-year +5.56% +4.05% +1.76%

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a proportion of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Salary Trend

3-month moving average salary quoted in jobs citing Apache Spark.

Salary trend for Apache Spark in the UK

Apache Spark
Salary Histogram

Salary distribution for jobs citing Apache Spark over the 6 months to 15 April 2021.

Salary histogram for Apache Spark in the UK

Apache Spark
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the UK over the 6 months to 15 April 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -9 982 £75,000 +3.45% 172
London +7 699 £77,500 +3.33% 120
UK excluding London +17 337 £62,216 +3.69% 64
Work from Home -21 223 £75,000 - 33
North of England +46 86 £55,000 - 14
South East +14 75 £58,497 -10.00% 13
Scotland +23 58 £71,250 +29.55% 10
South West +41 50 £65,000 +19.82% 7
Yorkshire +24 49 £55,000 - 6
North West +30 37 £60,000 +14.29% 8
East of England +37 31 £65,000 -13.33% 6
Midlands -57 23 £67,500 -6.90% 11
West Midlands +3 15 £75,000 +36.36% 8
Wales +36 10 £50,000 -16.67% 3
East Midlands -41 8 £65,000 -13.33% 3
Northern Ireland 0 3 £65,000 +18.18%
Isle of Man - 1 £37,500 -

For the 6 months to 15 April 2021, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Apache Spark.

1 744 (68.95%) Python
2 566 (52.46%) Hadoop
3 558 (51.71%) Big Data
4 521 (48.29%) AWS
5 508 (47.08%) SQL
6 504 (46.71%) Kafka
7 421 (39.02%) Java
8 386 (35.77%) Azure
9 357 (33.09%) Kubernetes
10 354 (32.81%) Agile Software Development
11 348 (32.25%) Docker
12 335 (31.05%) Data Science
13 331 (30.68%) Analytics
14 313 (29.01%) NoSQL
15 298 (27.62%) GCP
16 297 (27.53%) Scala
17 283 (26.23%) Software Engineering
18 274 (25.39%) ETL
19 270 (25.02%) Data Engineering
20 263 (24.37%) Finance
21 254 (23.54%) Machine Learning
22 227 (21.04%) DevOps
23 198 (18.35%) Microservices
24 197 (18.26%) Apache
25 185 (17.15%) Data Warehouse
26 184 (17.05%) Artificial Intelligence
27 174 (16.13%) Data Lake
28 166 (15.38%) Terraform
29 161 (14.92%) Apache Hive
30 157 (14.55%) JavaScript

Apache Spark
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 197 (18.26%) Apache
2 102 (9.45%) Apache Airflow
3 73 (6.77%) Elasticsearch
4 50 (4.63%) Luigi
5 32 (2.97%) Apache Pig
6 24 (2.22%) SAS
7 16 (1.48%) Apache Synapse
8 14 (1.30%) GemFire
9 12 (1.11%) Jupyter
9 12 (1.11%) Oracle Workflow
10 7 (0.65%) Tornado
11 6 (0.56%) Tomcat
12 5 (0.46%) Confluence
13 4 (0.37%) CMS
14 3 (0.28%) Apache Solr
15 2 (0.19%) BizTalk Server
15 2 (0.19%) DataSynapse
15 2 (0.19%) JBoss
15 2 (0.19%) Oracle SOA Suite
15 2 (0.19%) WebSphere
Applications
1 16 (1.48%) Microsoft Excel
2 10 (0.93%) SPSS
3 6 (0.56%) Gephi
4 2 (0.19%) Microsoft Office
4 2 (0.19%) Microsoft PowerPoint
4 2 (0.19%) SPSS Modeler
5 1 (0.093%) Microsoft Project
5 1 (0.093%) MS Visio
Business Applications
1 2 (0.19%) Dynamics CRM
2 1 (0.093%) SAP APO
Cloud Services
1 521 (48.29%) AWS
2 386 (35.77%) Azure
3 298 (27.62%) GCP
4 98 (9.08%) Amazon EMR
5 82 (7.60%) Serverless
6 71 (6.58%) Amazon Kinesis
7 68 (6.30%) Power Platform
8 59 (5.47%) Amazon S3
9 48 (4.45%) Azure Data Factory
9 48 (4.45%) GitHub
9 48 (4.45%) IBM Cloud
10 39 (3.61%) AWS Lambda
11 31 (2.87%) Cloud Computing
12 28 (2.59%) Amazon EC2
12 28 (2.59%) Cloud Functions
13 25 (2.32%) PaaS
13 25 (2.32%) SaaS
14 23 (2.13%) AWS CloudFormation
15 19 (1.76%) Amazon ECS
16 18 (1.67%) Google Compute Engine
Communications & Networking
1 28 (2.59%) Firewall
2 27 (2.50%) HTTP
3 20 (1.85%) Internet
4 14 (1.30%) DNS
4 14 (1.30%) TCP/IP
5 11 (1.02%) VPN
6 7 (0.65%) Wireless
7 6 (0.56%) Ethernet
7 6 (0.56%) Fibre Channel
7 6 (0.56%) IMAP
7 6 (0.56%) Intrusion Detection
7 6 (0.56%) LAN
7 6 (0.56%) MPLS
7 6 (0.56%) SAN
7 6 (0.56%) SMTP
7 6 (0.56%) SSL
7 6 (0.56%) WAN
8 5 (0.46%) DMZ
9 4 (0.37%) FTP
9 4 (0.37%) Kerberos
Database & Business Intelligence
1 566 (52.46%) Hadoop
2 558 (51.71%) Big Data
3 313 (29.01%) NoSQL
4 185 (17.15%) Data Warehouse
5 174 (16.13%) Data Lake
6 161 (14.92%) Apache Hive
7 151 (13.99%) Amazon Redshift
8 132 (12.23%) PostgreSQL
9 103 (9.55%) MongoDB
10 91 (8.43%) Amazon Athena
11 88 (8.16%) Relational Database
12 87 (8.06%) BigQuery
13 84 (7.78%) Apache Cassandra
14 76 (7.04%) SQL Server
15 74 (6.86%) HBase
16 72 (6.67%) Tableau
17 65 (6.02%) Power BI
18 62 (5.75%) Neo4j
19 58 (5.38%) Flink
20 53 (4.91%) Electronic Patient Records
Development Applications
1 144 (13.35%) Jenkins
2 117 (10.84%) Git (software)
3 49 (4.54%) Atlassian Bamboo
3 49 (4.54%) Vagrant
4 42 (3.89%) JIRA
5 31 (2.87%) Maven
6 29 (2.69%) Sonatype Nexus
7 22 (2.04%) Cucumber
7 22 (2.04%) Gradle
8 17 (1.58%) IntelliJ
9 15 (1.39%) Bitbucket
9 15 (1.39%) Cypress.io
9 15 (1.39%) Eclipse
10 14 (1.30%) JUnit
11 13 (1.20%) Mockito
11 13 (1.20%) Protractor
11 13 (1.20%) SonarQube
11 13 (1.20%) Storybook
12 12 (1.11%) TeamCity
13 11 (1.02%) GitLab
General
1 263 (24.37%) Finance
2 104 (9.64%) Banking
3 60 (5.56%) Retail
4 53 (4.91%) Front Office
5 39 (3.61%) Law
5 39 (3.61%) Marketing
6 35 (3.24%) Telecoms
7 30 (2.78%) Games
8 15 (1.39%) Advertising
9 14 (1.30%) Public Sector
10 10 (0.93%) Manufacturing
10 10 (0.93%) Retail Banking
11 9 (0.83%) Investment Banking
12 6 (0.56%) Electronics
12 6 (0.56%) Legal
13 4 (0.37%) Financial Institution
13 4 (0.37%) Publishing
14 2 (0.19%) Back Office
14 2 (0.19%) Billing
14 2 (0.19%) Czech Language
Job Titles
1 416 (38.55%) Data Engineer
2 238 (22.06%) Developer
3 111 (10.29%) Software Engineer
4 109 (10.10%) Architect
5 94 (8.71%) Big Data Engineer
6 86 (7.97%) Senior Data Engineer
7 79 (7.32%) AWS Engineer
8 76 (7.04%) Data Scientist
9 74 (6.86%) Data Architect
10 70 (6.49%) Java Developer
11 63 (5.84%) Python Developer
12 58 (5.38%) Java Engineer
12 58 (5.38%) Python Engineer
13 49 (4.54%) Full Stack Developer
14 47 (4.36%) Platform Engineer
15 44 (4.08%) Developer/Software Engineer
16 42 (3.89%) Cloud Engineer
17 41 (3.80%) .NET Developer
18 39 (3.61%) JavaScript Developer
19 36 (3.34%) .NET Software Engineer
Libraries, Frameworks & Software Standards
1 504 (46.71%) Kafka
2 124 (11.49%) React
3 105 (9.73%) Node.js
4 99 (9.18%) Spark Streaming
5 84 (7.78%) AngularJS
6 72 (6.67%) Spring
7 65 (6.02%) REST
8 61 (5.65%) .NET
9 59 (5.47%) Spring Boot
10 54 (5.00%) PySpark
11 53 (4.91%) Web Services
12 52 (4.82%) RESTful
13 47 (4.36%) Pandas
14 45 (4.17%) NumPy
15 44 (4.08%) Natural Language Processing
16 42 (3.89%) Vue.js
17 41 (3.80%) TensorFlow
18 39 (3.61%) KnockoutJS
19 37 (3.43%) .NET Core
20 36 (3.34%) Apache NiFi
Miscellaneous
1 105 (9.73%) Analytical Skills
2 87 (8.06%) Algorithms
3 40 (3.71%) Equities
4 38 (3.52%) Hedge funds
5 37 (3.43%) Greenfield Project
6 32 (2.97%) Data Structures
7 27 (2.50%) Hybrid Cloud
8 25 (2.32%) Tandem
9 18 (1.67%) Self-Motivation
10 17 (1.58%) Cloud Native
11 16 (1.48%) User Experience
12 14 (1.30%) Linux Command Line
12 14 (1.30%) Market Risk
12 14 (1.30%) Public Cloud
12 14 (1.30%) Video on Demand
13 12 (1.11%) Foreign Exchange (FX)
13 12 (1.11%) Internet of Things
14 10 (0.93%) Credit Risk
15 9 (0.83%) Video\Live Streaming
16 8 (0.74%) Life Science
Operating Systems
1 87 (8.06%) Linux
2 39 (3.61%) Unix
3 16 (1.48%) Ubuntu
4 9 (0.83%) Windows
5 8 (0.74%) Red Hat Enterprise Linux
6 6 (0.56%) CentOS
7 5 (0.46%) Oracle Linux
8 1 (0.093%) Android
8 1 (0.093%) Apple iOS
8 1 (0.093%) Mesosphere
Processes & Methodologies
1 354 (32.81%) Agile Software Development
2 335 (31.05%) Data Science
3 331 (30.68%) Analytics
4 283 (26.23%) Software Engineering
5 274 (25.39%) ETL
6 270 (25.02%) Data Engineering
7 254 (23.54%) Machine Learning
8 227 (21.04%) DevOps
9 198 (18.35%) Microservices
10 184 (17.05%) Artificial Intelligence
11 156 (14.46%) Business Intelligence
12 133 (12.33%) Data Modelling
13 131 (12.14%) Fintech
14 129 (11.96%) Continuous Integration
15 121 (11.21%) Visualisation
16 106 (9.82%) Computer Science
16 106 (9.82%) Data Analytics
17 98 (9.08%) Data Architecture
18 94 (8.71%) Open Source
19 92 (8.53%) Data Integration
Programming Languages
1 744 (68.95%) Python
2 508 (47.08%) SQL
3 421 (39.02%) Java
4 297 (27.53%) Scala
5 157 (14.55%) JavaScript
6 140 (12.97%) C#
7 96 (8.90%) R
8 45 (4.17%) C++
9 37 (3.43%) Go
10 34 (3.15%) C
11 33 (3.06%) Bash Shell
12 28 (2.59%) TypeScript
13 22 (2.04%) T-SQL
14 21 (1.95%) Shell Script
15 19 (1.76%) PHP
16 18 (1.67%) Kotlin
17 16 (1.48%) ES6
18 15 (1.39%) Perl
19 13 (1.20%) Groovy
19 13 (1.20%) Java 8
Qualifications
1 134 (12.42%) Degree
2 48 (4.45%) Computer Science Degree
3 26 (2.41%) Security Cleared
4 18 (1.67%) PhD
5 13 (1.20%) OCP
5 13 (1.20%) Oracle Certification
6 9 (0.83%) DV Cleared
6 9 (0.83%) Numerate Degree
6 9 (0.83%) SC Cleared
7 7 (0.65%) TOGAF Certification
8 6 (0.56%) CCNA
8 6 (0.56%) Cisco Certification
8 6 (0.56%) CISSP
8 6 (0.56%) CompTIA Security+
8 6 (0.56%) GIAC
8 6 (0.56%) Linux+ Certification
8 6 (0.56%) Microsoft Certification
8 6 (0.56%) Network+ Certification
8 6 (0.56%) SANS
8 6 (0.56%) SSCP
Quality Assurance & Compliance
1 65 (6.02%) QA
2 44 (4.08%) Data Quality
3 7 (0.65%) GDPR
4 1 (0.093%) ISO 9001
4 1 (0.093%) ISO/IEC 27001
System Software
1 348 (32.25%) Docker
2 49 (4.54%) VirtualBox
3 47 (4.36%) HDFS
4 17 (1.58%) Apache Flume
5 14 (1.30%) VMware Infrastructure
6 7 (0.65%) Apache ZooKeeper
7 3 (0.28%) Virtual Servers
8 2 (0.19%) Firmware
9 1 (0.093%) NFS
9 1 (0.093%) oVirt
9 1 (0.093%) Platform Symphony
9 1 (0.093%) Virtual Machines
Systems Management
1 357 (33.09%) Kubernetes
2 166 (15.38%) Terraform
3 128 (11.86%) Ansible
4 49 (4.54%) Docker Swarm
5 31 (2.87%) Puppet
6 21 (1.95%) Oozie
7 20 (1.85%) Opscode Chef
8 15 (1.39%) Hadoop YARN
9 13 (1.20%) Prometheus
10 11 (1.02%) Nagios
11 8 (0.74%) Kibana
12 7 (0.65%) Graphite
12 7 (0.65%) Opsview
13 6 (0.56%) Kubeflow
14 4 (0.37%) Ambari
14 4 (0.37%) Istio
15 2 (0.19%) Grafana
15 2 (0.19%) Mesos
16 1 (0.093%) Marathon
16 1 (0.093%) SELinux
Vendors
1 153 (14.18%) Google
2 130 (12.05%) Databricks
3 100 (9.27%) Microsoft
4 94 (8.71%) Cloudera
5 82 (7.60%) Talend
6 69 (6.39%) Oracle
7 65 (6.02%) Hortonworks
8 57 (5.28%) Informatica
9 55 (5.10%) IBM
10 20 (1.85%) Teradata
11 17 (1.58%) VMware
12 16 (1.48%) Red Hat
13 15 (1.39%) SAP
13 15 (1.39%) Splunk
14 13 (1.20%) Okta
15 9 (0.83%) MuleSoft
16 7 (0.65%) Alteryx
17 6 (0.56%) ESRI
17 6 (0.56%) MicroStrategy
17 6 (0.56%) StreamSets