Period
to 27 October 2021

The table below provides summary statistics for permanent job vacancies with an option to work from home (WFH) and a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 27 October 2021 with a comparison to the same period in the previous 2 years.

Apache Spark
UK > Work from Home
6 months to
27 Oct 2021
Same period 2020 Same period 2019
Rank 303 207 211
Rank change year-on-year -96 +4 +20
Permanent jobs citing Apache Spark 264 59 82
As % of all permanent jobs with a WFH option 0.94% 0.90% 1.07%
As % of the Application Platforms category 6.46% 6.05% 7.56%
Number of salaries quoted 193 55 80
10th Percentile £53,042 £35,000 £47,375
25th Percentile £65,000 £43,750 £47,500
Median annual salary (50th Percentile) £82,500 £62,500 £61,250
Median % change year-on-year +32.00% +2.04% -18.33%
75th Percentile £98,750 £88,125 £80,000
90th Percentile £107,500 £105,500 £101,500
UK median annual salary £75,000 £70,000 £72,500
% change year-on-year +7.14% -3.45% +3.57%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a WFH option and a requirement for application platform skills.

All Application Platform Skills
Work from Home
Permanent vacancies with a requirement for application platform skills 4,086 975 1,085
As % of all permanent jobs with a WFH option 14.53% 14.79% 14.20%
Number of salaries quoted 2,775 864 940
10th Percentile £32,500 £32,500 £32,725
25th Percentile £41,250 £41,250 £41,250
Median annual salary (50th Percentile) £57,500 £52,500 £55,000
Median % change year-on-year +9.52% -4.55% +4.76%
75th Percentile £75,000 £67,500 £73,750
90th Percentile £92,500 £80,000 £86,250
UK median annual salary £55,000 £55,000 £50,000
% change year-on-year - +10.00% -

Apache Spark
Trend for Jobs with a WFH Option

Job vacancies with a work from home option citing Apache Spark as a proportion of all IT jobs advertised.

Apache Spark trend for jobs with a WFH option

Apache Spark
Salary Trend for Jobs with a WFH Option

3-month moving average salary quoted in job vacancies with a work from home option citing Apache Spark.

Apache Spark salary trend for jobs with a WFH option

Apache Spark
Salary Histogram for Jobs with a WFH Option

Salary distribution for jobs with a work from home option citing Apache Spark over the 6 months to 27 October 2021.

Apache Spark salary histogram for jobs with a WFH option

For the 6 months to 27 October 2021, Apache Spark job roles required the following IT skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent jobs with a WFH option and a requirement for Apache Spark.

1 158 (59.85%) Python
2 138 (52.27%) SQL
3 135 (51.14%) Kafka
4 125 (47.35%) Java
5 119 (45.08%) Hadoop
6 118 (44.70%) Big Data
7 109 (41.29%) DevOps
8 105 (39.77%) Analytics
8 105 (39.77%) AWS
9 103 (39.02%) Azure
10 79 (29.92%) Software Engineering
11 76 (28.79%) Microservices
12 74 (28.03%) Jenkins
13 73 (27.65%) Agile Software Development
14 72 (27.27%) Scala
14 72 (27.27%) Data Science
15 69 (26.14%) Data Modelling
16 66 (25.00%) ETL
16 66 (25.00%) Terraform
17 65 (24.62%) Ansible
17 65 (24.62%) Data Pipeline
18 64 (24.24%) Social Skills
19 59 (22.35%) Continuous Integration
20 58 (21.97%) Business Intelligence
20 58 (21.97%) GCP
21 56 (21.21%) Data Warehouse
21 56 (21.21%) Apache Hive
22 55 (20.83%) Apache
22 55 (20.83%) Machine Learning
23 52 (19.70%) NoSQL

Apache Spark
Co-occurring IT Skills with a WFH Option by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 55 (20.83%) Apache
2 21 (7.95%) Apache Airflow
3 19 (7.20%) Apache Pig
4 17 (6.44%) Elasticsearch
5 8 (3.03%) Tomcat
6 4 (1.52%) Confluence
6 4 (1.52%) Oracle SOA Suite
7 3 (1.14%) Luigi
8 2 (0.76%) Apache Solr
8 2 (0.76%) Jupyter
9 1 (0.38%) SAS
9 1 (0.38%) SharePoint
Cloud Services
1 105 (39.77%) AWS
2 103 (39.02%) Azure
3 58 (21.97%) GCP
4 34 (12.88%) Serverless
5 29 (10.98%) Power Platform
6 28 (10.61%) Amazon EMR
7 24 (9.09%) Azure Data Factory
8 19 (7.20%) Snowflake
9 18 (6.82%) IaaS
10 17 (6.44%) SaaS
11 14 (5.30%) Azure Synapse Analytics
12 13 (4.92%) Azure Logic Apps
13 11 (4.17%) Azure Sentinel
13 11 (4.17%) Matillion
13 11 (4.17%) PaaS
14 10 (3.79%) AWS Lambda
14 10 (3.79%) Azure Functions
14 10 (3.79%) OpenShift
15 9 (3.41%) GitHub
16 8 (3.03%) Amazon SageMaker
Communications & Networking
1 7 (2.65%) NetScaler
2 1 (0.38%) Cisco ASA
2 1 (0.38%) Firewall
2 1 (0.38%) HTTP
Database & Business Intelligence
1 119 (45.08%) Hadoop
2 118 (44.70%) Big Data
3 56 (21.21%) Apache Hive
3 56 (21.21%) Data Warehouse
4 52 (19.70%) NoSQL
5 45 (17.05%) PostgreSQL
6 37 (14.02%) Relational Database
7 32 (12.12%) Amazon Redshift
7 32 (12.12%) SQL Server
8 31 (11.74%) MySQL
9 28 (10.61%) BigQuery
9 28 (10.61%) Data Lake
10 26 (9.85%) MongoDB
11 23 (8.71%) Power BI
12 21 (7.95%) Redis
13 20 (7.58%) Apache Cassandra
14 16 (6.06%) HBase
15 14 (5.30%) Flink
16 13 (4.92%) Metadata
16 13 (4.92%) SQL Server Integration Services
Development Applications
1 74 (28.03%) Jenkins
2 45 (17.05%) Git (software)
3 13 (4.92%) Atlassian Bamboo
4 10 (3.79%) Bitbucket
5 7 (2.65%) Spinnaker
6 6 (2.27%) GitLab
6 6 (2.27%) TeamCity
7 5 (1.89%) Visual Studio
8 4 (1.52%) CircleCI
8 4 (1.52%) JIRA
8 4 (1.52%) kappa
8 4 (1.52%) Team Foundation Server
8 4 (1.52%) Vagrant
9 3 (1.14%) Octopus Deploy
9 3 (1.14%) Sonatype Nexus
10 2 (0.76%) RStudio
10 2 (0.76%) SpecFlow
11 1 (0.38%) GoCD
11 1 (0.38%) Postman
11 1 (0.38%) SoapUI
General
1 64 (24.24%) Social Skills
2 45 (17.05%) Retail
3 40 (15.15%) Finance
4 34 (12.88%) Analytical Skills
5 21 (7.95%) Advertising
6 17 (6.44%) Games
7 16 (6.06%) Marketing
8 14 (5.30%) Inclusion and Diversity
9 12 (4.55%) Banking
10 11 (4.17%) Czech Language
11 8 (3.03%) Investment Banking
11 8 (3.03%) Public Sector
12 6 (2.27%) Pharmaceutical
12 6 (2.27%) Presentation Skills
13 4 (1.52%) Law
14 3 (1.14%) Legal
Job Titles
1 80 (30.30%) Data Engineer
2 63 (23.86%) Developer
3 50 (18.94%) Java Developer
4 46 (17.42%) Architect
5 31 (11.74%) Data Architect
5 31 (11.74%) Data Scientist
6 27 (10.23%) Software Engineer
7 23 (8.71%) Cloud Architect
8 21 (7.95%) Azure Engineer
8 21 (7.95%) Principal Developer
8 21 (7.95%) Principal Java Developer
9 20 (7.58%) Senior Data Engineer
9 20 (7.58%) Solutions Architect
10 15 (5.68%) Cloud Engineer
10 15 (5.68%) Senior Developer
11 14 (5.30%) Senior Java Developer
12 13 (4.92%) AWS Engineer
12 13 (4.92%) Lead Developer
12 13 (4.92%) Lead Java Developer
12 13 (4.92%) Python Engineer
Libraries, Frameworks & Software Standards
1 135 (51.14%) Kafka
2 51 (19.32%) RabbitMQ
3 43 (16.29%) Spring
4 42 (15.91%) Spring Boot
5 26 (9.85%) Node.js
6 20 (7.58%) React
6 20 (7.58%) Spark Streaming
7 19 (7.20%) Pandas
7 19 (7.20%) PySpark
8 18 (6.82%) scikit-learn
9 17 (6.44%) .NET
10 14 (5.30%) MLlib
10 14 (5.30%) NumPy
10 14 (5.30%) SparkR
11 13 (4.92%) Flask
11 13 (4.92%) TensorFlow
12 12 (4.55%) AngularJS
12 12 (4.55%) Apache Beam
12 12 (4.55%) JSON
13 11 (4.17%) HTML
Miscellaneous
1 41 (15.53%) Algorithms
2 19 (7.20%) Data Structures
3 17 (6.44%) Greenfield Project
4 15 (5.68%) Public Cloud
5 10 (3.79%) Reference Data
6 9 (3.41%) Enterprise Cloud
7 7 (2.65%) Cloud Native
8 6 (2.27%) Private Cloud
9 4 (1.52%) Life Science
9 4 (1.52%) Self-Motivation
9 4 (1.52%) Virtual Team
10 3 (1.14%) Data Centre
11 2 (0.76%) Enterprise Software
11 2 (0.76%) Lidar
11 2 (0.76%) Replication
12 1 (0.38%) Clustering
12 1 (0.38%) Media Streaming
12 1 (0.38%) Smartphone
12 1 (0.38%) Social Media
12 1 (0.38%) Social Network
Operating Systems
1 16 (6.06%) Linux
2 10 (3.79%) Windows
3 8 (3.03%) Windows Server
4 7 (2.65%) CentOS
5 5 (1.89%) Apple iOS
6 4 (1.52%) Unix
7 1 (0.38%) Windows Server 2012
Processes & Methodologies
1 109 (41.29%) DevOps
2 105 (39.77%) Analytics
3 79 (29.92%) Software Engineering
4 76 (28.79%) Microservices
5 73 (27.65%) Agile Software Development
6 72 (27.27%) Data Science
7 69 (26.14%) Data Modelling
8 66 (25.00%) ETL
9 65 (24.62%) Data Pipeline
10 59 (22.35%) Continuous Integration
11 58 (21.97%) Business Intelligence
12 55 (20.83%) Machine Learning
13 49 (18.56%) Artificial Intelligence
14 46 (17.42%) Data Engineering
15 45 (17.05%) Fintech
16 40 (15.15%) Scaled Agile Framework
17 38 (14.39%) Computer Science
18 36 (13.64%) People Management
19 35 (13.26%) Open Source
20 31 (11.74%) Data Analytics
Programming Languages
1 158 (59.85%) Python
2 138 (52.27%) SQL
3 125 (47.35%) Java
4 72 (27.27%) Scala
5 39 (14.77%) R
6 26 (9.85%) C#
7 11 (4.17%) Go
7 11 (4.17%) JavaScript
8 8 (3.03%) C
8 8 (3.03%) MATLAB
8 8 (3.03%) TypeScript
9 7 (2.65%) Bash Shell
9 7 (2.65%) PowerShell
10 6 (2.27%) F#
11 5 (1.89%) Apple Swift
11 5 (1.89%) Shell Script
12 4 (1.52%) Clojure
12 4 (1.52%) Java 11
13 3 (1.14%) Java 8
13 3 (1.14%) Ruby
Qualifications
1 44 (16.67%) Degree
2 16 (6.06%) Computer Science Degree
3 10 (3.79%) PhD
3 10 (3.79%) SC Cleared
3 10 (3.79%) Security Cleared
4 7 (2.65%) RHCE
5 4 (1.52%) Numerate Degree
6 2 (0.76%) AWS Certification
6 2 (0.76%) Master's Degree
6 2 (0.76%) MCDBA
6 2 (0.76%) MCT
6 2 (0.76%) Microsoft Certification
7 1 (0.38%) OCP
7 1 (0.38%) Oracle Certification
7 1 (0.38%) TOGAF Certification
Quality Assurance & Compliance
1 13 (4.92%) Data Quality
2 4 (1.52%) PMO
3 2 (0.76%) QA
4 1 (0.38%) COBIT
4 1 (0.38%) GDPR
4 1 (0.38%) ISO/IEC 27001
4 1 (0.38%) NIST
System Software
1 38 (14.39%) Docker
2 9 (3.41%) Active Directory
3 8 (3.03%) VMware Infrastructure
4 7 (2.65%) vSphere
4 7 (2.65%) XenApp
4 7 (2.65%) XenDesktop
5 5 (1.89%) Apache ZooKeeper
6 4 (1.52%) HDFS
6 4 (1.52%) VirtualBox
Systems Management
1 66 (25.00%) Terraform
2 65 (24.62%) Ansible
3 50 (18.94%) Kubernetes
4 15 (5.68%) Oozie
5 14 (5.30%) Hadoop YARN
6 11 (4.17%) Puppet
7 4 (1.52%) Docker Swarm
7 4 (1.52%) Opscode Chef
8 3 (1.14%) Istio
8 3 (1.14%) Knative
9 1 (0.38%) Active Directory Federation Services
9 1 (0.38%) Control-M
9 1 (0.38%) Failover Clustering
9 1 (0.38%) FortiGate
9 1 (0.38%) Single Sign-On
Vendors
1 38 (14.39%) Microsoft
2 36 (13.64%) Databricks
3 33 (12.50%) Google
4 24 (9.09%) Cloudera
5 23 (8.71%) Oracle
6 19 (7.20%) Hortonworks
7 12 (4.55%) Informatica
7 12 (4.55%) StreamSets
8 10 (3.79%) Apple
9 9 (3.41%) MKS
9 9 (3.41%) Red Hat
10 8 (3.03%) VMware
11 7 (2.65%) F5
11 7 (2.65%) Qubit
12 4 (1.52%) Talend
13 3 (1.14%) Dataiku
13 3 (1.14%) MapR
14 2 (0.76%) Teradata
15 1 (0.38%) CheckPoint
15 1 (0.38%) Salesforce.com