Period
to 28 July 2021

The table below provides summary statistics for permanent job vacancies with an option to work from home (WFH) and a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 28 July 2021 with a comparison to the same period in the previous 2 years.

Apache Spark
UK > Work from Home
6 months to
28 Jul 2021
Same period 2020 Same period 2019
Rank 259 182 197
Rank change year-on-year -77 +15 +18
Permanent jobs citing Apache Spark 248 56 114
As % of all permanent jobs with a WFH option 1.09% 1.04% 1.40%
As % of the Application Platforms category 7.12% 7.24% 9.55%
Number of salaries quoted 205 37 105
10th Percentile £52,500 £45,000 £47,500
25th Percentile £65,000 £48,750 £55,000
Median annual salary (50th Percentile) £80,000 £75,000 £75,000
Median % change year-on-year +6.67% - -
75th Percentile £96,250 £97,500 £100,000
90th Percentile £102,500 £105,000 £147,500
UK median annual salary £75,000 £75,000 £75,000

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a WFH option and a requirement for application platform skills.

All Application Platform Skills
Work from Home
Permanent vacancies with a requirement for application platform skills 3,484 773 1,194
As % of all permanent jobs with a WFH option 15.26% 14.30% 14.69%
Number of salaries quoted 2,542 669 1,022
10th Percentile £33,525 £35,000 £33,750
25th Percentile £42,000 £42,500 £42,500
Median annual salary (50th Percentile) £55,000 £57,500 £60,000
Median % change year-on-year -4.35% -4.17% +20.00%
75th Percentile £75,000 £71,250 £80,000
90th Percentile £92,500 £83,750 £90,000
UK median annual salary £54,250 £52,500 £50,000
% change year-on-year +3.33% +5.00% -

Apache Spark
Trend for Jobs with a WFH Option

Job vacancies with a work from home option citing Apache Spark as a proportion of all IT jobs advertised.

Apache Spark trend for jobs with a WFH option

Apache Spark
Salary Trend for Jobs with a WFH Option

3-month moving average salary quoted in job vacancies with a work from home option citing Apache Spark.

Apache Spark salary trend for jobs with a WFH option

Apache Spark
Salary Histogram for Jobs with a WFH Option

Salary distribution for jobs with a work from home option citing Apache Spark over the 6 months to 28 July 2021.

Apache Spark salary histogram for jobs with a WFH option

For the 6 months to 28 July 2021, Apache Spark job roles required the following IT skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent jobs with a WFH option and a requirement for Apache Spark.

1 150 (60.48%) SQL
2 142 (57.26%) Python
3 141 (56.85%) Kafka
4 124 (50.00%) Hadoop
5 119 (47.98%) Java
6 112 (45.16%) Big Data
7 111 (44.76%) Software Engineering
8 107 (43.15%) AWS
9 103 (41.53%) DevOps
10 98 (39.52%) Azure
11 96 (38.71%) Agile Software Development
12 92 (37.10%) Kubernetes
13 79 (31.85%) Analytics
14 77 (31.05%) Microservices
14 77 (31.05%) Data Pipeline
15 75 (30.24%) Docker
16 71 (28.63%) Fintech
17 70 (28.23%) Terraform
18 61 (24.60%) Data Science
19 58 (23.39%) Jenkins
19 58 (23.39%) Apache Hive
19 58 (23.39%) NoSQL
19 58 (23.39%) Social Skills
20 56 (22.58%) PostgreSQL
21 53 (21.37%) Data Modelling
22 52 (20.97%) ETL
23 51 (20.56%) Data Engineering
24 50 (20.16%) GCP
24 50 (20.16%) Continuous Integration
25 49 (19.76%) Machine Learning

Apache Spark
Co-occurring IT Skills with a WFH Option by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 48 (19.35%) Apache
2 21 (8.47%) Apache Airflow
3 14 (5.65%) Elasticsearch
4 9 (3.63%) Apache Pig
5 5 (2.02%) Luigi
6 3 (1.21%) Jupyter
6 3 (1.21%) Tomcat
7 2 (0.81%) Apache Solr
Cloud Services
1 107 (43.15%) AWS
2 98 (39.52%) Azure
3 50 (20.16%) GCP
4 29 (11.69%) SaaS
5 28 (11.29%) Amazon EMR
6 22 (8.87%) Serverless
7 21 (8.47%) Azure Data Factory
8 19 (7.66%) Power Platform
9 15 (6.05%) PaaS
10 14 (5.65%) Snowflake
11 10 (4.03%) Amazon Kinesis
11 10 (4.03%) Azure Logic Apps
11 10 (4.03%) Azure Sentinel
12 8 (3.23%) Azure Functions
12 8 (3.23%) GitHub
13 7 (2.82%) Amazon SageMaker
13 7 (2.82%) AWS CloudFormation
13 7 (2.82%) AWS Lambda
13 7 (2.82%) Azure Cosmos DB
14 6 (2.42%) Google Data Studio
Communications & Networking
1 3 (1.21%) NetScaler
2 1 (0.40%) Cisco ASA
2 1 (0.40%) Firewall
Database & Business Intelligence
1 124 (50.00%) Hadoop
2 112 (45.16%) Big Data
3 58 (23.39%) Apache Hive
3 58 (23.39%) NoSQL
4 56 (22.58%) PostgreSQL
5 47 (18.95%) MongoDB
6 40 (16.13%) Data Warehouse
7 37 (14.92%) Amazon Redshift
8 28 (11.29%) Data Lake
9 27 (10.89%) Redis
10 25 (10.08%) BigQuery
11 23 (9.27%) SQL Server
12 21 (8.47%) Apache Cassandra
13 17 (6.85%) Power BI
14 15 (6.05%) Neo4j
15 14 (5.65%) Bigtable
15 14 (5.65%) Relational Database
16 13 (5.24%) Metadata
16 13 (5.24%) MySQL
17 12 (4.84%) HBase
Development Applications
1 58 (23.39%) Jenkins
2 38 (15.32%) Git (software)
3 7 (2.82%) Atlassian Bamboo
3 7 (2.82%) JIRA
4 6 (2.42%) Spinnaker
4 6 (2.42%) Vagrant
5 5 (2.02%) Visual Studio
6 4 (1.61%) Team Foundation Server
7 3 (1.21%) Bitbucket
7 3 (1.21%) Postman
7 3 (1.21%) REST Assured
7 3 (1.21%) Selenium
7 3 (1.21%) TeamCity
7 3 (1.21%) TestNG
7 3 (1.21%) WebDriver
8 2 (0.81%) Cucumber
8 2 (0.81%) Gradle
8 2 (0.81%) Maven
8 2 (0.81%) SonarQube
9 1 (0.40%) kappa
General
1 58 (23.39%) Social Skills
2 31 (12.50%) Finance
2 31 (12.50%) Retail
3 25 (10.08%) Games
4 20 (8.06%) Analytical Skills
5 15 (6.05%) Marketing
6 14 (5.65%) Banking
7 12 (4.84%) Advertising
7 12 (4.84%) Czech Language
8 10 (4.03%) Presentation Skills
9 9 (3.63%) Inclusion and Diversity
10 6 (2.42%) Influencing Skills
11 5 (2.02%) Pharmaceutical
12 4 (1.61%) Law
12 4 (1.61%) Retail Banking
13 2 (0.81%) Front Office
13 2 (0.81%) Legal
13 2 (0.81%) Public Sector
14 1 (0.40%) Investment Banking
14 1 (0.40%) Telecoms
Job Titles
1 79 (31.85%) Developer
2 78 (31.45%) Data Engineer
3 47 (18.95%) Java Developer
4 44 (17.74%) Software Engineer
5 29 (11.69%) Architect
6 24 (9.68%) Senior Data Engineer
7 22 (8.87%) Principal Developer
7 22 (8.87%) Principal Java Developer
7 22 (8.87%) Python Developer
8 21 (8.47%) Data Architect
8 21 (8.47%) Full Stack Developer
9 20 (8.06%) AWS Engineer
9 20 (8.06%) Azure Engineer
9 20 (8.06%) JavaScript Developer
10 18 (7.26%) Python Engineer
11 15 (6.05%) Cloud Engineer
12 13 (5.24%) Java Engineer
13 12 (4.84%) Data Scientist
13 12 (4.84%) Senior Software Engineer
14 11 (4.44%) Cloud Architect
Libraries, Frameworks & Software Standards
1 141 (56.85%) Kafka
2 44 (17.74%) Spring
2 44 (17.74%) Spring Boot
3 41 (16.53%) Node.js
4 40 (16.13%) RabbitMQ
5 33 (13.31%) AngularJS
5 33 (13.31%) React
6 28 (11.29%) REST
7 25 (10.08%) KnockoutJS
8 21 (8.47%) Apache Beam
8 21 (8.47%) Vue.js
9 20 (8.06%) Spark Streaming
10 17 (6.85%) .NET
11 13 (5.24%) GraphQL
12 11 (4.44%) HTML
12 11 (4.44%) Pandas
12 11 (4.44%) PySpark
12 11 (4.44%) scikit-learn
13 8 (3.23%) CSS
13 8 (3.23%) Flask
Miscellaneous
1 19 (7.66%) Algorithms
2 15 (6.05%) Data Structures
3 8 (3.23%) Enterprise Cloud
3 8 (3.23%) Reference Data
4 5 (2.02%) Greenfield Project
4 5 (2.02%) Public Cloud
5 4 (1.61%) Virtual Team
6 3 (1.21%) Cloud Native
7 2 (0.81%) Lidar
7 2 (0.81%) Private Cloud
7 2 (0.81%) Replication
8 1 (0.40%) Clustering
8 1 (0.40%) Hedge funds
8 1 (0.40%) Legacy Applications
8 1 (0.40%) Social Media
8 1 (0.40%) Video on Demand
Operating Systems
1 12 (4.84%) Linux
2 5 (2.02%) Apple iOS
3 4 (1.61%) Windows
3 4 (1.61%) Windows Server
4 3 (1.21%) CentOS
5 1 (0.40%) Windows Server 2012
Processes & Methodologies
1 111 (44.76%) Software Engineering
2 103 (41.53%) DevOps
3 96 (38.71%) Agile Software Development
4 79 (31.85%) Analytics
5 77 (31.05%) Data Pipeline
5 77 (31.05%) Microservices
6 71 (28.63%) Fintech
7 61 (24.60%) Data Science
8 53 (21.37%) Data Modelling
9 52 (20.97%) ETL
10 51 (20.56%) Data Engineering
11 50 (20.16%) Continuous Integration
12 49 (19.76%) Machine Learning
13 37 (14.92%) Business Intelligence
14 30 (12.10%) Computer Science
15 29 (11.69%) Open Source
15 29 (11.69%) People Management
15 29 (11.69%) Scaled Agile Framework
16 27 (10.89%) Data Analytics
17 25 (10.08%) Full Stack Development
Programming Languages
1 150 (60.48%) SQL
2 142 (57.26%) Python
3 119 (47.98%) Java
4 45 (18.15%) C#
4 45 (18.15%) Scala
5 39 (15.73%) JavaScript
6 21 (8.47%) Go
7 16 (6.45%) Java 11
8 15 (6.05%) R
9 13 (5.24%) Java 8
10 7 (2.82%) PowerShell
11 5 (2.02%) Apple Swift
12 4 (1.61%) C++
12 4 (1.61%) Shell Script
12 4 (1.61%) TypeScript
13 3 (1.21%) C
13 3 (1.21%) Clojure
14 2 (0.81%) MATLAB
14 2 (0.81%) T-SQL
14 2 (0.81%) U-SQL
Qualifications
1 32 (12.90%) Degree
2 13 (5.24%) Computer Science Degree
3 9 (3.63%) SC Cleared
3 9 (3.63%) Security Cleared
4 4 (1.61%) PhD
5 3 (1.21%) Numerate Degree
5 3 (1.21%) RHCE
6 2 (0.81%) Master's Degree
6 2 (0.81%) MCDBA
6 2 (0.81%) MCT
6 2 (0.81%) Microsoft Certification
7 1 (0.40%) AWS Certification
7 1 (0.40%) OCP
7 1 (0.40%) Oracle Certification
Quality Assurance & Compliance
1 13 (5.24%) Data Quality
2 4 (1.61%) QA
3 2 (0.81%) COBIT
3 2 (0.81%) ISO/IEC 27001
3 2 (0.81%) NIST
System Software
1 75 (30.24%) Docker
2 6 (2.42%) VirtualBox
3 5 (2.02%) Active Directory
3 5 (2.02%) Apache ZooKeeper
4 4 (1.61%) Apache Flume
4 4 (1.61%) HDFS
4 4 (1.61%) VMware Infrastructure
5 3 (1.21%) vSphere
5 3 (1.21%) XenApp
5 3 (1.21%) XenDesktop
Systems Management
1 92 (37.10%) Kubernetes
2 70 (28.23%) Terraform
3 49 (19.76%) Ansible
4 11 (4.44%) Hadoop YARN
5 9 (3.63%) Puppet
6 6 (2.42%) Docker Swarm
7 4 (1.61%) Oozie
8 1 (0.40%) Active Directory Federation Services
8 1 (0.40%) Control-M
8 1 (0.40%) Failover Clustering
8 1 (0.40%) FortiGate
8 1 (0.40%) Opscode Chef
8 1 (0.40%) Single Sign-On
Vendors
1 35 (14.11%) Google
2 32 (12.90%) Databricks
3 28 (11.29%) Microsoft
4 19 (7.66%) Cloudera
5 13 (5.24%) Qubit
6 12 (4.84%) Hortonworks
7 9 (3.63%) Talend
8 8 (3.23%) MKS
9 7 (2.82%) Informatica
10 6 (2.42%) Apple
11 5 (2.02%) Oracle
12 4 (1.61%) IBM
12 4 (1.61%) VMware
13 3 (1.21%) F5
13 3 (1.21%) Red Hat
14 2 (0.81%) Dataiku
14 2 (0.81%) Teradata
15 1 (0.40%) CheckPoint
15 1 (0.40%) ESRI
15 1 (0.40%) StreamSets