Period
to

The table below provides summary statistics for permanent job vacancies with an option to work from home (WFH) and a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 28 May 2022 with a comparison to the same period in the previous 2 years.

Apache Spark
UK > Work from Home
6 months to
28 May 2022
Same period 2021 Same period 2020
Rank 277 279 198
Rank change year-on-year +2 -81 -9
Permanent jobs citing Apache Spark 906 339 124
As % of all permanent jobs with a WFH option 1.24% 1.05% 1.36%
As % of the Application Platforms category 8.36% 7.40% 9.34%
Number of salaries quoted 489 299 95
10th Percentile £50,000 £47,500 £47,500
25th Percentile £60,625 £57,375 £57,500
Median annual salary (50th Percentile) £80,000 £80,000 £80,000
Median % change year-on-year - - +6.67%
75th Percentile £95,000 £92,500 £92,500
90th Percentile £108,750 £98,750 £105,000
UK median annual salary £77,500 £75,000 £75,000
% change year-on-year +3.33% - -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a WFH option and a requirement for application platform skills.

All Application Platform Skills
Work from Home
Permanent vacancies with a requirement for application platform skills 10,839 4,581 1,328
As % of all permanent jobs with a WFH option 14.79% 14.17% 14.55%
Number of salaries quoted 5,346 3,587 1,110
10th Percentile £32,500 £32,150 £35,000
25th Percentile £42,500 £41,250 £42,500
Median annual salary (50th Percentile) £57,500 £55,000 £57,500
Median % change year-on-year +4.55% -4.35% -2.95%
75th Percentile £77,500 £73,750 £73,750
90th Percentile £94,375 £87,500 £86,250
UK median annual salary £55,000 £52,500 £52,500
% change year-on-year +4.76% - +5.00%

Apache Spark
Trend for Jobs with a WFH Option

Job vacancies with a work from home option citing Apache Spark as a proportion of all IT jobs advertised.

Apache Spark trend for jobs with a WFH option

Apache Spark
Salary Trend for Jobs with a WFH Option

3-month moving average salary quoted in job vacancies with a work from home option citing Apache Spark.

Apache Spark salary trend for jobs with a WFH option

Apache Spark
Salary Histogram for Jobs with a WFH Option

Salary distribution for jobs with a work from home option citing Apache Spark over the 6 months to 28 May 2022.

Apache Spark salary histogram for jobs with a WFH option

For the 6 months to 28 May 2022, Apache Spark job roles required the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent jobs with a WFH option and a requirement for Apache Spark.

1 637 (70.31%) Python
2 461 (50.88%) SQL
3 449 (49.56%) AWS
4 411 (45.36%) Hadoop
5 410 (45.25%) Big Data
6 381 (42.05%) Kafka
7 364 (40.18%) Java
8 338 (37.31%) Data Engineering
9 337 (37.20%) Agile
10 323 (35.65%) Analytics
11 322 (35.54%) Data Pipeline
12 304 (33.55%) Scala
13 300 (33.11%) GCP
14 276 (30.46%) Data Warehouse
15 275 (30.35%) ETL
15 275 (30.35%) Azure
16 267 (29.47%) Data Science
17 236 (26.05%) Finance
18 234 (25.83%) Data Lake
19 232 (25.61%) Machine Learning
20 225 (24.83%) NoSQL
21 213 (23.51%) Data Modelling
22 206 (22.74%) Databricks
23 205 (22.63%) Social Skills
24 204 (22.52%) Docker
25 201 (22.19%) Apache
25 201 (22.19%) Apache Airflow
26 194 (21.41%) Amazon S3
27 183 (20.20%) Software Engineering
28 166 (18.32%) CI/CD

Apache Spark
Co-occurring IT Skills with a WFH Option by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 201 (22.19%) Apache
1 201 (22.19%) Apache Airflow
2 34 (3.75%) Elasticsearch
3 22 (2.43%) Jupyter
4 19 (2.10%) Oracle Workflow
5 15 (1.66%) Luigi
6 14 (1.55%) Tomcat
7 12 (1.32%) Apache Pig
8 11 (1.21%) SAS
9 6 (0.66%) Confluence
10 5 (0.55%) BizTalk Server
11 4 (0.44%) Apache Solr
12 2 (0.22%) JBoss
12 2 (0.22%) MQSeries
12 2 (0.22%) SharePoint
12 2 (0.22%) WebLogic
12 2 (0.22%) WebSphere
13 1 (0.11%) Apache Pulsar
13 1 (0.11%) CICS
Applications
1 11 (1.21%) Microsoft Office
1 11 (1.21%) Microsoft PowerPoint
2 10 (1.10%) Microsoft Excel
2 10 (1.10%) MS Visio
3 4 (0.44%) SPSS
4 2 (0.22%) Campfire
5 1 (0.11%) Chatbot
5 1 (0.11%) Gephi
Business Applications
1 4 (0.44%) Sentinel
Cloud Services
1 449 (49.56%) AWS
2 300 (33.11%) GCP
3 275 (30.35%) Azure
4 194 (21.41%) Amazon S3
5 117 (12.91%) Snowflake
6 95 (10.49%) Amazon EMR
7 73 (8.06%) AWS CloudFormation
7 73 (8.06%) Serverless
8 64 (7.06%) Amazon Kinesis
8 64 (7.06%) GitHub
9 63 (6.95%) AWS Lambda
10 58 (6.40%) Datadog
11 53 (5.85%) Cloud Computing
12 51 (5.63%) Azure Data Factory
13 39 (4.30%) SaaS
14 36 (3.97%) Power Platform
15 33 (3.64%) Cloud Composer
16 31 (3.42%) dbt
17 28 (3.09%) Azure Synapse Analytics
18 24 (2.65%) Amazon EC2
Communications & Networking
1 33 (3.64%) HTTP
2 12 (1.32%) Internet
3 10 (1.10%) Firewall
4 6 (0.66%) Network Security
5 4 (0.44%) Wireless
6 3 (0.33%) Reverse Proxy
6 3 (0.33%) Wi-Fi
7 2 (0.22%) Broadband
7 2 (0.22%) Kerberos
7 2 (0.22%) SNA
8 1 (0.11%) SAN
Database & Business Intelligence
1 411 (45.36%) Hadoop
2 410 (45.25%) Big Data
3 276 (30.46%) Data Warehouse
4 234 (25.83%) Data Lake
5 225 (24.83%) NoSQL
6 159 (17.55%) Apache Hive
7 129 (14.24%) Relational Database
8 126 (13.91%) Amazon Redshift
9 122 (13.47%) PostgreSQL
10 121 (13.36%) Flink
11 115 (12.69%) Amazon Athena
12 103 (11.37%) BigQuery
13 97 (10.71%) Metadata
14 70 (7.73%) MongoDB
15 69 (7.62%) SQL Server
16 66 (7.28%) RDBMS
17 60 (6.62%) Pentaho
18 55 (6.07%) Spark SQL
19 53 (5.85%) Apache Cassandra
20 49 (5.41%) HBase
Development Applications
1 121 (13.36%) Git (software)
2 89 (9.82%) Jenkins
3 51 (5.63%) JIRA
4 43 (4.75%) CircleCI
5 15 (1.66%) ERwin
6 14 (1.55%) ER Studio
7 13 (1.43%) Bitbucket
8 12 (1.32%) GitLab
9 6 (0.66%) Sonatype Nexus
10 4 (0.44%) Metasploit
10 4 (0.44%) PowerDesigner
10 4 (0.44%) SonarQube
10 4 (0.44%) Subversion
11 3 (0.33%) GoCD
11 3 (0.33%) Selenium
11 3 (0.33%) Spinnaker
11 3 (0.33%) Terragrunt
12 2 (0.22%) Cucumber
12 2 (0.22%) git-flow
12 2 (0.22%) SQL Server Management Studio (SSMS)
General
1 236 (26.05%) Finance
2 205 (22.63%) Social Skills
3 104 (11.48%) Analytical Skills
4 98 (10.82%) Banking
5 74 (8.17%) Advertising
6 73 (8.06%) Inclusion and Diversity
6 73 (8.06%) Retail
7 58 (6.40%) Games
8 43 (4.75%) Marketing
9 33 (3.64%) Legal
10 31 (3.42%) Public Sector
11 21 (2.32%) Investment Banking
11 21 (2.32%) Local Government
12 16 (1.77%) Presentation Skills
13 12 (1.32%) Pharmaceutical
14 9 (0.99%) Law
14 9 (0.99%) Manufacturing
15 7 (0.77%) Financial Institution
16 5 (0.55%) Humanities
16 5 (0.55%) Social Science
Job Titles
1 543 (59.93%) Data Engineer
2 149 (16.45%) Senior Data Engineer
3 73 (8.06%) Software Engineer
4 72 (7.95%) Developer
5 63 (6.95%) Java Engineer
6 56 (6.18%) AWS Engineer
7 47 (5.19%) Big Data Engineer
7 47 (5.19%) Cloud Engineer
8 41 (4.53%) Architect
9 38 (4.19%) Python Engineer
10 36 (3.97%) Data Scientist
11 33 (3.64%) Junior
12 29 (3.20%) Java Developer
12 29 (3.20%) Senior Developer
13 28 (3.09%) Consultant
13 28 (3.09%) Data Architect
13 28 (3.09%) Graduate
14 27 (2.98%) Data Consultant
15 26 (2.87%) Graduate Java Engineer
15 26 (2.87%) Junior Data Engineer
Libraries, Frameworks & Software Standards
1 381 (42.05%) Kafka
2 146 (16.11%) PySpark
3 89 (9.82%) Spark Streaming
4 57 (6.29%) TensorFlow
5 48 (5.30%) Apache Beam
6 40 (4.42%) RabbitMQ
6 40 (4.42%) SQLAlchemy
7 39 (4.30%) OAuth
8 34 (3.75%) .NET
8 34 (3.75%) RESTful
9 33 (3.64%) Apache Avro
10 32 (3.53%) Pandas
11 31 (3.42%) Elastic Stack
12 28 (3.09%) Web Services
13 23 (2.54%) Keras
14 21 (2.32%) scikit-learn
15 20 (2.21%) Hibernate
16 19 (2.10%) AngularJS
16 19 (2.10%) H2O
16 19 (2.10%) Spring
Miscellaneous
1 112 (12.36%) Algorithms
2 81 (8.94%) Data Structures
3 52 (5.74%) Greenfield Project
4 50 (5.52%) Cloud Native
5 43 (4.75%) IoT
6 38 (4.19%) Public Cloud
7 31 (3.42%) Management Information System
8 30 (3.31%) Sports Betting
9 19 (2.10%) Legacy Systems
10 17 (1.88%) Data Centre
11 16 (1.77%) User Experience
12 15 (1.66%) Reinsurance
13 14 (1.55%) Cyberattack
14 13 (1.43%) Cyber Defence
15 10 (1.10%) Blockchain
16 9 (0.99%) Self-Motivation
17 8 (0.88%) Mobile App
18 7 (0.77%) Blog
18 7 (0.77%) NHS
19 6 (0.66%) Digital Media
Operating Systems
1 65 (7.17%) Linux
2 24 (2.65%) Windows
3 22 (2.43%) CentOS
4 11 (1.21%) Unix
5 10 (1.10%) Ubuntu
6 4 (0.44%) Kali Linux
7 3 (0.33%) Android
8 2 (0.22%) Apple iOS
9 1 (0.11%) Mac OS X
9 1 (0.11%) Windows Server
9 1 (0.11%) zOS
Processes & Methodologies
1 338 (37.31%) Data Engineering
2 337 (37.20%) Agile
3 323 (35.65%) Analytics
4 322 (35.54%) Data Pipeline
5 275 (30.35%) ETL
6 267 (29.47%) Data Science
7 232 (25.61%) Machine Learning
8 213 (23.51%) Data Modelling
9 183 (20.20%) Software Engineering
10 166 (18.32%) CI/CD
11 148 (16.34%) Problem-Solving
12 145 (16.00%) Data Analysis
13 136 (15.01%) SDLC
14 129 (14.24%) Computer Science
15 128 (14.13%) Business Intelligence
16 124 (13.69%) DevOps
17 114 (12.58%) Data Architecture
18 105 (11.59%) Artificial Intelligence
19 103 (11.37%) Data Management
20 99 (10.93%) Visualisation
Programming Languages
1 637 (70.31%) Python
2 461 (50.88%) SQL
3 364 (40.18%) Java
4 304 (33.55%) Scala
5 78 (8.61%) R
6 50 (5.52%) C#
7 38 (4.19%) JavaScript
8 27 (2.98%) Bash
9 23 (2.54%) TypeScript
10 21 (2.32%) T-SQL
11 20 (2.21%) Kotlin
12 15 (1.66%) Clojure
12 15 (1.66%) Go
13 14 (1.55%) C++
14 7 (0.77%) C
14 7 (0.77%) HiveQL
14 7 (0.77%) Ruby
15 6 (0.66%) PowerShell
16 5 (0.55%) Groovy
17 4 (0.44%) Shell Script
Qualifications
1 84 (9.27%) Degree
2 32 (3.53%) Computer Science Degree
3 22 (2.43%) Master's Degree
4 12 (1.32%) RHCE
5 10 (1.10%) PhD
6 6 (0.66%) Security Cleared
7 5 (0.55%) SC Cleared
8 2 (0.22%) AWS Certification
9 1 (0.11%) BPSS Clearance
9 1 (0.11%) CBAP
9 1 (0.11%) Numerate Degree
9 1 (0.11%) Tableau Certification
Quality Assurance & Compliance
1 79 (8.72%) Data Quality
2 50 (5.52%) QA
3 7 (0.77%) GDPR
4 4 (0.44%) Cyber Essentials
4 4 (0.44%) Cyber Essentials PLUS
4 4 (0.44%) ISO/IEC 27001
4 4 (0.44%) NIST
4 4 (0.44%) PCI DSS
5 1 (0.11%) Accessibility
5 1 (0.11%) SLA
System Software
1 204 (22.52%) Docker
2 50 (5.52%) Apache Flume
3 29 (3.20%) HDFS
4 22 (2.43%) VMware Infrastructure
5 21 (2.32%) Apache ZooKeeper
6 12 (1.32%) Active Directory
6 12 (1.32%) GlusterFS
6 12 (1.32%) vSphere
7 3 (0.33%) Ceph
7 3 (0.33%) NFS
8 1 (0.11%) Virtual Machines
Systems Management
1 158 (17.44%) Kubernetes
2 151 (16.67%) Terraform
3 65 (7.17%) Oozie
4 46 (5.08%) Opscode Chef
5 30 (3.31%) Ansible
6 27 (2.98%) Consul
6 27 (2.98%) Hadoop YARN
7 19 (2.10%) Puppet
8 15 (1.66%) Prometheus
9 11 (1.21%) Grafana
10 10 (1.10%) Kibana
11 4 (0.44%) Control-M
11 4 (0.44%) Mesos
11 4 (0.44%) Rundeck
11 4 (0.44%) SCCM
12 3 (0.33%) Graphite
12 3 (0.33%) Nagios
12 3 (0.33%) Zipkin
13 1 (0.11%) OMEGAMON
Vendors
1 206 (22.74%) Databricks
2 122 (13.47%) Google
3 112 (12.36%) Microsoft
4 78 (8.61%) Informatica
5 65 (7.17%) Talend
6 61 (6.73%) Cloudera
7 51 (5.63%) Oracle
8 27 (2.98%) Marin
9 24 (2.65%) Splunk
10 22 (2.43%) Red Hat
10 22 (2.43%) VMware
11 21 (2.32%) Salesforce.com
12 19 (2.10%) Hitachi
13 13 (1.43%) Apple
14 12 (1.32%) F5
14 12 (1.32%) IBM
15 8 (0.88%) Teradata
16 7 (0.77%) Cisco
17 5 (0.55%) Hortonworks
17 5 (0.55%) MuleSoft