Period
to 16 January 2021

The following table provides summary statistics for permanent job vacancies with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 16 January 2021 with a comparison to the same period in the previous 2 years.

Apache Spark
UK
6 months to
16 Jan 2021
Same period 2020 Same period 2019
Rank 235 240 238
Rank change year-on-year +5 -2 +27
Permanent jobs citing Apache Spark 835 1,739 2,211
As % of all permanent jobs advertised in the UK 1.34% 1.38% 1.46%
As % of the Application Platforms category 9.61% 9.38% 9.11%
Number of salaries quoted 811 1,494 1,835
Median annual salary £65,000 £72,500 £72,500
Median salary % change year-on-year -10.34% - +3.57%
10th Percentile £42,500 £47,500 £45,000
90th Percentile £103,750 £105,000 £102,500
UK excluding London median annual salary £55,000 £60,000 £63,677
% change year-on-year -8.33% -5.77% +20.83%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for application platform skills.

All Application Platform Skills
UK
Permanent vacancies with a requirement for application platform skills 8,687 18,548 24,270
As % of all permanent IT jobs advertised in the UK 13.95% 14.71% 16.06%
Number of salaries quoted 7,006 15,007 19,590
Median annual salary £55,000 £50,000 £50,000
Median salary % change year-on-year +10.00% - +5.26%
10th Percentile £31,250 £27,500 £28,250
90th Percentile £86,250 £85,000 £85,000
UK excluding London median annual salary £45,000 £45,000 £43,250
% change year-on-year - +4.05% +8.13%

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a proportion of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Salary Trend

3-month moving average salary quoted in jobs citing Apache Spark.

Salary trend for Apache Spark in the UK

Apache Spark
Salary Histogram

Salary distribution for jobs citing Apache Spark over the 6 months to 16 January 2021.

Salary histogram for Apache Spark in the UK

Apache Spark
Top 16 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the UK over the 6 months to 16 January 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England -18 726 £66,250 -8.62% 138
London +5 525 £76,250 -1.61% 95
UK excluding London +48 273 £55,000 -8.33% 44
Work from Home -2 135 £62,500 -10.71% 32
South East +96 70 £50,000 -13.04% 10
North of England +27 60 £55,000 -3.97% 17
Scotland +61 52 £70,000 +25.56% 2
Wales - 33 £45,000 - 3
North West +52 31 £55,000 -8.33% 6
Yorkshire +13 29 £55,000 - 11
South West +29 25 £53,750 +2.38% 3
East of England +73 15 £65,000 -13.33% 3
Midlands -5 15 £65,000 -10.34% 6
East Midlands -12 8 £66,250 -8.62% 1
West Midlands +55 7 £62,500 +4.17% 5
Northern Ireland +22 3 £65,000 +62.50%

For the 6 months to 16 January 2021, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Apache Spark.

1 576 (68.98%) Python
2 398 (47.66%) Big Data
3 392 (46.95%) Hadoop
4 376 (45.03%) SQL
5 371 (44.43%) AWS
6 359 (42.99%) Kafka
7 356 (42.63%) Java
8 287 (34.37%) Azure
9 277 (33.17%) Scala
10 252 (30.18%) Data Science
11 238 (28.50%) GCP
12 231 (27.66%) Apache
13 223 (26.71%) Finance
14 215 (25.75%) Analytics
15 213 (25.51%) NoSQL
16 206 (24.67%) Machine Learning
17 205 (24.55%) Agile Software Development
18 204 (24.43%) DevOps
19 197 (23.59%) Docker
20 193 (23.11%) Kubernetes
21 190 (22.75%) ETL
22 151 (18.08%) PostgreSQL
23 148 (17.72%) Microservices
24 147 (17.60%) Data Lake
25 146 (17.49%) Data Warehouse
26 132 (15.81%) Data Engineering
27 131 (15.69%) Terraform
28 129 (15.45%) Business Intelligence
29 125 (14.97%) Elasticsearch
29 125 (14.97%) Databricks

Apache Spark
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 231 (27.66%) Apache
2 125 (14.97%) Elasticsearch
3 55 (6.59%) Apache Airflow
4 26 (3.11%) Luigi
5 19 (2.28%) SAS
6 16 (1.92%) Apache Pig
7 13 (1.56%) Tornado
8 12 (1.44%) Apache Synapse
9 11 (1.32%) Jupyter
10 9 (1.08%) Tomcat
11 7 (0.84%) CMS
12 5 (0.60%) Oracle Workflow
13 4 (0.48%) Apache Solr
13 4 (0.48%) Cloud Foundry
14 2 (0.24%) Confluence
14 2 (0.24%) Oracle SOA Suite
15 1 (0.12%) BizTalk Server
15 1 (0.12%) GlassFish
Applications
1 16 (1.92%) Microsoft Excel
2 7 (0.84%) SPSS
3 3 (0.36%) SPSS Modeler
Business Applications
1 3 (0.36%) Distributed Ledger
1 3 (0.36%) Oracle Cash Management
2 1 (0.12%) Dynamics CRM
2 1 (0.12%) SAP APO
Cloud Services
1 371 (44.43%) AWS
2 287 (34.37%) Azure
3 238 (28.50%) GCP
4 74 (8.86%) GitHub
5 52 (6.23%) Power Platform
6 51 (6.11%) Stackdriver
7 50 (5.99%) Serverless
8 48 (5.75%) Amazon S3
9 47 (5.63%) Amazon Kinesis
10 40 (4.79%) Azure Data Factory
11 39 (4.67%) Amazon EMR
12 35 (4.19%) AWS Lambda
13 21 (2.51%) Cloud Computing
13 21 (2.51%) OpenShift
14 20 (2.40%) SaaS
15 18 (2.16%) Amazon ECS
16 15 (1.80%) AWS CloudFormation
17 13 (1.56%) PaaS
18 12 (1.44%) Amazon EC2
18 12 (1.44%) Amazon SQS
Communications & Networking
1 27 (3.23%) Firewall
2 26 (3.11%) HTTP
3 16 (1.92%) TCP/IP
4 13 (1.56%) Internet
4 13 (1.56%) Kerberos
5 8 (0.96%) DNS
5 8 (0.96%) NetScaler
6 6 (0.72%) Fibre Channel
6 6 (0.72%) LAN
6 6 (0.72%) SAN
6 6 (0.72%) VPN
6 6 (0.72%) WAN
7 5 (0.60%) 5G
7 5 (0.60%) DMZ
7 5 (0.60%) Parlay
8 4 (0.48%) 802.11
8 4 (0.48%) SNMP
8 4 (0.48%) Software-Defined Networking
8 4 (0.48%) SSH
8 4 (0.48%) Wireless
Database & Business Intelligence
1 398 (47.66%) Big Data
2 392 (46.95%) Hadoop
3 213 (25.51%) NoSQL
4 151 (18.08%) PostgreSQL
5 147 (17.60%) Data Lake
6 146 (17.49%) Data Warehouse
7 106 (12.69%) Apache Hive
8 105 (12.57%) Apache Cassandra
9 90 (10.78%) Amazon Redshift
10 85 (10.18%) SQL Server
11 78 (9.34%) MongoDB
12 70 (8.38%) Tableau
13 69 (8.26%) Relational Database
14 61 (7.31%) BigQuery
15 54 (6.47%) HBase
16 52 (6.23%) Power BI
17 39 (4.67%) Amazon Athena
18 36 (4.31%) Flink
19 35 (4.19%) RDBMS
20 34 (4.07%) Snowflake Schema
Development Applications
1 81 (9.70%) Git (software)
2 65 (7.78%) Jenkins
3 52 (6.23%) Gradle
4 22 (2.63%) Maven
5 21 (2.51%) TeamCity
6 18 (2.16%) JIRA
7 11 (1.32%) Cucumber
8 8 (0.96%) Sonatype Nexus
8 8 (0.96%) Visual Studio
9 7 (0.84%) Selenium
10 6 (0.72%) Atlassian Bamboo
11 5 (0.60%) Octopus Deploy
11 5 (0.60%) Vagrant
12 4 (0.48%) CA Release Automation
12 4 (0.48%) CircleCI
12 4 (0.48%) ERwin
12 4 (0.48%) GitLab
12 4 (0.48%) Oracle APEX
13 2 (0.24%) ER Studio
14 1 (0.12%) GoCD
General
1 223 (26.71%) Finance
2 80 (9.58%) Banking
3 68 (8.14%) Public Sector
4 52 (6.23%) Retail
5 42 (5.03%) Marketing
6 39 (4.67%) Telecoms
7 38 (4.55%) Front Office
8 27 (3.23%) Games
9 23 (2.75%) Advertising
10 15 (1.80%) Law
11 14 (1.68%) Manufacturing
12 8 (0.96%) Electronics
12 8 (0.96%) Investment Banking
13 5 (0.60%) Pharmaceutical
14 3 (0.36%) Retail Banking
15 2 (0.24%) Automotive
15 2 (0.24%) Back Office
15 2 (0.24%) Billing
15 2 (0.24%) Financial Institution
16 1 (0.12%) Legal
Job Titles
1 302 (36.17%) Data Engineer
2 218 (26.11%) Developer
3 77 (9.22%) Java Developer
4 75 (8.98%) Data Scientist
5 65 (7.78%) Big Data Engineer
5 65 (7.78%) Software Engineer
6 64 (7.66%) Architect
7 62 (7.43%) Senior Developer
8 53 (6.35%) Java Engineer
9 51 (6.11%) Senior Data Engineer
10 44 (5.27%) Python Engineer
11 38 (4.55%) Data Architect
12 36 (4.31%) Developer/Software Engineer
13 34 (4.07%) AWS Engineer
14 32 (3.83%) DevOps Engineer
15 30 (3.59%) .NET Developer
16 29 (3.47%) Scala Engineer
17 28 (3.35%) Consultant
17 28 (3.35%) Python Developer
18 25 (2.99%) .NET Software Engineer
Libraries, Frameworks & Software Standards
1 359 (42.99%) Kafka
2 100 (11.98%) Spring
3 72 (8.62%) RabbitMQ
4 69 (8.26%) Spring Boot
5 53 (6.35%) React
5 53 (6.35%) TensorFlow
6 51 (6.11%) Node.js
6 51 (6.11%) PySpark
6 51 (6.11%) Spring Integration
7 41 (4.91%) .NET
8 39 (4.67%) scikit-learn
8 39 (4.67%) Web Services
9 36 (4.31%) RESTful
10 34 (4.07%) Natural Language Processing
10 34 (4.07%) Pandas
11 29 (3.47%) Apache NiFi
12 28 (3.35%) .NET Core
13 26 (3.11%) NumPy
13 26 (3.11%) REST
13 26 (3.11%) Spark Streaming
Miscellaneous
1 96 (11.50%) Algorithms
2 81 (9.70%) Analytical Skills
3 31 (3.71%) Greenfield Project
4 26 (3.11%) Data Structures
5 24 (2.87%) Tandem
6 22 (2.63%) Cloud Native
7 20 (2.40%) Equities
8 18 (2.16%) Management Information System
9 17 (2.04%) Hedge funds
10 14 (1.68%) Data Centre
11 13 (1.56%) FMCG
11 13 (1.56%) Public Cloud
11 13 (1.56%) Self-Motivation
12 12 (1.44%) Market Risk
12 12 (1.44%) Video on Demand
13 11 (1.32%) Blog
13 11 (1.32%) Credit Risk
14 10 (1.20%) Enterprise Software
15 9 (1.08%) Hybrid Cloud
16 8 (0.96%) Internet of Things
Operating Systems
1 86 (10.30%) Linux
2 18 (2.16%) Ubuntu
3 17 (2.04%) Unix
4 16 (1.92%) Windows
5 15 (1.80%) CentOS
5 15 (1.80%) Red Hat Enterprise Linux
6 8 (0.96%) Windows Server
7 5 (0.60%) Oracle Linux
8 1 (0.12%) Amazon Linux
8 1 (0.12%) Android
8 1 (0.12%) Apple iOS
Processes & Methodologies
1 252 (30.18%) Data Science
2 215 (25.75%) Analytics
3 206 (24.67%) Machine Learning
4 205 (24.55%) Agile Software Development
5 204 (24.43%) DevOps
6 190 (22.75%) ETL
7 148 (17.72%) Microservices
8 132 (15.81%) Data Engineering
9 129 (15.45%) Business Intelligence
10 120 (14.37%) Software Engineering
11 105 (12.57%) Artificial Intelligence
12 100 (11.98%) Visualisation
13 96 (11.50%) Data Modelling
14 81 (9.70%) Mathematics
15 80 (9.58%) Computer Science
16 72 (8.62%) Data Integration
17 62 (7.43%) Data Analytics
18 61 (7.31%) Fintech
19 60 (7.19%) SDLC
20 59 (7.07%) Service Management
Programming Languages
1 576 (68.98%) Python
2 376 (45.03%) SQL
3 356 (42.63%) Java
4 277 (33.17%) Scala
5 106 (12.69%) JavaScript
6 96 (11.50%) R
7 86 (10.30%) C#
8 39 (4.67%) Bash Shell
9 29 (3.47%) C++
10 24 (2.87%) Go
11 18 (2.16%) TypeScript
12 17 (2.04%) Kotlin
13 16 (1.92%) C
13 16 (1.92%) PHP
14 13 (1.56%) Shell Script
15 12 (1.44%) Ruby
16 11 (1.32%) PowerShell
17 8 (0.96%) ES6
17 8 (0.96%) Java 8
17 8 (0.96%) T-SQL
Qualifications
1 104 (12.46%) Degree
2 57 (6.83%) Security Cleared
3 55 (6.59%) SC Cleared
4 35 (4.19%) Computer Science Degree
5 21 (2.51%) PhD
6 14 (1.68%) Numerate Degree
7 10 (1.20%) Postgraduate
8 8 (0.96%) RHCE
9 7 (0.84%) Master's Degree
9 7 (0.84%) Sun Certification
10 4 (0.48%) AWS Certification
11 3 (0.36%) MCDBA
11 3 (0.36%) MCT
11 3 (0.36%) Microsoft Certification
11 3 (0.36%) TOGAF Certification
12 1 (0.12%) DV Cleared
12 1 (0.12%) HND
Quality Assurance & Compliance
1 49 (5.87%) QA
2 21 (2.51%) Data Quality
3 16 (1.92%) GDPR
4 3 (0.36%) ISO 20022
4 3 (0.36%) PSD2
4 3 (0.36%) TARGET2
5 2 (0.24%) ISO 9001
5 2 (0.24%) ISO/IEC 27001
6 1 (0.12%) IFRS
6 1 (0.12%) IFRS 9
6 1 (0.12%) ISO 26262
6 1 (0.12%) MiFID
6 1 (0.12%) PMO
6 1 (0.12%) Sarbanes-Oxley
6 1 (0.12%) SLA
System Software
1 197 (23.59%) Docker
2 33 (3.95%) HDFS
3 12 (1.44%) VMware Infrastructure
4 9 (1.08%) Active Directory
5 8 (0.96%) vSphere
5 8 (0.96%) XenApp
5 8 (0.96%) XenDesktop
6 6 (0.72%) Apache Flume
7 5 (0.60%) VirtualBox
8 3 (0.36%) Apache ZooKeeper
9 2 (0.24%) Virtual Machines
10 1 (0.12%) NFS
10 1 (0.12%) oVirt
Systems Management
1 193 (23.11%) Kubernetes
2 131 (15.69%) Terraform
3 70 (8.38%) Ansible
4 51 (6.11%) Grafana
5 34 (4.07%) Puppet
6 21 (2.51%) Oozie
7 17 (2.04%) Opscode Chef
8 12 (1.44%) Hadoop YARN
9 11 (1.32%) Nagios
10 7 (0.84%) Graphite
10 7 (0.84%) Opsview
11 5 (0.60%) Docker Swarm
12 3 (0.36%) Mesos
13 2 (0.24%) Ambari
13 2 (0.24%) Kibana
13 2 (0.24%) SELinux
13 2 (0.24%) Single Sign-On
14 1 (0.12%) MIIS
14 1 (0.12%) Packer
14 1 (0.12%) Prometheus
Vendors
1 125 (14.97%) Databricks
2 118 (14.13%) Google
3 73 (8.74%) Microsoft
4 62 (7.43%) Splunk
5 59 (7.07%) Oracle
6 51 (6.11%) ServiceNow
7 34 (4.07%) Cloudera
7 34 (4.07%) Red Hat
8 27 (3.23%) Talend
9 19 (2.28%) Informatica
10 12 (1.44%) SAP
10 12 (1.44%) VMware
11 10 (1.20%) StreamSets
11 10 (1.20%) Vertica
12 8 (0.96%) F5
12 8 (0.96%) Hortonworks
12 8 (0.96%) MuleSoft
12 8 (0.96%) Teradata
13 6 (0.72%) Ab Initio
13 6 (0.72%) IBM