Period
to 18 January 2021

The following table provides summary statistics for permanent job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 18 January 2021 with a comparison to the same period in the previous 2 years.

Apache Spark
UK > England
6 months to
18 Jan 2021
Same period 2020 Same period 2019
Rank 247 232 237
Rank change year-on-year -15 +5 +24
Permanent jobs citing Apache Spark 726 1,674 2,086
As % of all permanent jobs advertised in England 1.28% 1.44% 1.48%
As % of the Application Platforms category 9.16% 9.93% 9.24%
Number of salaries quoted 698 1,453 1,716
Median annual salary £67,500 £72,500 £75,000
Median salary % change year-on-year -6.90% -3.33% +7.14%
10th Percentile £43,375 £47,500 £47,500
90th Percentile £103,750 £105,000 £105,000
UK median annual salary £65,000 £72,500 £72,500
% change year-on-year -10.34% - +3.57%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Permanent vacancies with a requirement for application platform skills 7,928 16,856 22,576
As % of all permanent IT jobs advertised in England 14.03% 14.54% 16.03%
Number of salaries quoted 6,454 13,704 18,374
Median annual salary £55,000 £50,000 £50,000
Median salary % change year-on-year +10.00% - +3.09%
10th Percentile £31,250 £27,500 £28,020
90th Percentile £87,500 £85,000 £85,000
UK median annual salary £55,000 £50,000 £50,000
% change year-on-year +10.00% - +5.26%

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a proportion of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Salary Trend in England

3-month moving average salary quoted in jobs citing Apache Spark in England.

Salary trend for Apache Spark in England

Apache Spark
Salary Histogram in England

Salary distribution for jobs citing Apache Spark in England over the 6 months to 18 January 2021.

Salary histogram for Apache Spark in England

Apache Spark
Job Locations in England

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the England region over the 6 months to 18 January 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
London +3 525 £76,250 -1.61% 104
South East +100 70 £50,000 -9.09% 13
North of England +17 59 £55,000 -3.97% 17
North West +58 30 £55,000 -8.33% 5
Yorkshire +27 29 £55,000 - 12
South West +27 25 £53,750 - 4
East of England +74 15 £65,000 -13.33% 4
Midlands +15 15 £65,000 -10.34% 7
East Midlands -8 8 £66,250 -8.62% 1
West Midlands +53 7 £62,500 +4.17% 6
Apache Spark
UK

For the 6 months to 18 January 2021, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the England region with a requirement for Apache Spark.

1 510 (70.25%) Python
2 357 (49.17%) Big Data
3 342 (47.11%) Hadoop
4 341 (46.97%) AWS
5 335 (46.14%) SQL
6 319 (43.94%) Java
7 312 (42.98%) Kafka
8 255 (35.12%) Azure
9 231 (31.82%) Scala
10 210 (28.93%) Data Science
11 205 (28.24%) Finance
12 200 (27.55%) Apache
13 196 (27.00%) NoSQL
13 196 (27.00%) GCP
14 195 (26.86%) Analytics
15 191 (26.31%) Agile Software Development
16 189 (26.03%) Machine Learning
17 184 (25.34%) Docker
18 175 (24.10%) ETL
19 174 (23.97%) Kubernetes
20 166 (22.87%) DevOps
21 136 (18.73%) Data Lake
22 131 (18.04%) Microservices
23 125 (17.22%) Data Warehouse
24 120 (16.53%) Databricks
25 113 (15.56%) Terraform
25 113 (15.56%) Data Engineering
26 112 (15.43%) PostgreSQL
27 111 (15.29%) Software Engineering
28 108 (14.88%) Business Intelligence

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 200 (27.55%) Apache
2 95 (13.09%) Elasticsearch
3 54 (7.44%) Apache Airflow
4 24 (3.31%) Luigi
5 18 (2.48%) SAS
6 16 (2.20%) Apache Pig
7 13 (1.79%) Tornado
8 12 (1.65%) Apache Synapse
9 11 (1.52%) Jupyter
10 7 (0.96%) CMS
11 5 (0.69%) Oracle Workflow
12 3 (0.41%) Cloud Foundry
13 2 (0.28%) Confluence
13 2 (0.28%) Oracle SOA Suite
14 1 (0.14%) ArcGIS
14 1 (0.14%) BizTalk Server
14 1 (0.14%) GlassFish
14 1 (0.14%) Tomcat
Applications
1 16 (2.20%) Microsoft Excel
2 7 (0.96%) SPSS
3 3 (0.41%) SPSS Modeler
Business Applications
1 3 (0.41%) Distributed Ledger
1 3 (0.41%) Oracle Cash Management
2 1 (0.14%) Dynamics CRM
2 1 (0.14%) SAP APO
Cloud Services
1 341 (46.97%) AWS
2 255 (35.12%) Azure
3 196 (27.00%) GCP
4 57 (7.85%) GitHub
5 48 (6.61%) Serverless
6 46 (6.34%) Power Platform
7 44 (6.06%) Amazon Kinesis
8 42 (5.79%) Amazon S3
9 38 (5.23%) Azure Data Factory
10 35 (4.82%) Amazon EMR
11 34 (4.68%) Stackdriver
12 32 (4.41%) AWS Lambda
13 18 (2.48%) Cloud Computing
14 17 (2.34%) Amazon ECS
14 17 (2.34%) SaaS
15 15 (2.07%) AWS CloudFormation
16 13 (1.79%) Amazon EC2
16 13 (1.79%) OpenShift
17 11 (1.52%) Amazon SQS
18 10 (1.38%) PaaS
Communications & Networking
1 24 (3.31%) Firewall
2 20 (2.75%) HTTP
3 16 (2.20%) TCP/IP
4 13 (1.79%) Internet
5 11 (1.52%) Kerberos
6 7 (0.96%) DNS
7 5 (0.69%) Parlay
8 4 (0.55%) 5G
8 4 (0.55%) 802.11
8 4 (0.55%) Fibre Channel
8 4 (0.55%) FTP
8 4 (0.55%) LAN
8 4 (0.55%) SAN
8 4 (0.55%) SNMP
8 4 (0.55%) SSH
8 4 (0.55%) VPN
8 4 (0.55%) WAN
8 4 (0.55%) Wireless
9 3 (0.41%) Reverse Proxy
9 3 (0.41%) Software-Defined Networking
Database & Business Intelligence
1 357 (49.17%) Big Data
2 342 (47.11%) Hadoop
3 196 (27.00%) NoSQL
4 136 (18.73%) Data Lake
5 125 (17.22%) Data Warehouse
6 112 (15.43%) PostgreSQL
7 100 (13.77%) Apache Cassandra
8 97 (13.36%) Apache Hive
9 85 (11.71%) Amazon Redshift
10 78 (10.74%) SQL Server
11 70 (9.64%) MongoDB
12 61 (8.40%) Relational Database
13 57 (7.85%) Tableau
14 46 (6.34%) HBase
14 46 (6.34%) Power BI
15 43 (5.92%) BigQuery
16 40 (5.51%) Amazon Athena
17 33 (4.55%) Flink
18 32 (4.41%) Snowflake Schema
19 30 (4.13%) MySQL
Development Applications
1 71 (9.78%) Git (software)
2 59 (8.13%) Jenkins
3 35 (4.82%) Gradle
4 20 (2.75%) JIRA
4 20 (2.75%) Maven
5 19 (2.62%) TeamCity
6 11 (1.52%) Cucumber
7 9 (1.24%) Visual Studio
8 7 (0.96%) Atlassian Bamboo
8 7 (0.96%) Selenium
8 7 (0.96%) Sonatype Nexus
9 6 (0.83%) Vagrant
10 5 (0.69%) Octopus Deploy
11 4 (0.55%) CircleCI
11 4 (0.55%) GitLab
12 2 (0.28%) CA Release Automation
12 2 (0.28%) ER Studio
12 2 (0.28%) ERwin
12 2 (0.28%) Oracle APEX
13 1 (0.14%) Oracle Forms
General
1 205 (28.24%) Finance
2 74 (10.19%) Banking
3 52 (7.16%) Retail
4 51 (7.02%) Public Sector
5 42 (5.79%) Marketing
6 38 (5.23%) Telecoms
7 37 (5.10%) Front Office
8 27 (3.72%) Games
9 15 (2.07%) Law
10 14 (1.93%) Manufacturing
11 12 (1.65%) Advertising
12 9 (1.24%) Investment Banking
13 8 (1.10%) Electronics
14 5 (0.69%) Pharmaceutical
15 3 (0.41%) Retail Banking
16 2 (0.28%) Automotive
16 2 (0.28%) Back Office
16 2 (0.28%) Billing
16 2 (0.28%) Financial Institution
17 1 (0.14%) Publishing
Job Titles
1 271 (37.33%) Data Engineer
2 186 (25.62%) Developer
3 73 (10.06%) Data Scientist
4 63 (8.68%) Java Developer
5 62 (8.54%) Software Engineer
6 58 (7.99%) Big Data Engineer
7 54 (7.44%) Architect
8 52 (7.16%) Java Engineer
8 52 (7.16%) Senior Data Engineer
9 45 (6.20%) Senior Developer
10 42 (5.79%) Python Engineer
11 36 (4.96%) Developer/Software Engineer
12 34 (4.68%) AWS Engineer
13 32 (4.41%) Data Architect
14 31 (4.27%) DevOps Engineer
15 29 (3.99%) Scala Engineer
16 28 (3.86%) .NET Developer
17 27 (3.72%) Python Developer
18 25 (3.44%) .NET Software Developer
18 25 (3.44%) .NET Software Engineer
Libraries, Frameworks & Software Standards
1 312 (42.98%) Kafka
2 76 (10.47%) Spring
3 58 (7.99%) Spring Boot
4 55 (7.58%) RabbitMQ
5 52 (7.16%) React
5 52 (7.16%) TensorFlow
6 48 (6.61%) Node.js
6 48 (6.61%) PySpark
7 37 (5.10%) .NET
7 37 (5.10%) Web Services
8 36 (4.96%) scikit-learn
9 34 (4.68%) Spring Integration
10 32 (4.41%) Natural Language Processing
10 32 (4.41%) RESTful
11 31 (4.27%) Pandas
12 30 (4.13%) Apache NiFi
13 27 (3.72%) .NET Core
14 25 (3.44%) Spark Streaming
15 23 (3.17%) NumPy
16 21 (2.89%) REST
Miscellaneous
1 87 (11.98%) Algorithms
2 79 (10.88%) Analytical Skills
3 30 (4.13%) Greenfield Project
4 24 (3.31%) Data Structures
5 22 (3.03%) Cloud Native
6 20 (2.75%) Equities
7 18 (2.48%) Hedge funds
8 17 (2.34%) Management Information System
9 13 (1.79%) Public Cloud
9 13 (1.79%) Self-Motivation
10 12 (1.65%) Blog
10 12 (1.65%) FMCG
10 12 (1.65%) Market Risk
10 12 (1.65%) Video on Demand
11 11 (1.52%) Credit Risk
12 9 (1.24%) Enterprise Software
12 9 (1.24%) Hybrid Cloud
13 8 (1.10%) Internet of Things
14 7 (0.96%) Mobile App
14 7 (0.96%) User Experience
Operating Systems
1 70 (9.64%) Linux
2 16 (2.20%) Ubuntu
3 15 (2.07%) Red Hat Enterprise Linux
4 11 (1.52%) Unix
5 7 (0.96%) Windows
6 5 (0.69%) CentOS
7 3 (0.41%) Oracle Linux
8 1 (0.14%) Amazon Linux
8 1 (0.14%) Android
8 1 (0.14%) Apple iOS
Processes & Methodologies
1 210 (28.93%) Data Science
2 195 (26.86%) Analytics
3 191 (26.31%) Agile Software Development
4 189 (26.03%) Machine Learning
5 175 (24.10%) ETL
6 166 (22.87%) DevOps
7 131 (18.04%) Microservices
8 113 (15.56%) Data Engineering
9 111 (15.29%) Software Engineering
10 108 (14.88%) Business Intelligence
11 92 (12.67%) Artificial Intelligence
12 89 (12.26%) Data Modelling
13 88 (12.12%) Visualisation
14 76 (10.47%) Computer Science
15 75 (10.33%) Mathematics
16 61 (8.40%) Data Integration
16 61 (8.40%) Fintech
17 55 (7.58%) SDLC
18 52 (7.16%) Data Analytics
19 51 (7.02%) Open Source
Programming Languages
1 510 (70.25%) Python
2 335 (46.14%) SQL
3 319 (43.94%) Java
4 231 (31.82%) Scala
5 98 (13.50%) JavaScript
6 82 (11.29%) C#
7 78 (10.74%) R
8 40 (5.51%) Bash Shell
9 27 (3.72%) C++
10 22 (3.03%) Go
11 17 (2.34%) C
11 17 (2.34%) Kotlin
12 16 (2.20%) PHP
12 16 (2.20%) TypeScript
13 11 (1.52%) Ruby
14 8 (1.10%) ES6
14 8 (1.10%) PowerShell
14 8 (1.10%) Shell Script
14 8 (1.10%) T-SQL
15 7 (0.96%) Julia
Qualifications
1 95 (13.09%) Degree
2 41 (5.65%) Security Cleared
3 39 (5.37%) SC Cleared
4 32 (4.41%) Computer Science Degree
5 20 (2.75%) PhD
6 12 (1.65%) Numerate Degree
7 10 (1.38%) Postgraduate
8 7 (0.96%) Sun Certification
9 6 (0.83%) Master's Degree
10 3 (0.41%) AWS Certification
10 3 (0.41%) TOGAF Certification
11 1 (0.14%) DV Cleared
11 1 (0.14%) HND
Quality Assurance & Compliance
1 45 (6.20%) QA
2 15 (2.07%) GDPR
3 13 (1.79%) Data Quality
4 3 (0.41%) ISO 20022
4 3 (0.41%) PSD2
4 3 (0.41%) TARGET2
5 2 (0.28%) ISO 9001
5 2 (0.28%) ISO/IEC 27001
6 1 (0.14%) IFRS
6 1 (0.14%) IFRS 9
6 1 (0.14%) ISO 26262
6 1 (0.14%) MiFID
6 1 (0.14%) PMO
6 1 (0.14%) Sarbanes-Oxley
6 1 (0.14%) SLA
System Software
1 184 (25.34%) Docker
2 30 (4.13%) HDFS
3 6 (0.83%) Apache Flume
3 6 (0.83%) VirtualBox
4 4 (0.55%) VMware Infrastructure
5 3 (0.41%) Apache ZooKeeper
6 2 (0.28%) Virtual Machines
7 1 (0.14%) Active Directory
7 1 (0.14%) NFS
7 1 (0.14%) oVirt
Systems Management
1 174 (23.97%) Kubernetes
2 113 (15.56%) Terraform
3 61 (8.40%) Ansible
4 35 (4.82%) Grafana
5 24 (3.31%) Puppet
6 20 (2.75%) Oozie
7 15 (2.07%) Opscode Chef
8 12 (1.65%) Hadoop YARN
9 11 (1.52%) Nagios
10 7 (0.96%) Graphite
10 7 (0.96%) Opsview
11 6 (0.83%) Docker Swarm
12 3 (0.41%) Mesos
13 2 (0.28%) Ambari
13 2 (0.28%) Kibana
13 2 (0.28%) Prometheus
13 2 (0.28%) SELinux
13 2 (0.28%) Single Sign-On
14 1 (0.14%) MIIS
14 1 (0.14%) Packer
Vendors
1 120 (16.53%) Databricks
2 100 (13.77%) Google
3 73 (10.06%) Microsoft
4 47 (6.47%) Oracle
5 42 (5.79%) Splunk
6 34 (4.68%) ServiceNow
7 28 (3.86%) Cloudera
8 27 (3.72%) Talend
9 24 (3.31%) Red Hat
10 14 (1.93%) Informatica
11 11 (1.52%) SAP
12 10 (1.38%) StreamSets
13 9 (1.24%) Hortonworks
13 9 (1.24%) Vertica
14 8 (1.10%) Teradata
15 7 (0.96%) IBM
16 6 (0.83%) MuleSoft
17 4 (0.55%) Ab Initio
17 4 (0.55%) Alteryx
17 4 (0.55%) VMware