Period
to 1 October 2020

The following table provides summary statistics for permanent job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 1 October 2020 with a comparison to the same period in the previous 2 years.

Apache Spark
UK > England
6 months to
1 Oct 2020
Same period 2019 Same period 2018
Rank 238 223 240
Rank change year-on-year -15 +17 +39
Permanent jobs citing Apache Spark 613 1,985 2,180
As % of all permanent jobs advertised in England 1.39% 1.53% 1.41%
As % of the Application Platforms category 9.73% 10.32% 8.50%
Number of salaries quoted 515 1,717 1,736
Median annual salary £74,000 £72,500 £72,500
Median salary % change year-on-year +2.07% - +8.21%
10th Percentile £46,250 £47,500 £45,000
90th Percentile £115,000 £110,000 £100,000
UK median annual salary £70,000 £72,500 £70,000
% change year-on-year -3.45% +3.57% +7.69%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Permanent vacancies with a requirement for application platform skills 6,298 19,230 25,646
As % of all permanent IT jobs advertised in England 14.26% 14.84% 16.61%
Number of salaries quoted 5,236 15,316 20,718
Median annual salary £55,000 £50,000 £50,000
Median salary % change year-on-year +10.00% - +5.26%
10th Percentile £32,500 £27,500 £28,000
90th Percentile £90,000 £85,000 £82,500
UK median annual salary £55,000 £50,000 £50,000
% change year-on-year +10.00% - +5.26%

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a proportion of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Salary Trend in England

3-month moving average salary quoted in jobs citing Apache Spark in England.

Salary trend for Apache Spark in England

Apache Spark
Salary Histogram in England

Salary distribution for jobs citing Apache Spark in England over the 6 months to 1 October 2020.

Salary histogram for Apache Spark in England

Apache Spark
Job Locations in England

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the England region over the 6 months to 1 October 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
London -16 435 £80,000 - 78
South East +154 86 £55,000 -8.33% 10
North of England +93 30 £60,000 - 8
East of England +113 22 £70,000 +7.69% 3
North West +78 21 £57,500 -11.54% 5
South West +10 20 £47,500 -13.64% 3
Midlands +68 19 £87,500 +45.83% 2
East Midlands +37 16 £100,000 +73.91% 1
Yorkshire +111 9 £60,000 +9.09% 3
West Midlands +92 3 £56,250 -19.64% 1
Apache Spark
UK

For the 6 months to 1 October 2020, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the England region with a requirement for Apache Spark.

1 411 (67.05%) Python
2 343 (55.95%) Big Data
3 341 (55.63%) Hadoop
4 284 (46.33%) AWS
5 277 (45.19%) Java
6 271 (44.21%) Kafka
7 251 (40.95%) SQL
8 225 (36.70%) Scala
9 203 (33.12%) Analytics
10 193 (31.48%) Agile Software Development
11 190 (31.00%) Azure
12 185 (30.18%) Apache
13 184 (30.02%) DevOps
14 183 (29.85%) Finance
15 179 (29.20%) Data Science
16 172 (28.06%) Machine Learning
17 163 (26.59%) Kubernetes
18 156 (25.45%) GCP
19 142 (23.16%) Degree
20 135 (22.02%) NoSQL
21 123 (20.07%) Docker
22 116 (18.92%) ETL
23 112 (18.27%) Apache Hive
24 107 (17.46%) Artificial Intelligence
25 100 (16.31%) Data Lake
26 99 (16.15%) Data Warehouse
27 98 (15.99%) R
28 94 (15.33%) Mathematics
29 93 (15.17%) Analytical Skills
30 90 (14.68%) Data Engineering

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 185 (30.18%) Apache
2 71 (11.58%) Elasticsearch
3 33 (5.38%) Apache Airflow
4 21 (3.43%) Jupyter
5 16 (2.61%) Cloud Foundry
6 14 (2.28%) Luigi
6 14 (2.28%) SAS
7 12 (1.96%) Tomcat
8 10 (1.63%) CMS
9 9 (1.47%) Apache Pig
9 9 (1.47%) Apache Synapse
10 8 (1.31%) Confluence
11 5 (0.82%) nginx
12 3 (0.49%) Apache Solr
12 3 (0.49%) Oracle Coherence
12 3 (0.49%) Pachyderm
12 3 (0.49%) Tornado
12 3 (0.49%) WebSphere
13 1 (0.16%) Camunda
13 1 (0.16%) JBoss
Applications
1 5 (0.82%) Microsoft Excel
2 2 (0.33%) Microsoft PowerPoint
2 2 (0.33%) MS Visio
Business Applications
1 4 (0.65%) Eikon
2 3 (0.49%) Distributed Ledger
2 3 (0.49%) Oracle Cash Management
3 2 (0.33%) Sparx Enterprise Architect
4 1 (0.16%) SAP CRM
4 1 (0.16%) SAP ERP
4 1 (0.16%) SAP IS-U
Cloud Services
1 284 (46.33%) AWS
2 190 (31.00%) Azure
3 156 (25.45%) GCP
4 56 (9.14%) Amazon S3
5 51 (8.32%) Amazon EMR
6 49 (7.99%) Serverless
7 41 (6.69%) GitHub
8 38 (6.20%) Power Platform
9 36 (5.87%) Stackdriver
10 32 (5.22%) Amazon Kinesis
11 31 (5.06%) Amazon EC2
12 28 (4.57%) SaaS
13 26 (4.24%) Azure Data Factory
14 24 (3.92%) Amazon SQS
15 23 (3.75%) Amazon CloudWatch
16 19 (3.10%) Amazon GuardDuty
16 19 (3.10%) AWS CloudTrail
16 19 (3.10%) AWS Lambda
17 14 (2.28%) Logic Apps
18 13 (2.12%) AWS CloudFormation
Communications & Networking
1 11 (1.79%) Kerberos
2 10 (1.63%) FTP
3 8 (1.31%) 5G
3 8 (1.31%) HTTP
4 6 (0.98%) TCP/IP
5 5 (0.82%) Firewall
6 4 (0.65%) Internet
6 4 (0.65%) SSL
7 3 (0.49%) AMQP
7 3 (0.49%) Parlay
8 2 (0.33%) HTTPS
8 2 (0.33%) Wi-Fi
9 1 (0.16%) 3G
9 1 (0.16%) 4G
9 1 (0.16%) DSL
9 1 (0.16%) Network Security
9 1 (0.16%) OpenVPN
9 1 (0.16%) Skype
9 1 (0.16%) VPN
Database & Business Intelligence
1 343 (55.95%) Big Data
2 341 (55.63%) Hadoop
3 135 (22.02%) NoSQL
4 112 (18.27%) Apache Hive
5 100 (16.31%) Data Lake
6 99 (16.15%) Data Warehouse
7 84 (13.70%) PostgreSQL
8 83 (13.54%) Amazon Redshift
8 83 (13.54%) Apache Cassandra
9 66 (10.77%) SQL Server
10 60 (9.79%) Flink
11 57 (9.30%) MongoDB
12 54 (8.81%) Relational Database
13 47 (7.67%) HBase
14 43 (7.01%) Data Mining
15 38 (6.20%) Power BI
16 36 (5.87%) Impala
17 33 (5.38%) BigQuery
18 32 (5.22%) RDBMS
19 30 (4.89%) Snowflake Schema
Development Applications
1 89 (14.52%) Git (software)
2 59 (9.62%) Jenkins
3 40 (6.53%) Gradle
4 25 (4.08%) TeamCity
5 23 (3.75%) JIRA
6 20 (3.26%) Atlassian Bamboo
7 14 (2.28%) Cucumber
8 12 (1.96%) GitLab
9 9 (1.47%) CircleCI
9 9 (1.47%) Octopus Deploy
10 6 (0.98%) Artifactory
10 6 (0.98%) Bitbucket
10 6 (0.98%) Selenium
10 6 (0.98%) Sonatype Nexus
10 6 (0.98%) Team Foundation Server
10 6 (0.98%) Travis CI
11 5 (0.82%) GoCD
11 5 (0.82%) Maven
12 4 (0.65%) JUnit
13 2 (0.33%) ERwin
General
1 183 (29.85%) Finance
2 68 (11.09%) Banking
3 43 (7.01%) Marketing
4 42 (6.85%) Public Sector
5 27 (4.40%) Front Office
6 23 (3.75%) Retail
7 16 (2.61%) Manufacturing
7 16 (2.61%) Telecoms
8 14 (2.28%) Games
9 11 (1.79%) Investment Banking
10 10 (1.63%) Advertising
11 8 (1.31%) Pharmaceutical
12 6 (0.98%) Law
12 6 (0.98%) Retail Banking
13 4 (0.65%) Electronics
13 4 (0.65%) Financial Institution
14 3 (0.49%) Legal
15 2 (0.33%) Automotive
15 2 (0.33%) Private Banking
16 1 (0.16%) Aerospace
Job Titles
1 219 (35.73%) Data Engineer
2 156 (25.45%) Developer
3 75 (12.23%) Data Scientist
4 62 (10.11%) Senior Data Engineer
5 57 (9.30%) Architect
6 53 (8.65%) Java Developer
7 52 (8.48%) Senior Developer
8 49 (7.99%) Big Data Engineer
9 44 (7.18%) Python Engineer
10 41 (6.69%) Senior Data Scientist
11 40 (6.53%) Data Architect
12 35 (5.71%) AWS Engineer
13 22 (3.59%) Cloud Engineer
14 21 (3.43%) Full Stack Developer
14 21 (3.43%) Python Developer
15 20 (3.26%) Digital Developer
15 20 (3.26%) Digital Transformation
16 19 (3.10%) Big Data Architect
16 19 (3.10%) Java Engineer
16 19 (3.10%) Scala Developer
Libraries, Frameworks & Software Standards
1 271 (44.21%) Kafka
2 75 (12.23%) Spring
3 57 (9.30%) RabbitMQ
4 52 (8.48%) Natural Language Processing
5 49 (7.99%) TensorFlow
6 46 (7.50%) PySpark
7 43 (7.01%) scikit-learn
8 37 (6.04%) Pandas
8 37 (6.04%) Spring Boot
9 36 (5.87%) Spring Integration
10 33 (5.38%) React
11 31 (5.06%) AngularJS
11 31 (5.06%) NumPy
12 29 (4.73%) Node.js
13 26 (4.24%) Keras
13 26 (4.24%) Spark Streaming
14 24 (3.92%) RESTful
15 21 (3.43%) REST
16 20 (3.26%) Web Services
17 19 (3.10%) Akka
Miscellaneous
1 93 (15.17%) Analytical Skills
2 88 (14.36%) Algorithms
3 34 (5.55%) Hedge funds
4 31 (5.06%) Cloud Native
4 31 (5.06%) Management Information System
5 27 (4.40%) Data Structures
5 27 (4.40%) Equities
6 21 (3.43%) Self-Motivation
7 20 (3.26%) Hybrid Cloud
8 18 (2.94%) Greenfield Project
9 14 (2.28%) Enterprise Software
10 12 (1.96%) Blog
11 11 (1.79%) FMCG
11 11 (1.79%) Market Risk
12 10 (1.63%) Data Centre
12 10 (1.63%) Life Science
13 9 (1.47%) Robotics
14 8 (1.31%) Public Cloud
15 6 (0.98%) Mobile App
15 6 (0.98%) Virtual Team
Operating Systems
1 66 (10.77%) Linux
2 22 (3.59%) Unix
3 17 (2.77%) Red Hat Enterprise Linux
4 3 (0.49%) Ubuntu
5 2 (0.33%) DC/OS
6 1 (0.16%) Amazon Linux
6 1 (0.16%) CentOS
6 1 (0.16%) Windows
Processes & Methodologies
1 203 (33.12%) Analytics
2 193 (31.48%) Agile Software Development
3 184 (30.02%) DevOps
4 179 (29.20%) Data Science
5 172 (28.06%) Machine Learning
6 116 (18.92%) ETL
7 107 (17.46%) Artificial Intelligence
8 94 (15.33%) Mathematics
9 90 (14.68%) Data Engineering
10 88 (14.36%) Visualisation
11 87 (14.19%) Computer Science
12 81 (13.21%) Microservices
13 80 (13.05%) Business Intelligence
14 67 (10.93%) Software Engineering
15 66 (10.77%) Data Modelling
16 58 (9.46%) Scrum
17 56 (9.14%) Statistics
18 55 (8.97%) Data Visualisation
19 51 (8.32%) Data Analytics
20 47 (7.67%) Mentoring
Programming Languages
1 411 (67.05%) Python
2 277 (45.19%) Java
3 251 (40.95%) SQL
4 225 (36.70%) Scala
5 98 (15.99%) R
6 62 (10.11%) JavaScript
7 44 (7.18%) C#
8 26 (4.24%) C++
9 25 (4.08%) Bash Shell
10 21 (3.43%) Go
11 17 (2.77%) Kotlin
12 13 (2.12%) C
13 11 (1.79%) Java 8
13 11 (1.79%) PHP
13 11 (1.79%) Ruby
13 11 (1.79%) TypeScript
14 10 (1.63%) Shell Script
15 8 (1.31%) Clojure
15 8 (1.31%) Korn
15 8 (1.31%) Rust
Qualifications
1 142 (23.16%) Degree
2 76 (12.40%) Computer Science Degree
3 40 (6.53%) Security Cleared
4 39 (6.36%) SC Cleared
5 22 (3.59%) PhD
6 18 (2.94%) Master's Degree
7 14 (2.28%) AWS Certification
8 8 (1.31%) Numerate Degree
9 6 (0.98%) PMI Certification
10 5 (0.82%) TOGAF Certification
11 4 (0.65%) Postgraduate
12 3 (0.49%) Microsoft Certification
13 1 (0.16%) HND
Quality Assurance & Compliance
1 25 (4.08%) GDPR
2 13 (2.12%) QA
3 7 (1.14%) Data Quality
4 3 (0.49%) ISO 20022
4 3 (0.49%) PSD2
4 3 (0.49%) TARGET2
5 2 (0.33%) Sarbanes-Oxley
6 1 (0.16%) IFRS
6 1 (0.16%) IFRS 9
6 1 (0.16%) ISO 26262
6 1 (0.16%) ISO 9001
6 1 (0.16%) ISO/IEC 27001
6 1 (0.16%) MiFID
6 1 (0.16%) PMO
6 1 (0.16%) SLA
System Software
1 123 (20.07%) Docker
2 34 (5.55%) HDFS
3 11 (1.79%) Apache ZooKeeper
4 8 (1.31%) Apache Flume
5 4 (0.65%) Varnish
6 2 (0.33%) Active Directory
6 2 (0.33%) Virtual Machines
6 2 (0.33%) VirtualBox
Systems Management
1 163 (26.59%) Kubernetes
2 83 (13.54%) Terraform
3 42 (6.85%) Ansible
4 36 (5.87%) Grafana
5 14 (2.28%) Puppet
6 13 (2.12%) Hadoop YARN
7 12 (1.96%) Kibana
7 12 (1.96%) Oozie
8 11 (1.79%) Opscode Chef
9 6 (0.98%) Ambari
10 4 (0.65%) Mesos
10 4 (0.65%) Nagios
10 4 (0.65%) Zipkin
11 3 (0.49%) Cobbler
11 3 (0.49%) Red Hat Satellite
12 2 (0.33%) Docker Swarm
12 2 (0.33%) Marathon
12 2 (0.33%) Prometheus
12 2 (0.33%) Single Sign-On
13 1 (0.16%) SELinux
Vendors
1 80 (13.05%) Google
2 77 (12.56%) Databricks
3 58 (9.46%) Splunk
4 55 (8.97%) Tableau
5 44 (7.18%) Oracle
6 41 (6.69%) Cloudera
7 37 (6.04%) ServiceNow
8 35 (5.71%) Microsoft
9 18 (2.94%) Hortonworks
10 17 (2.77%) Teradata
11 16 (2.61%) Informatica
12 15 (2.45%) Red Hat
13 13 (2.12%) IBM
14 10 (1.63%) StreamSets
15 9 (1.47%) Talend
16 7 (1.14%) Ab Initio
17 6 (0.98%) Facebook
17 6 (0.98%) Netezza
17 6 (0.98%) Vertica
18 5 (0.82%) MapR