Period
to 4 June 2020

The following table provides summary statistics for permanent job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 4 June 2020 with a comparison to the same period in the previous 2 years.

Apache Spark
UK > England
6 months to
4 Jun 2020
Same period 2019 Same period 2018
Rank 241 227 231
Rank change year-on-year -14 +4 +64
Permanent jobs citing Apache Spark 1,100 2,020 2,480
As % of all permanent jobs advertised in England 1.37% 1.53% 1.55%
As % of the Application Platforms category 9.66% 10.05% 9.17%
Number of salaries quoted 863 1,694 2,040
Median annual salary £75,000 £75,000 £72,500
Median salary % change year-on-year - +3.45% +7.41%
10th Percentile £47,500 £48,926 £42,500
90th Percentile £110,000 £110,000 £102,500
UK median annual salary £75,000 £75,000 £70,000
% change year-on-year - +7.14% +3.70%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Permanent vacancies with a requirement for application platform skills 11,387 20,104 27,037
As % of all permanent IT jobs advertised in England 14.19% 15.21% 16.87%
Number of salaries quoted 9,024 15,815 21,959
Median annual salary £52,500 £50,000 £50,000
Median salary % change year-on-year +5.00% - +5.26%
10th Percentile £30,000 £28,375 £28,000
90th Percentile £87,500 £85,000 £80,000
UK median annual salary £52,500 £50,000 £50,000
% change year-on-year +5.00% - +8.70%

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a proportion of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Salary Trend in England

3-month moving average salary quoted in jobs citing Apache Spark in England.

Salary trend for Apache Spark in England

Apache Spark
Salary Histogram in England

Salary distribution for jobs citing Apache Spark in England over the 6 months to 4 June 2020.

Salary histogram for Apache Spark in England

Apache Spark
Job Locations in England

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the England region over the 6 months to 4 June 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
London -24 793 £77,500 -3.13% 76
South East +97 132 £67,500 -10.00% 15
Midlands +81 65 £72,500 +11.54% 1
North of England +63 50 £55,000 -4.35% 17
East Midlands +40 42 £75,000 +30.43% 1
East of England +62 32 £75,000 +25.00% 15
North West +54 30 £55,000 -8.33% 11
South West -13 28 £50,000 -9.09%
West Midlands +68 23 £55,000 -21.43%
Yorkshire +79 17 £46,500 -6.38% 6
North East +2 3 £57,500 +16.55%
Apache Spark
UK

For the 6 months to 4 June 2020, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the England region with a requirement for Apache Spark.

1 738 (67.09%) Python
2 729 (66.27%) Hadoop
3 689 (62.64%) Big Data
4 511 (46.45%) AWS
5 483 (43.91%) Java
6 463 (42.09%) Analytics
7 454 (41.27%) Agile Software Development
8 449 (40.82%) SQL
9 441 (40.09%) Kafka
10 429 (39.00%) Scala
11 423 (38.45%) Data Science
12 336 (30.55%) Machine Learning
13 320 (29.09%) Azure
14 292 (26.55%) NoSQL
15 265 (24.09%) DevOps
16 260 (23.64%) Docker
17 257 (23.36%) GCP
18 253 (23.00%) Finance
19 242 (22.00%) Degree
20 233 (21.18%) ETL
21 225 (20.45%) Apache Hive
22 224 (20.36%) Kubernetes
23 217 (19.73%) Data Engineering
24 202 (18.36%) Artificial Intelligence
25 200 (18.18%) Software Engineering
26 189 (17.18%) Apache
27 180 (16.36%) Data Warehouse
27 180 (16.36%) R
28 173 (15.73%) Data Analytics
29 162 (14.73%) Business Intelligence

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 189 (17.18%) Apache
2 95 (8.64%) Elasticsearch
3 69 (6.27%) Apache Airflow
4 37 (3.36%) Jupyter
5 35 (3.18%) Apache Solr
6 29 (2.64%) SAS
7 28 (2.55%) Luigi
8 20 (1.82%) Confluence
9 14 (1.27%) Apache Pig
9 14 (1.27%) Tomcat
10 12 (1.09%) CMS
11 8 (0.73%) Oracle Coherence
12 7 (0.64%) nginx
13 6 (0.55%) JBoss
13 6 (0.55%) Tornado
13 6 (0.55%) WildFly
14 5 (0.45%) Camunda
15 4 (0.36%) Cloud Foundry
15 4 (0.36%) Oracle Workflow
16 3 (0.27%) WebSphere
Applications
1 8 (0.73%) SPSS
2 6 (0.55%) Microsoft Excel
3 3 (0.27%) Gephi
4 2 (0.18%) Microsoft PowerPoint
4 2 (0.18%) MS Visio
5 1 (0.091%) Chatbot
5 1 (0.091%) Weka
Business Applications
1 5 (0.45%) SAP CRM
1 5 (0.45%) SAP ERP
1 5 (0.45%) SAP IS-U
2 3 (0.27%) Sparx Enterprise Architect
3 2 (0.18%) Eikon
3 2 (0.18%) SAP BPC
3 2 (0.18%) SAP S/4HANA
4 1 (0.091%) Adobe Campaign
Cloud Services
1 511 (46.45%) AWS
2 320 (29.09%) Azure
3 257 (23.36%) GCP
4 114 (10.36%) Amazon EMR
5 86 (7.82%) Amazon S3
6 85 (7.73%) Power Platform
7 60 (5.45%) Amazon Kinesis
7 60 (5.45%) Serverless
8 55 (5.00%) Amazon EC2
9 38 (3.45%) Azure Data Factory
10 33 (3.00%) Cloud Computing
11 31 (2.82%) AWS Lambda
12 26 (2.36%) IBM Cloud
13 22 (2.00%) SaaS
14 21 (1.91%) Amazon CloudWatch
15 20 (1.82%) Amazon SQS
16 17 (1.55%) Dropbox
16 17 (1.55%) Slack
17 16 (1.45%) PaaS
18 13 (1.18%) GitHub
Communications & Networking
1 14 (1.27%) FTP
2 12 (1.09%) HTTP
3 8 (0.73%) Intrusion Detection
3 8 (0.73%) SSL
4 6 (0.55%) TCP/IP
5 4 (0.36%) Firewall
6 3 (0.27%) 5G
6 3 (0.27%) AMQP
6 3 (0.27%) DNS
6 3 (0.27%) SAN
7 2 (0.18%) 3G
7 2 (0.18%) 4G
7 2 (0.18%) Ethernet
7 2 (0.18%) HTTPS
7 2 (0.18%) LTE
7 2 (0.18%) MPLS
7 2 (0.18%) VPN
7 2 (0.18%) Wi-Fi
7 2 (0.18%) WLAN
8 1 (0.091%) IPFS
Database & Business Intelligence
1 729 (66.27%) Hadoop
2 689 (62.64%) Big Data
3 292 (26.55%) NoSQL
4 225 (20.45%) Apache Hive
5 180 (16.36%) Data Warehouse
6 155 (14.09%) Apache Cassandra
7 150 (13.64%) Amazon Redshift
8 140 (12.73%) MongoDB
9 128 (11.64%) SQL Server
10 122 (11.09%) HBase
11 116 (10.55%) PostgreSQL
12 114 (10.36%) Data Lake
13 98 (8.91%) Relational Database
14 90 (8.18%) Neo4j
15 85 (7.73%) Power BI
16 81 (7.36%) Impala
17 77 (7.00%) Flink
18 64 (5.82%) BigQuery
19 57 (5.18%) Amazon Athena
20 48 (4.36%) Data Mining
Development Applications
1 128 (11.64%) Git (software)
2 117 (10.64%) Jenkins
3 56 (5.09%) Atlassian Bamboo
4 43 (3.91%) JIRA
5 36 (3.27%) Vagrant
6 22 (2.00%) JUnit
7 21 (1.91%) Maven
7 21 (1.91%) TeamCity
8 16 (1.45%) Visual Studio
9 15 (1.36%) Bitbucket
10 14 (1.27%) Sonatype Nexus
11 13 (1.18%) Visual Studio Team System
12 12 (1.09%) GitLab
13 9 (0.82%) Artifactory
13 9 (0.82%) Cucumber
14 8 (0.73%) GoCD
14 8 (0.73%) Octopus Deploy
15 7 (0.64%) CircleCI
16 6 (0.55%) git-flow
16 6 (0.55%) IBM UrbanCode
General
1 253 (23.00%) Finance
2 86 (7.82%) Banking
3 71 (6.45%) Retail
4 54 (4.91%) Marketing
5 46 (4.18%) Front Office
6 36 (3.27%) Manufacturing
7 32 (2.91%) Telecoms
8 29 (2.64%) Public Sector
9 22 (2.00%) Games
10 20 (1.82%) Automotive
11 18 (1.64%) Investment Banking
11 18 (1.64%) Legal
12 14 (1.27%) Aerospace
13 9 (0.82%) Advertising
14 5 (0.45%) Retail Banking
15 4 (0.36%) Financial Institution
16 3 (0.27%) Electronics
16 3 (0.27%) Pharmaceutical
17 2 (0.18%) Private Banking
17 2 (0.18%) Publishing
Job Titles
1 384 (34.91%) Data Engineer
2 223 (20.27%) Developer
3 138 (12.55%) Architect
4 110 (10.00%) Big Data Engineer
5 107 (9.73%) Data Architect
6 100 (9.09%) Java Developer
7 94 (8.55%) Data Scientist
8 75 (6.82%) Senior Developer
9 67 (6.09%) Senior Data Engineer
10 51 (4.64%) AWS Engineer
11 49 (4.45%) Python Engineer
12 48 (4.36%) Senior Data Scientist
13 47 (4.27%) Big Data Architect
13 47 (4.27%) Software Engineer
14 44 (4.00%) Senior Java Developer
15 38 (3.45%) Scala Developer
16 31 (2.82%) Platform Engineer
17 30 (2.73%) Cloud Engineer
17 30 (2.73%) DevOps Engineer
18 28 (2.55%) Analyst
Libraries, Frameworks & Software Standards
1 441 (40.09%) Kafka
2 76 (6.91%) Spark Streaming
3 74 (6.73%) Natural Language Processing
4 73 (6.64%) Spring
4 73 (6.64%) TensorFlow
5 64 (5.82%) React
6 58 (5.27%) scikit-learn
7 55 (5.00%) Web Services
8 53 (4.82%) AngularJS
9 48 (4.36%) Node.js
10 47 (4.27%) REST
11 46 (4.18%) .NET
12 43 (3.91%) PySpark
12 43 (3.91%) Spring Boot
13 40 (3.64%) NumPy
13 40 (3.64%) Pandas
14 34 (3.09%) RabbitMQ
15 29 (2.64%) HTML
16 28 (2.55%) Akka
16 28 (2.55%) SciPy
Miscellaneous
1 148 (13.45%) Analytical Skills
2 110 (10.00%) Algorithms
3 71 (6.45%) Greenfield Project
4 55 (5.00%) Data Structures
5 44 (4.00%) Hedge funds
6 43 (3.91%) Blog
7 36 (3.27%) Cloud Native
8 35 (3.18%) Public Cloud
9 34 (3.09%) Management Information System
10 28 (2.55%) Acoustics
11 27 (2.45%) User Experience
12 23 (2.09%) Enterprise Software
13 20 (1.82%) Equities
14 19 (1.73%) Self-Motivation
15 17 (1.55%) Mobile App
16 15 (1.36%) Hybrid Cloud
16 15 (1.36%) Market Risk
17 13 (1.18%) Virtual Team
18 12 (1.09%) Robotics
19 11 (1.00%) Life Science
Operating Systems
1 142 (12.91%) Linux
2 54 (4.91%) Unix
3 23 (2.09%) Windows
4 9 (0.82%) Red Hat Enterprise Linux
5 7 (0.64%) Android
5 7 (0.64%) Apple iOS
5 7 (0.64%) VMS
6 4 (0.36%) Windows Server
7 1 (0.091%) Ubuntu
Processes & Methodologies
1 463 (42.09%) Analytics
2 454 (41.27%) Agile Software Development
3 423 (38.45%) Data Science
4 336 (30.55%) Machine Learning
5 265 (24.09%) DevOps
6 233 (21.18%) ETL
7 217 (19.73%) Data Engineering
8 202 (18.36%) Artificial Intelligence
9 200 (18.18%) Software Engineering
10 173 (15.73%) Data Analytics
11 162 (14.73%) Business Intelligence
12 160 (14.55%) Data Modelling
13 148 (13.45%) Computer Science
14 129 (11.73%) Microservices
14 129 (11.73%) Visualisation
15 127 (11.55%) Continuous Integration
16 109 (9.91%) Scrum
17 105 (9.55%) Fintech
18 95 (8.64%) Data Analysis
19 93 (8.45%) Open Source
Programming Languages
1 738 (67.09%) Python
2 483 (43.91%) Java
3 449 (40.82%) SQL
4 429 (39.00%) Scala
5 180 (16.36%) R
6 108 (9.82%) JavaScript
7 85 (7.73%) C++
8 80 (7.27%) C#
9 58 (5.27%) Bash Shell
10 38 (3.45%) Go
11 28 (2.55%) MATLAB
12 26 (2.36%) C
13 23 (2.09%) PHP
13 23 (2.09%) Shell Script
14 22 (2.00%) Kotlin
15 20 (1.82%) Java 8
16 19 (1.73%) TypeScript
17 18 (1.64%) Perl
18 12 (1.09%) PL/SQL
18 12 (1.09%) T-SQL
Qualifications
1 242 (22.00%) Degree
2 114 (10.36%) Computer Science Degree
3 57 (5.18%) Security Cleared
4 51 (4.64%) SC Cleared
5 35 (3.18%) PhD
6 30 (2.73%) DV Cleared
7 18 (1.64%) AWS Certification
8 16 (1.45%) Master's Degree
9 10 (0.91%) Numerate Degree
10 8 (0.73%) Microsoft Certification
10 8 (0.73%) PMI Certification
10 8 (0.73%) TOGAF Certification
11 6 (0.55%) MBA
12 5 (0.45%) Postgraduate
13 4 (0.36%) CCNA
13 4 (0.36%) GIAC
13 4 (0.36%) Linux+ Certification
13 4 (0.36%) MCSE
13 4 (0.36%) Network+ Certification
13 4 (0.36%) SSCP
Quality Assurance & Compliance
1 39 (3.55%) Data Quality
2 20 (1.82%) GDPR
2 20 (1.82%) QA
3 7 (0.64%) NIST
4 5 (0.45%) Sarbanes-Oxley
5 2 (0.18%) SLA
6 1 (0.091%) Actionable Recommendations
6 1 (0.091%) MiFID
System Software
1 260 (23.64%) Docker
2 62 (5.64%) HDFS
3 38 (3.45%) Apache Flume
4 36 (3.27%) VirtualBox
5 34 (3.09%) Apache ZooKeeper
6 5 (0.45%) Active Directory
7 2 (0.18%) Varnish
8 1 (0.091%) Alluxio
8 1 (0.091%) Ceph
8 1 (0.091%) etcd
8 1 (0.091%) Gluster
8 1 (0.091%) Hyper-V
8 1 (0.091%) VMware Infrastructure
Systems Management
1 224 (20.36%) Kubernetes
2 117 (10.64%) Ansible
3 88 (8.00%) Terraform
4 43 (3.91%) Puppet
5 37 (3.36%) Docker Swarm
6 28 (2.55%) Oozie
7 25 (2.27%) Hadoop YARN
8 24 (2.18%) Salt
9 23 (2.09%) Opscode Chef
10 19 (1.73%) Kibana
11 12 (1.09%) Mesos
12 10 (0.91%) Nagios
13 9 (0.82%) Prometheus
14 6 (0.55%) Grafana
15 4 (0.36%) Ambari
15 4 (0.36%) Chronos
15 4 (0.36%) HAProxy
15 4 (0.36%) Marathon
15 4 (0.36%) Zipkin
16 2 (0.18%) logstash
Vendors
1 155 (14.09%) Google
2 103 (9.36%) Tableau
3 92 (8.36%) Oracle
4 82 (7.45%) Microsoft
5 78 (7.09%) Cloudera
6 72 (6.55%) Databricks
7 58 (5.27%) Informatica
8 57 (5.18%) Talend
9 55 (5.00%) Hortonworks
10 51 (4.64%) IBM
11 44 (4.00%) Splunk
12 31 (2.82%) Teradata
13 17 (1.55%) CrowdStrike
14 15 (1.36%) Netezza
14 15 (1.36%) Sybase
15 14 (1.27%) Ab Initio
16 12 (1.09%) Facebook
16 12 (1.09%) StreamSets
16 12 (1.09%) Vertica
17 10 (0.91%) Dataiku