Job Vacancy Trend Salary Trend Salary Histogram Job Locations Skill Sets
Period
to 5 August 2020

The following table provides summary statistics for permanent job vacancies with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 5 August 2020 with a comparison to the same period in the previous 2 years.

Apache Spark
UK
6 months to
5 Aug 2020		 Same period 2019 Same period 2018
Rank 239 225 240
Rank change year-on-year -14 +15 +54
Permanent jobs citing Apache Spark 895 2,179 2,501
As % of all permanent jobs advertised in the UK 1.35% 1.52% 1.46%
As % of the Application Platforms category 9.55% 10.25% 8.78%
Number of salaries quoted 666 1,876 2,009
Median annual salary £75,000 £75,000 £70,000
Median salary % change year-on-year - +7.14% +4.48%
10th Percentile £47,500 £47,500 £43,500
90th Percentile £115,000 £110,625 £100,000
UK excluding London median annual salary £62,500 £60,000 £57,500
% change year-on-year +4.17% +4.35% +0.88%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for application platform skills.

All Application Platform Skills
UK
Permanent vacancies with a requirement for application platform skills 9,369 21,254 28,474
As % of all permanent IT jobs advertised in the UK 14.18% 14.86% 16.61%
Number of salaries quoted 7,388 16,810 22,783
Median annual salary £52,500 £50,000 £50,000
Median salary % change year-on-year +5.00% - +5.26%
10th Percentile £31,250 £27,750 £28,000
90th Percentile £90,000 £85,000 £81,250
UK excluding London median annual salary £45,000 £43,500 £43,750
% change year-on-year +3.45% -0.57% +8.02%

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a proportion of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Salary Trend

3-month moving average salary quoted in jobs citing Apache Spark.

Salary trend for Apache Spark in the UK

Apache Spark
Salary Histogram

Salary distribution for jobs citing Apache Spark over the 6 months to 5 August 2020.

Salary histogram for Apache Spark in the UK

Apache Spark
Top 16 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the UK over the 6 months to 5 August 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year		 Matching
Permanent
IT Job Ads		 Median Salary
Past 6 Months		 Median Salary
% Change
on Same Period
Last Year		 Live
Job
Vacancies
England -21 819 £75,000 - 114
London -32 574 £80,000 - 75
UK excluding London +66 323 £62,500 +4.17% 50
South East +121 110 £67,500 +12.50% 13
Work from Home +27 65 £75,000 - 20
Scotland +75 51 £55,000 -3.51% 9
Midlands +67 45 £67,500 +3.85% 11
North of England +84 35 £55,000 -8.33% 7
East Midlands +26 32 £75,000 +30.43% 9
East of England +72 31 £70,000 +16.67% 4
North West +73 26 £55,000 -15.38% 1
South West -14 26 £50,000 -9.09% 4
Wales +57 25 £50,000 +17.65% 2
West Midlands +88 13 £55,000 -21.43% 2
Yorkshire +88 6 £46,000 -18.06% 6
North East +16 3 £57,500 +33.90%

For the 6 months to 5 August 2020, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Apache Spark.

1 591 (66.03%) Hadoop
2 590 (65.92%) Python
3 529 (59.11%) Big Data
4 406 (45.36%) Java
5 392 (43.80%) AWS
6 360 (40.22%) Kafka
7 345 (38.55%) SQL
8 341 (38.10%) Analytics
9 305 (34.08%) Agile Software Development
10 304 (33.97%) Scala
11 286 (31.96%) Azure
12 285 (31.84%) Data Science
13 257 (28.72%) Machine Learning
14 246 (27.49%) Finance
15 239 (26.70%) NoSQL
16 225 (25.14%) DevOps
17 206 (23.02%) GCP
18 197 (22.01%) Degree
19 195 (21.79%) Apache
20 184 (20.56%) R
21 182 (20.34%) Apache Hive
22 179 (20.00%) Kubernetes
23 166 (18.55%) ETL
24 164 (18.32%) Docker
25 149 (16.65%) Data Engineering
26 147 (16.42%) Artificial Intelligence
27 145 (16.20%) Business Intelligence
27 145 (16.20%) Apache Cassandra
28 144 (16.09%) Software Engineering
29 136 (15.20%) Data Warehouse

Apache Spark
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 195 (21.79%) Apache
2 75 (8.38%) Elasticsearch
3 40 (4.47%) Apache Airflow
4 32 (3.58%) Jupyter
5 26 (2.91%) Apache Solr
5 26 (2.91%) SAS
6 21 (2.35%) Tomcat
7 20 (2.23%) Luigi
8 16 (1.79%) Confluence
9 15 (1.68%) Cloud Foundry
10 12 (1.34%) Apache Pig
10 12 (1.34%) CMS
11 9 (1.01%) Apache Synapse
11 9 (1.01%) Mule
12 8 (0.89%) nginx
12 8 (0.89%) Oracle Coherence
13 7 (0.78%) SharePoint
14 6 (0.67%) Tornado
15 4 (0.45%) Camunda
15 4 (0.45%) WildFly
Applications
1 7 (0.78%) Microsoft Excel
2 3 (0.34%) Gephi
3 2 (0.22%) Microsoft PowerPoint
3 2 (0.22%) MS Visio
4 1 (0.11%) SPSS
Business Applications
1 5 (0.56%) SAP CRM
1 5 (0.56%) SAP ERP
1 5 (0.56%) SAP IS-U
2 4 (0.45%) Eikon
3 2 (0.22%) Sparx Enterprise Architect
Cloud Services
1 392 (43.80%) AWS
2 286 (31.96%) Azure
3 206 (23.02%) GCP
4 78 (8.72%) Amazon EMR
5 77 (8.60%) Power Platform
6 66 (7.37%) Amazon S3
7 51 (5.70%) Serverless
8 42 (4.69%) Azure Data Factory
9 40 (4.47%) Amazon EC2
10 34 (3.80%) Amazon Kinesis
11 25 (2.79%) Amazon CloudWatch
12 24 (2.68%) Amazon SQS
12 24 (2.68%) Cloud Computing
13 23 (2.57%) SaaS
14 21 (2.35%) GitHub
15 18 (2.01%) Amazon GuardDuty
15 18 (2.01%) AWS CloudTrail
15 18 (2.01%) AWS Lambda
16 17 (1.90%) Dropbox
16 17 (1.90%) Slack
Communications & Networking
1 12 (1.34%) FTP
1 12 (1.34%) HTTP
2 8 (0.89%) SSL
3 6 (0.67%) Firewall
3 6 (0.67%) Intrusion Detection
3 6 (0.67%) TCP/IP
4 5 (0.56%) 5G
4 5 (0.56%) Kerberos
5 3 (0.34%) AMQP
6 2 (0.22%) DNS
6 2 (0.22%) Ethernet
6 2 (0.22%) MPLS
6 2 (0.22%) SAN
6 2 (0.22%) VPN
7 1 (0.11%) DSL
7 1 (0.11%) HTTPS
7 1 (0.11%) Network Security
7 1 (0.11%) Skype
7 1 (0.11%) Wi-Fi
Database & Business Intelligence
1 591 (66.03%) Hadoop
2 529 (59.11%) Big Data
3 239 (26.70%) NoSQL
4 182 (20.34%) Apache Hive
5 145 (16.20%) Apache Cassandra
6 136 (15.20%) Data Warehouse
7 116 (12.96%) Amazon Redshift
8 111 (12.40%) SQL Server
9 110 (12.29%) MongoDB
10 103 (11.51%) HBase
11 100 (11.17%) Data Lake
12 85 (9.50%) PostgreSQL
13 77 (8.60%) Power BI
14 67 (7.49%) Flink
15 64 (7.15%) Impala
16 56 (6.26%) Relational Database
17 49 (5.47%) Data Mining
18 45 (5.03%) BigQuery
19 42 (4.69%) Neo4j
20 40 (4.47%) Cosmos DB
Development Applications
1 105 (11.73%) Git (software)
2 71 (7.93%) Jenkins
3 44 (4.92%) Atlassian Bamboo
4 30 (3.35%) JIRA
5 23 (2.57%) JUnit
6 21 (2.35%) TeamCity
7 20 (2.23%) Gradle
8 17 (1.90%) Vagrant
9 16 (1.79%) Bitbucket
10 15 (1.68%) GitLab
11 14 (1.56%) Sonatype Nexus
12 13 (1.45%) Maven
13 10 (1.12%) Visual Studio
14 9 (1.01%) Artifactory
14 9 (1.01%) CircleCI
14 9 (1.01%) Cucumber
14 9 (1.01%) Octopus Deploy
15 7 (0.78%) GoCD
16 6 (0.67%) Travis CI
17 5 (0.56%) IBM UrbanCode
General
1 246 (27.49%) Finance
2 98 (10.95%) Banking
3 53 (5.92%) Marketing
4 49 (5.47%) Retail
5 37 (4.13%) Front Office
6 32 (3.58%) Public Sector
7 29 (3.24%) Investment Banking
8 28 (3.13%) Manufacturing
9 21 (2.35%) Telecoms
10 15 (1.68%) Games
11 13 (1.45%) Aerospace
12 10 (1.12%) Advertising
13 6 (0.67%) Legal
13 6 (0.67%) Retail Banking
14 5 (0.56%) Financial Institution
14 5 (0.56%) Pharmaceutical
15 4 (0.45%) Automotive
16 3 (0.34%) Law
17 2 (0.22%) Electronics
17 2 (0.22%) Publishing
Job Titles
1 321 (35.87%) Data Engineer
2 195 (21.79%) Developer
3 102 (11.40%) Architect
4 89 (9.94%) Big Data Engineer
5 84 (9.39%) Data Scientist
6 75 (8.38%) Data Architect
7 69 (7.71%) Java Developer
8 65 (7.26%) Senior Data Engineer
9 60 (6.70%) Senior Developer
10 52 (5.81%) Python Engineer
11 47 (5.25%) Senior Data Scientist
12 46 (5.14%) AWS Engineer
13 42 (4.69%) Software Engineer
14 35 (3.91%) Big Data Architect
15 30 (3.35%) Scala Developer
16 27 (3.02%) Cloud Engineer
16 27 (3.02%) Platform Engineer
17 20 (2.23%) Full Stack Developer
17 20 (2.23%) Solutions Architect
18 19 (2.12%) Security Engineer
Libraries, Frameworks & Software Standards
1 360 (40.22%) Kafka
2 91 (10.17%) Spring
3 55 (6.15%) Natural Language Processing
4 52 (5.81%) React
5 50 (5.59%) AngularJS
6 48 (5.36%) scikit-learn
7 46 (5.14%) Node.js
7 46 (5.14%) REST
7 46 (5.14%) Spark Streaming
7 46 (5.14%) Spring Boot
7 46 (5.14%) TensorFlow
7 46 (5.14%) Web Services
8 42 (4.69%) RabbitMQ
9 41 (4.58%) Pandas
10 40 (4.47%) NumPy
11 38 (4.25%) PySpark
12 37 (4.13%) .NET
13 36 (4.02%) HTML
14 28 (3.13%) JMS
15 26 (2.91%) SciPy
Miscellaneous
1 120 (13.41%) Analytical Skills
2 95 (10.61%) Algorithms
3 44 (4.92%) Hedge funds
4 43 (4.80%) Data Structures
5 40 (4.47%) Greenfield Project
6 37 (4.13%) Cloud Native
7 34 (3.80%) Management Information System
8 32 (3.58%) Blog
9 30 (3.35%) Equities
10 29 (3.24%) User Experience
11 22 (2.46%) Self-Motivation
12 20 (2.23%) Hybrid Cloud
13 19 (2.12%) Enterprise Software
14 18 (2.01%) Market Risk
15 16 (1.79%) Public Cloud
16 15 (1.68%) Acoustics
17 13 (1.45%) Life Science
17 13 (1.45%) Virtual Team
18 12 (1.34%) Data Centre
18 12 (1.34%) Robotics
Operating Systems
1 105 (11.73%) Linux
2 43 (4.80%) Unix
3 16 (1.79%) Red Hat Enterprise Linux
4 14 (1.56%) Windows
5 5 (0.56%) Android
5 5 (0.56%) Apple iOS
6 3 (0.34%) Ubuntu
6 3 (0.34%) Windows Server
7 2 (0.22%) DC/OS
8 1 (0.11%) Amazon Linux
8 1 (0.11%) CentOS
Processes & Methodologies
1 341 (38.10%) Analytics
2 305 (34.08%) Agile Software Development
3 285 (31.84%) Data Science
4 257 (28.72%) Machine Learning
5 225 (25.14%) DevOps
6 166 (18.55%) ETL
7 149 (16.65%) Data Engineering
8 147 (16.42%) Artificial Intelligence
9 145 (16.20%) Business Intelligence
10 144 (16.09%) Software Engineering
11 130 (14.53%) Data Analytics
12 125 (13.97%) Visualisation
13 121 (13.52%) Data Modelling
14 115 (12.85%) Computer Science
15 106 (11.84%) Microservices
16 95 (10.61%) Mathematics
17 80 (8.94%) Continuous Integration
17 80 (8.94%) Scrum
18 77 (8.60%) TDD
19 76 (8.49%) Data Visualisation
Programming Languages
1 590 (65.92%) Python
2 406 (45.36%) Java
3 345 (38.55%) SQL
4 304 (33.97%) Scala
5 184 (20.56%) R
6 97 (10.84%) JavaScript
7 72 (8.04%) C#
8 65 (7.26%) C++
9 40 (4.47%) Bash Shell
10 26 (2.91%) Go
11 25 (2.79%) Java 8
12 21 (2.35%) C
13 18 (2.01%) PHP
14 16 (1.79%) Ruby
15 15 (1.68%) Kotlin
15 15 (1.68%) Perl
15 15 (1.68%) TypeScript
16 13 (1.45%) MATLAB
16 13 (1.45%) Shell Script
17 12 (1.34%) Korn
Qualifications
1 197 (22.01%) Degree
2 100 (11.17%) Computer Science Degree
3 55 (6.15%) Security Cleared
4 49 (5.47%) SC Cleared
5 21 (2.35%) DV Cleared
5 21 (2.35%) PhD
6 18 (2.01%) Master's Degree
7 16 (1.79%) AWS Certification
8 12 (1.34%) Numerate Degree
9 8 (0.89%) PMI Certification
9 8 (0.89%) TOGAF Certification
10 6 (0.67%) Microsoft Certification
11 3 (0.34%) MBA
12 2 (0.22%) CCNA
12 2 (0.22%) CEH
12 2 (0.22%) Cisco Certification
12 2 (0.22%) CISSP
12 2 (0.22%) GIAC
12 2 (0.22%) Linux+ Certification
12 2 (0.22%) Network+ Certification
Quality Assurance & Compliance
1 24 (2.68%) QA
2 21 (2.35%) Data Quality
3 20 (2.23%) GDPR
4 7 (0.78%) NIST
4 7 (0.78%) Sarbanes-Oxley
5 1 (0.11%) Actionable Recommendations
5 1 (0.11%) IFRS
5 1 (0.11%) IFRS 9
5 1 (0.11%) ISO 9001
5 1 (0.11%) ISO/IEC 27001
System Software
1 164 (18.32%) Docker
2 45 (5.03%) HDFS
3 22 (2.46%) Apache ZooKeeper
4 17 (1.90%) VirtualBox
5 12 (1.34%) Apache Flume
6 4 (0.45%) Active Directory
6 4 (0.45%) Varnish
7 1 (0.11%) Virtual Machines
Systems Management
1 179 (20.00%) Kubernetes
2 72 (8.04%) Terraform
3 57 (6.37%) Ansible
4 26 (2.91%) Puppet
5 20 (2.23%) Kibana
6 17 (1.90%) Docker Swarm
6 17 (1.90%) Hadoop YARN
6 17 (1.90%) Opscode Chef
7 16 (1.79%) Grafana
8 15 (1.68%) Oozie
9 10 (1.12%) Salt
10 5 (0.56%) Ambari
10 5 (0.56%) Mesos
11 4 (0.45%) Nagios
11 4 (0.45%) Zipkin
12 3 (0.34%) Cobbler
12 3 (0.34%) Red Hat Satellite
13 2 (0.22%) logstash
13 2 (0.22%) Marathon
13 2 (0.22%) Prometheus
Vendors
1 114 (12.74%) Google
2 101 (11.28%) Tableau
3 84 (9.39%) Databricks
4 74 (8.27%) Microsoft
5 70 (7.82%) Splunk
6 57 (6.37%) Oracle
7 56 (6.26%) Cloudera
8 33 (3.69%) IBM
9 31 (3.46%) Hortonworks
10 26 (2.91%) Teradata
11 24 (2.68%) Informatica
12 23 (2.57%) Talend
13 19 (2.12%) ServiceNow
14 17 (1.90%) CrowdStrike
15 12 (1.34%) StreamSets
16 11 (1.23%) Netezza
17 10 (1.12%) Red Hat
18 8 (0.89%) Cognos
18 8 (0.89%) SAP
18 8 (0.89%) Sybase
126 Apache Spark job vacancies Nationwide