Period
to 26 February 2020

The following table provides summary statistics for permanent job vacancies with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 26 February 2020 with a comparison to the same period in the previous 2 years.

Apache Spark
UK
6 months to
26 Feb 2020
Same period 2019 Same period 2018
Rank 237 244 243
Rank change year-on-year +7 -1 +72
Permanent jobs citing Apache Spark 1,683 2,181 2,540
As % of all permanent IT jobs advertised in the UK 1.37% 1.44% 1.45%
As % of the Application Platforms category 9.33% 9.18% 8.56%
Number of salaries quoted 1,351 1,713 2,064
UK median annual salary £72,500 £74,000 £70,000
Median salary % change year-on-year -2.03% +5.71% +3.70%
10th Percentile £47,500 £45,000 £42,500
90th Percentile £105,000 £106,000 £105,000
UK excluding London median annual salary £60,000 £62,138 £55,000
% change year-on-year -3.44% +12.98% -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for application platform skills.

All Application Platform Skills
UK
Permanent vacancies with a requirement for application platform skills 18,032 23,769 29,670
As % of all permanent IT jobs advertised in the UK 14.65% 15.70% 16.92%
Number of salaries quoted 14,513 18,952 24,514
UK median annual salary £50,000 £50,000 £47,500
Median salary % change year-on-year - +5.26% +5.56%
10th Percentile £28,000 £28,250 £27,500
90th Percentile £85,000 £85,000 £78,750
UK excluding London median annual salary £45,000 £43,000 £41,500
% change year-on-year +4.65% +3.61% +3.75%

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a percentage of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Salary Trend

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark.

Salary trend for Apache Spark in the UK

Apache Spark
Salary Histogram

The salary distribution of IT jobs citing Apache Spark over the 6 months to 26 February 2020.

Salary histogram for Apache Spark in the UK

Apache Spark
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the UK over the 6 months to 26 February 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
England +10 1,626 £72,500 -3.33% 283
London -1 1,145 £75,000 -6.25% 203
UK excluding London -10 525 £60,000 -3.44% 105
South East +3 149 £60,000 -11.11% 31
North of England +9 117 £55,000 -15.38% 14
Midlands +60 110 £72,500 -3.33% 14
East Midlands +42 81 £73,750 -1.67% 5
East of England +54 55 £75,000 +3.45% 8
Yorkshire +40 55 £55,000 +10.73% 6
North West +14 55 £57,500 -17.86% 8
Scotland +3 45 £56,250 +12.50% 20
South West -59 42 £52,500 -4.98% 13
West Midlands +40 29 £60,000 +140.00% 9
North East +1 7 £54,671 -27.10%
Northern Ireland +12 3 £55,000 +29.41% 3
Wales -11 3 - - 2
Channel Islands +2 1 - -

For the 6 months to 26 February 2020, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Apache Spark.

1 1,062 (63.10%) Python
2 1,046 (62.15%) Hadoop
3 1,027 (61.02%) Big Data
4 769 (45.69%) AWS
5 751 (44.62%) Java
6 694 (41.24%) Agile Software Development
7 684 (40.64%) SQL
8 666 (39.57%) Analytics
9 661 (39.28%) Scala
10 627 (37.25%) Data Science
11 606 (36.01%) Kafka
12 482 (28.64%) Machine Learning
13 478 (28.40%) Azure
14 404 (24.00%) NoSQL
15 399 (23.71%) Docker
15 399 (23.71%) GCP
16 396 (23.53%) Apache Hive
17 376 (22.34%) ETL
18 357 (21.21%) DevOps
19 346 (20.56%) Data Engineering
20 343 (20.38%) Finance
21 340 (20.20%) Kubernetes
22 318 (18.89%) Software Engineering
23 291 (17.29%) Degree
24 286 (16.99%) Business Intelligence
25 276 (16.40%) Data Modelling
25 276 (16.40%) Linux
26 274 (16.28%) Data Warehouse
27 272 (16.16%) Apache
28 264 (15.69%) Artificial Intelligence

Apache Spark
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 272 (16.16%) Apache
2 166 (9.86%) Elasticsearch
3 98 (5.82%) Apache Airflow
4 50 (2.97%) Luigi
5 44 (2.61%) Apache Solr
6 34 (2.02%) SAS
7 27 (1.60%) Jupyter
8 23 (1.37%) Confluence
9 22 (1.31%) Apache Pig
10 15 (0.89%) nginx
11 12 (0.71%) OpenStack
12 8 (0.48%) SharePoint
13 6 (0.36%) IPython
14 5 (0.30%) Hyperledger
15 4 (0.24%) DataSynapse
15 4 (0.24%) JBoss
15 4 (0.24%) Mule
15 4 (0.24%) Oracle Workflow
15 4 (0.24%) WildFly
16 3 (0.18%) Tomcat
Applications
1 8 (0.48%) SPSS
2 5 (0.30%) Gephi
2 5 (0.30%) Microsoft Excel
3 2 (0.12%) Chatbot
4 1 (0.059%) Microsoft Office
4 1 (0.059%) Microsoft PowerPoint
4 1 (0.059%) Weka
Business Applications
1 6 (0.36%) Sparx Enterprise Architect
2 2 (0.12%) Eikon
2 2 (0.12%) SAP BPC
2 2 (0.12%) SAP S/4HANA
3 1 (0.059%) Adobe Campaign
3 1 (0.059%) Magento
3 1 (0.059%) NetSuite
Cloud Services
1 769 (45.69%) AWS
2 478 (28.40%) Azure
3 399 (23.71%) GCP
4 140 (8.32%) Amazon EMR
5 132 (7.84%) Amazon S3
6 108 (6.42%) Power Platform
7 92 (5.47%) Amazon Kinesis
8 87 (5.17%) Serverless
9 64 (3.80%) AWS Lambda
10 55 (3.27%) PaaS
11 47 (2.79%) Amazon EC2
12 42 (2.50%) Cloud Computing
13 40 (2.38%) SaaS
14 38 (2.26%) IBM Cloud
15 33 (1.96%) OpenShift
16 32 (1.90%) Azure Data Factory
17 31 (1.84%) GitHub
18 27 (1.60%) Azure Machine Learning
19 26 (1.54%) Amazon SageMaker
20 17 (1.01%) IaaS
Communications & Networking
1 38 (2.26%) HTTP
2 21 (1.25%) Firewall
3 15 (0.89%) HTTPS
4 14 (0.83%) VPN
5 11 (0.65%) SSL
5 11 (0.65%) TCP/IP
6 10 (0.59%) Kerberos
7 9 (0.53%) Internet
8 7 (0.42%) Intrusion Detection
9 6 (0.36%) 5G
9 6 (0.36%) DNS
9 6 (0.36%) FTP
10 5 (0.30%) 3G
10 5 (0.30%) 4G
10 5 (0.30%) Ethernet
10 5 (0.30%) MPLS
10 5 (0.30%) Wi-Fi
11 2 (0.12%) LTE
11 2 (0.12%) SAN
11 2 (0.12%) WLAN
Database & Business Intelligence
1 1,046 (62.15%) Hadoop
2 1,027 (61.02%) Big Data
3 404 (24.00%) NoSQL
4 396 (23.53%) Apache Hive
5 274 (16.28%) Data Warehouse
6 223 (13.25%) Amazon Redshift
7 211 (12.54%) PostgreSQL
8 201 (11.94%) Apache Cassandra
9 172 (10.22%) Relational Database
10 169 (10.04%) HBase
11 163 (9.69%) MongoDB
12 155 (9.21%) SQL Server
13 146 (8.67%) Impala
14 132 (7.84%) Data Lake
15 123 (7.31%) Neo4j
16 121 (7.19%) BigQuery
17 108 (6.42%) Power BI
18 100 (5.94%) Flink
19 86 (5.11%) Amazon Athena
20 76 (4.52%) Snowflake Schema
Development Applications
1 238 (14.14%) Git (software)
2 176 (10.46%) Jenkins
3 103 (6.12%) JIRA
4 68 (4.04%) Atlassian Bamboo
5 50 (2.97%) Vagrant
6 44 (2.61%) JUnit
7 32 (1.90%) Maven
8 20 (1.19%) Bitbucket
8 20 (1.19%) git-flow
9 19 (1.13%) Subversion
9 19 (1.13%) Visual Studio Team System
10 16 (0.95%) Visual Studio
11 14 (0.83%) Gradle
11 14 (0.83%) TeamCity
12 13 (0.77%) Mockito
13 12 (0.71%) Eclipse
14 11 (0.65%) IntelliJ
14 11 (0.65%) kappa
15 10 (0.59%) Mercurial
15 10 (0.59%) Sonatype Nexus
General
1 343 (20.38%) Finance
2 104 (6.18%) Retail
3 97 (5.76%) Banking
4 61 (3.62%) Marketing
5 50 (2.97%) Public Sector
6 42 (2.50%) Manufacturing
6 42 (2.50%) Telecoms
7 41 (2.44%) Automotive
7 41 (2.44%) Front Office
8 39 (2.32%) Legal
9 33 (1.96%) Games
10 18 (1.07%) Advertising
11 11 (0.65%) Investment Banking
12 10 (0.59%) Financial Institution
13 8 (0.48%) Billing
13 8 (0.48%) Electronics
14 6 (0.36%) Online Betting
14 6 (0.36%) Publishing
15 5 (0.30%) Back Office
15 5 (0.30%) Retail Banking
Job Titles
1 524 (31.13%) Data Engineer
2 375 (22.28%) Developer
3 208 (12.36%) Architect
3 208 (12.36%) Java Developer
4 156 (9.27%) Data Architect
5 147 (8.73%) Data Scientist
6 114 (6.77%) Senior Developer
7 109 (6.48%) Big Data Engineer
8 83 (4.93%) Senior Data Scientist
9 82 (4.87%) Software Engineer
10 81 (4.81%) Senior Data Engineer
11 78 (4.63%) Senior Java Developer
12 71 (4.22%) Python Engineer
12 71 (4.22%) Scala Developer
13 56 (3.33%) Java Engineer
14 48 (2.85%) Big Data Architect
14 48 (2.85%) Python Developer
15 47 (2.79%) DevOps Engineer
16 46 (2.73%) Analyst
17 45 (2.67%) Big Data Developer
Libraries, Frameworks & Software Standards
1 606 (36.01%) Kafka
2 119 (7.07%) TensorFlow
3 117 (6.95%) Node.js
4 116 (6.89%) REST
5 115 (6.83%) Natural Language Processing
6 114 (6.77%) Spring
7 102 (6.06%) Web Services
8 98 (5.82%) React
9 92 (5.47%) Spark Streaming
10 85 (5.05%) AngularJS
11 83 (4.93%) scikit-learn
12 74 (4.40%) RESTful
13 65 (3.86%) HTML
14 56 (3.33%) Pandas
15 54 (3.21%) Akka
16 53 (3.15%) Spring Boot
17 49 (2.91%) .NET
18 44 (2.61%) Dask
18 44 (2.61%) JSON
19 43 (2.55%) Elastic Stack
Miscellaneous
1 238 (14.14%) Analytical Skills
2 171 (10.16%) Algorithms
3 94 (5.59%) Greenfield Project
4 92 (5.47%) Data Structures
5 64 (3.80%) Public Cloud
6 43 (2.55%) Management Information System
7 42 (2.50%) Blog
8 40 (2.38%) Self-Motivation
9 33 (1.96%) Acoustics
10 30 (1.78%) Cloud Native
11 27 (1.60%) Hedge funds
12 26 (1.54%) Mobile App
12 26 (1.54%) User Experience
12 26 (1.54%) Virtual Team
13 17 (1.01%) Video on Demand
14 16 (0.95%) Social Media
15 15 (0.89%) Enterprise Software
16 14 (0.83%) Data Centre
17 11 (0.65%) Cybercrime
17 11 (0.65%) Derivative
Operating Systems
1 276 (16.40%) Linux
2 74 (4.40%) Unix
3 26 (1.54%) Windows
4 12 (0.71%) Android
4 12 (0.71%) Apple iOS
5 7 (0.42%) VMS
6 3 (0.18%) Red Hat Enterprise Linux
7 2 (0.12%) Windows Server
8 1 (0.059%) CoreOS
8 1 (0.059%) Ubuntu
Processes & Methodologies
1 694 (41.24%) Agile Software Development
2 666 (39.57%) Analytics
3 627 (37.25%) Data Science
4 482 (28.64%) Machine Learning
5 376 (22.34%) ETL
6 357 (21.21%) DevOps
7 346 (20.56%) Data Engineering
8 318 (18.89%) Software Engineering
9 286 (16.99%) Business Intelligence
10 276 (16.40%) Data Modelling
11 264 (15.69%) Artificial Intelligence
12 251 (14.91%) Computer Science
13 245 (14.56%) Data Analytics
14 226 (13.43%) Microservices
15 218 (12.95%) Visualisation
16 169 (10.04%) Continuous Integration
17 158 (9.39%) TDD
18 153 (9.09%) Fintech
18 153 (9.09%) Scrum
19 146 (8.67%) Statistics
Programming Languages
1 1,062 (63.10%) Python
2 751 (44.62%) Java
3 684 (40.64%) SQL
4 661 (39.28%) Scala
5 202 (12.00%) JavaScript
6 193 (11.47%) R
7 125 (7.43%) C++
8 120 (7.13%) C#
9 82 (4.87%) Bash Shell
10 79 (4.69%) Go
11 61 (3.62%) C
12 43 (2.55%) Shell Script
13 42 (2.50%) Java 8
13 42 (2.50%) PHP
14 39 (2.32%) Kotlin
15 34 (2.02%) TypeScript
16 30 (1.78%) MATLAB
17 27 (1.60%) PL/SQL
18 23 (1.37%) ES6
18 23 (1.37%) Ruby
Qualifications
1 291 (17.29%) Degree
2 124 (7.37%) Computer Science Degree
3 72 (4.28%) PhD
4 62 (3.68%) Security Cleared
5 50 (2.97%) SC Cleared
6 42 (2.50%) DV Cleared
7 14 (0.83%) Numerate Degree
8 13 (0.77%) CCNA
8 13 (0.77%) Cisco Certification
8 13 (0.77%) Master's Degree
9 12 (0.71%) AWS Certification
9 12 (0.71%) CISSP
9 12 (0.71%) CompTIA Security+
9 12 (0.71%) GIAC
9 12 (0.71%) Linux+ Certification
9 12 (0.71%) MCSE
9 12 (0.71%) Microsoft Certification
9 12 (0.71%) Network+ Certification
9 12 (0.71%) SANS
9 12 (0.71%) SSCP
Quality Assurance & Compliance
1 94 (5.59%) Data Quality
2 61 (3.62%) QA
3 27 (1.60%) GDPR
4 9 (0.53%) ISO/IEC 27001
5 4 (0.24%) MiFID
6 3 (0.18%) CDISC
6 3 (0.18%) EMIR
6 3 (0.18%) HL7
6 3 (0.18%) ISO 26262
7 2 (0.12%) Sarbanes-Oxley
7 2 (0.12%) SLA
8 1 (0.059%) Actionable Recommendations
System Software
1 399 (23.71%) Docker
2 109 (6.48%) HDFS
3 51 (3.03%) Apache Flume
4 50 (2.97%) VirtualBox
5 41 (2.44%) Apache ZooKeeper
6 3 (0.18%) Varnish
7 2 (0.12%) Active Directory
8 1 (0.059%) Alluxio
8 1 (0.059%) Ceph
8 1 (0.059%) etcd
8 1 (0.059%) Gluster
8 1 (0.059%) Hyper-V
8 1 (0.059%) VMware Infrastructure
Systems Management
1 340 (20.20%) Kubernetes
2 183 (10.87%) Ansible
3 146 (8.67%) Terraform
4 87 (5.17%) Oozie
5 72 (4.28%) Puppet
6 56 (3.33%) Docker Swarm
7 48 (2.85%) Hadoop YARN
8 32 (1.90%) Salt
9 27 (1.60%) Opscode Chef
10 14 (0.83%) Mesos
11 13 (0.77%) Prometheus
12 10 (0.59%) Ambari
12 10 (0.59%) Grafana
13 9 (0.53%) Kibana
13 9 (0.53%) Single Sign-On
14 6 (0.36%) Chronos
14 6 (0.36%) Consul
14 6 (0.36%) Nagios
15 5 (0.30%) BMC PATROL
15 5 (0.30%) Tivoli
Vendors
1 257 (15.27%) Google
2 148 (8.79%) Tableau
3 128 (7.61%) Cloudera
4 120 (7.13%) Oracle
5 107 (6.36%) Microsoft
6 103 (6.12%) Hortonworks
7 91 (5.41%) Databricks
8 89 (5.29%) Informatica
9 83 (4.93%) Talend
10 69 (4.10%) IBM
11 35 (2.08%) Teradata
12 34 (2.02%) Splunk
13 25 (1.49%) WorldPay
14 21 (1.25%) Sybase
15 20 (1.19%) FIS
16 19 (1.13%) SAP
17 18 (1.07%) Dataiku
17 18 (1.07%) MapR
18 17 (1.01%) TIBCO
19 16 (0.95%) Vertica