Job Vacancy Trend Salary Trend Salary Histogram Job Locations Skill Sets
Period
to 25 October 2020

The following table provides summary statistics for permanent job vacancies with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 25 October 2020 with a comparison to the same period in the previous 2 years.

Apache Spark
UK
6 months to
25 Oct 2020		 Same period 2019 Same period 2018
Rank 235 235 245
Rank change year-on-year 0 +10 +38
Permanent jobs citing Apache Spark 677 1,966 2,289
As % of all permanent jobs advertised in the UK 1.39% 1.44% 1.39%
As % of the Application Platforms category 9.90% 9.81% 8.51%
Number of salaries quoted 593 1,690 1,817
Median annual salary £70,000 £72,500 £70,000
Median salary % change year-on-year -3.45% +3.57% +7.69%
10th Percentile £42,500 £47,500 £45,000
90th Percentile £105,000 £110,000 £101,250
UK excluding London median annual salary £55,000 £59,750 £60,000
% change year-on-year -7.95% -0.42% +13.85%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for application platform skills.

All Application Platform Skills
UK
Permanent vacancies with a requirement for application platform skills 6,838 20,034 26,891
As % of all permanent IT jobs advertised in the UK 14.00% 14.71% 16.35%
Number of salaries quoted 5,637 16,002 21,565
Median annual salary £55,000 £50,000 £50,000
Median salary % change year-on-year +10.00% - +5.26%
10th Percentile £31,250 £27,500 £28,000
90th Percentile £90,000 £85,000 £82,500
UK excluding London median annual salary £45,000 £43,199 £43,633
% change year-on-year +4.17% -0.99% +9.08%

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a proportion of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Salary Trend

3-month moving average salary quoted in jobs citing Apache Spark.

Salary trend for Apache Spark in the UK

Apache Spark
Salary Histogram

Salary distribution for jobs citing Apache Spark over the 6 months to 25 October 2020.

Salary histogram for Apache Spark in the UK

Apache Spark
Top 15 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the UK over the 6 months to 25 October 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year		 Matching
Permanent
IT Job Ads		 Median Salary
Past 6 Months		 Median Salary
% Change
on Same Period
Last Year		 Live
Job
Vacancies
England -7 597 £72,500 - 138
London -10 420 £80,000 - 95
UK excluding London +50 246 £55,000 -7.95% 54
South East +137 80 £55,000 -8.33% 13
Work from Home +13 66 £65,000 +6.12% 19
Wales +85 40 £45,000 +5.88%
Scotland +72 36 £62,500 +8.70% 12
North of England +79 34 £60,000 - 14
North West +94 25 £58,750 -9.62% 9
East of England +101 23 £70,000 +7.69% 5
South West +6 17 £47,500 -13.64% 3
Midlands +61 16 £75,000 +7.14% 6
East Midlands +41 14 £75,000 +11.11% 3
Yorkshire +72 9 £60,000 - 5
West Midlands +93 2 £50,000 -28.57% 3

For the 6 months to 25 October 2020, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads with a requirement for Apache Spark.

1 448 (66.17%) Python
2 358 (52.88%) Hadoop
3 343 (50.66%) Big Data
4 306 (45.20%) AWS
4 306 (45.20%) Java
5 297 (43.87%) SQL
6 279 (41.21%) Kafka
7 233 (34.42%) Scala
8 217 (32.05%) Apache
9 211 (31.17%) Azure
10 202 (29.84%) DevOps
11 199 (29.39%) Finance
12 195 (28.80%) Analytics
13 193 (28.51%) Data Science
14 179 (26.44%) Agile Software Development
15 177 (26.14%) Machine Learning
15 177 (26.14%) Kubernetes
16 176 (26.00%) GCP
17 149 (22.01%) NoSQL
18 139 (20.53%) ETL
19 136 (20.09%) Degree
20 130 (19.20%) Docker
21 117 (17.28%) PostgreSQL
22 116 (17.13%) Data Warehouse
23 109 (16.10%) Terraform
24 108 (15.95%) Apache Hive
24 108 (15.95%) R
25 106 (15.66%) Spring
25 106 (15.66%) Elasticsearch
26 103 (15.21%) Microservices

Apache Spark
Co-occurring IT Skills by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 217 (32.05%) Apache
2 106 (15.66%) Elasticsearch
3 31 (4.58%) Apache Airflow
4 21 (3.10%) Tomcat
5 19 (2.81%) SAS
6 16 (2.36%) Cloud Foundry
6 16 (2.36%) Luigi
7 14 (2.07%) Jupyter
8 9 (1.33%) CMS
9 8 (1.18%) Apache Synapse
9 8 (1.18%) Confluence
10 7 (1.03%) Apache Pig
11 5 (0.74%) Tornado
12 4 (0.59%) nginx
13 3 (0.44%) Pachyderm
13 3 (0.44%) WebSphere
14 2 (0.30%) Apache Solr
15 1 (0.15%) BizTalk Server
15 1 (0.15%) GlassFish
15 1 (0.15%) Mule
Applications
1 9 (1.33%) Microsoft Excel
2 2 (0.30%) Microsoft PowerPoint
2 2 (0.30%) MS Visio
2 2 (0.30%) SPSS
2 2 (0.30%) SPSS Modeler
Business Applications
1 4 (0.59%) Eikon
2 3 (0.44%) Distributed Ledger
2 3 (0.44%) Oracle Cash Management
3 2 (0.30%) Sparx Enterprise Architect
4 1 (0.15%) Dynamics CRM
Cloud Services
1 306 (45.20%) AWS
2 211 (31.17%) Azure
3 176 (26.00%) GCP
4 66 (9.75%) GitHub
5 59 (8.71%) Serverless
6 58 (8.57%) Amazon S3
7 55 (8.12%) Stackdriver
8 47 (6.94%) Amazon EMR
9 46 (6.79%) Power Platform
10 34 (5.02%) Amazon Kinesis
11 33 (4.87%) AWS Lambda
12 29 (4.28%) SaaS
13 28 (4.14%) Amazon EC2
14 24 (3.55%) Azure Data Factory
15 21 (3.10%) Amazon SQS
16 20 (2.95%) Amazon CloudWatch
16 20 (2.95%) OpenShift
17 17 (2.51%) Amazon ECS
17 17 (2.51%) Amazon GuardDuty
17 17 (2.51%) AWS CloudTrail
Communications & Networking
1 14 (2.07%) HTTP
2 11 (1.62%) Kerberos
3 10 (1.48%) Firewall
4 9 (1.33%) TCP/IP
5 8 (1.18%) 5G
5 8 (1.18%) NetScaler
6 6 (0.89%) FTP
6 6 (0.89%) Internet
7 5 (0.74%) Parlay
8 3 (0.44%) AMQP
8 3 (0.44%) SSL
8 3 (0.44%) Wireless
9 2 (0.30%) DNS
9 2 (0.30%) HTTPS
9 2 (0.30%) Wi-Fi
10 1 (0.15%) 3G
10 1 (0.15%) 4G
10 1 (0.15%) Network Security
10 1 (0.15%) OpenVPN
10 1 (0.15%) VPN
Database & Business Intelligence
1 358 (52.88%) Hadoop
2 343 (50.66%) Big Data
3 149 (22.01%) NoSQL
4 117 (17.28%) PostgreSQL
5 116 (17.13%) Data Warehouse
6 108 (15.95%) Apache Hive
7 103 (15.21%) Data Lake
8 98 (14.48%) Apache Cassandra
9 89 (13.15%) Amazon Redshift
10 69 (10.19%) Tableau
11 64 (9.45%) SQL Server
12 60 (8.86%) MongoDB
13 54 (7.98%) Relational Database
14 50 (7.39%) Flink
15 46 (6.79%) Power BI
16 42 (6.20%) HBase
17 36 (5.32%) BigQuery
17 36 (5.32%) Impala
18 32 (4.73%) RDBMS
18 32 (4.73%) Snowflake Schema
Development Applications
1 86 (12.70%) Git (software)
2 66 (9.75%) Jenkins
3 58 (8.57%) Gradle
4 26 (3.84%) JIRA
4 26 (3.84%) TeamCity
5 17 (2.51%) Atlassian Bamboo
6 13 (1.92%) Cucumber
7 9 (1.33%) CircleCI
7 9 (1.33%) Sonatype Nexus
8 8 (1.18%) Maven
9 7 (1.03%) GitLab
9 7 (1.03%) Octopus Deploy
9 7 (1.03%) Selenium
10 6 (0.89%) Bitbucket
10 6 (0.89%) Travis CI
11 5 (0.74%) JUnit
12 4 (0.59%) CA Release Automation
12 4 (0.59%) GoCD
13 3 (0.44%) Team Foundation Server
14 2 (0.30%) ERwin
General
1 199 (29.39%) Finance
2 79 (11.67%) Banking
3 60 (8.86%) Public Sector
4 38 (5.61%) Marketing
5 26 (3.84%) Retail
6 25 (3.69%) Front Office
7 22 (3.25%) Advertising
7 22 (3.25%) Telecoms
8 17 (2.51%) Games
9 16 (2.36%) Investment Banking
9 16 (2.36%) Manufacturing
10 8 (1.18%) Pharmaceutical
11 7 (1.03%) Law
12 5 (0.74%) Electronics
12 5 (0.74%) Financial Institution
12 5 (0.74%) Retail Banking
13 3 (0.44%) Legal
14 2 (0.30%) Automotive
14 2 (0.30%) Private Banking
Job Titles
1 221 (32.64%) Data Engineer
2 195 (28.80%) Developer
3 73 (10.78%) Java Developer
4 71 (10.49%) Senior Developer
5 70 (10.34%) Data Scientist
6 61 (9.01%) Architect
7 56 (8.27%) Senior Data Engineer
8 44 (6.50%) Python Engineer
9 40 (5.91%) Big Data Engineer
9 40 (5.91%) Data Architect
10 37 (5.47%) Senior Data Scientist
11 33 (4.87%) AWS Engineer
12 28 (4.14%) Digital Developer
12 28 (4.14%) Digital Transformation
13 24 (3.55%) Scala Developer
13 24 (3.55%) Software Engineer
14 23 (3.40%) Java Engineer
15 21 (3.10%) Consultant
15 21 (3.10%) Senior Java Developer
16 20 (2.95%) Full Stack Developer
Libraries, Frameworks & Software Standards
1 279 (41.21%) Kafka
2 106 (15.66%) Spring
3 76 (11.23%) RabbitMQ
4 57 (8.42%) Spring Boot
5 55 (8.12%) Spring Integration
6 46 (6.79%) Natural Language Processing
6 46 (6.79%) TensorFlow
7 42 (6.20%) PySpark
7 42 (6.20%) scikit-learn
8 39 (5.76%) Node.js
9 34 (5.02%) React
10 33 (4.87%) Pandas
11 31 (4.58%) AngularJS
11 31 (4.58%) RESTful
12 29 (4.28%) NumPy
13 27 (3.99%) Web Services
14 25 (3.69%) Keras
15 24 (3.55%) REST
16 22 (3.25%) HTML
16 22 (3.25%) JSON
Miscellaneous
1 85 (12.56%) Algorithms
1 85 (12.56%) Analytical Skills
2 29 (4.28%) Cloud Native
2 29 (4.28%) Data Structures
3 28 (4.14%) Management Information System
4 24 (3.55%) Hedge funds
5 23 (3.40%) Self-Motivation
6 21 (3.10%) Equities
7 18 (2.66%) Data Centre
7 18 (2.66%) Greenfield Project
8 13 (1.92%) Blog
8 13 (1.92%) Hybrid Cloud
8 13 (1.92%) Market Risk
9 12 (1.77%) FMCG
10 10 (1.48%) Enterprise Software
11 9 (1.33%) Life Science
12 8 (1.18%) Robotics
13 7 (1.03%) Mobile App
13 7 (1.03%) Public Cloud
13 7 (1.03%) Replication
Operating Systems
1 81 (11.96%) Linux
2 26 (3.84%) Unix
3 17 (2.51%) Red Hat Enterprise Linux
4 9 (1.33%) CentOS
4 9 (1.33%) Windows
5 8 (1.18%) Windows Server
6 6 (0.89%) Ubuntu
7 2 (0.30%) DC/OS
8 1 (0.15%) Amazon Linux
8 1 (0.15%) Android
8 1 (0.15%) Apple iOS
Processes & Methodologies
1 202 (29.84%) DevOps
2 195 (28.80%) Analytics
3 193 (28.51%) Data Science
4 179 (26.44%) Agile Software Development
5 177 (26.14%) Machine Learning
6 139 (20.53%) ETL
7 103 (15.21%) Microservices
8 102 (15.07%) Artificial Intelligence
9 98 (14.48%) Visualisation
10 92 (13.59%) Business Intelligence
11 88 (13.00%) Data Engineering
11 88 (13.00%) Mathematics
12 86 (12.70%) Computer Science
13 73 (10.78%) Data Modelling
14 70 (10.34%) Software Engineering
15 69 (10.19%) Data Visualisation
16 66 (9.75%) Service Management
17 60 (8.86%) Statistics
18 56 (8.27%) Data Integration
19 55 (8.12%) Data Analytics
Programming Languages
1 448 (66.17%) Python
2 306 (45.20%) Java
3 297 (43.87%) SQL
4 233 (34.42%) Scala
5 108 (15.95%) R
6 69 (10.19%) JavaScript
7 52 (7.68%) C#
8 26 (3.84%) C++
9 24 (3.55%) Bash Shell
10 23 (3.40%) Go
11 18 (2.66%) Kotlin
12 16 (2.36%) Java 8
12 16 (2.36%) Shell Script
13 13 (1.92%) TypeScript
14 12 (1.77%) PHP
15 11 (1.62%) C
15 11 (1.62%) Ruby
16 10 (1.48%) PowerShell
17 8 (1.18%) Clojure
17 8 (1.18%) F#
Qualifications
1 136 (20.09%) Degree
2 68 (10.04%) Computer Science Degree
3 56 (8.27%) SC Cleared
3 56 (8.27%) Security Cleared
4 19 (2.81%) PhD
5 16 (2.36%) Master's Degree
6 12 (1.77%) AWS Certification
7 11 (1.62%) Numerate Degree
8 8 (1.18%) RHCE
9 5 (0.74%) Postgraduate
10 3 (0.44%) MCDBA
10 3 (0.44%) MCT
10 3 (0.44%) Microsoft Certification
10 3 (0.44%) Sun Certification
10 3 (0.44%) TOGAF Certification
11 1 (0.15%) HND
Quality Assurance & Compliance
1 20 (2.95%) GDPR
1 20 (2.95%) QA
2 14 (2.07%) Data Quality
3 3 (0.44%) ISO 20022
3 3 (0.44%) PSD2
3 3 (0.44%) TARGET2
4 2 (0.30%) ISO 9001
4 2 (0.30%) ISO/IEC 27001
4 2 (0.30%) Sarbanes-Oxley
5 1 (0.15%) IFRS
5 1 (0.15%) IFRS 9
5 1 (0.15%) ISO 26262
5 1 (0.15%) MiFID
5 1 (0.15%) PMO
5 1 (0.15%) SLA
System Software
1 130 (19.20%) Docker
2 28 (4.14%) HDFS
3 10 (1.48%) Apache ZooKeeper
4 9 (1.33%) Active Directory
5 8 (1.18%) Apache Flume
5 8 (1.18%) VMware Infrastructure
5 8 (1.18%) vSphere
5 8 (1.18%) XenApp
5 8 (1.18%) XenDesktop
6 4 (0.59%) Varnish
7 2 (0.30%) Virtual Machines
8 1 (0.15%) VirtualBox
Systems Management
1 177 (26.14%) Kubernetes
2 109 (16.10%) Terraform
3 55 (8.12%) Ansible
3 55 (8.12%) Grafana
4 25 (3.69%) Puppet
5 10 (1.48%) Oozie
5 10 (1.48%) Opscode Chef
6 8 (1.18%) Hadoop YARN
6 8 (1.18%) Kibana
7 4 (0.59%) Ambari
7 4 (0.59%) Mesos
7 4 (0.59%) Zipkin
8 3 (0.44%) Cobbler
8 3 (0.44%) Nagios
8 3 (0.44%) Red Hat Satellite
9 2 (0.30%) Marathon
9 2 (0.30%) Single Sign-On
10 1 (0.15%) Docker Swarm
10 1 (0.15%) Packer
10 1 (0.15%) Prometheus
Vendors
1 102 (15.07%) Google
2 84 (12.41%) Databricks
3 77 (11.37%) Splunk
4 55 (8.12%) ServiceNow
5 46 (6.79%) Oracle
6 34 (5.02%) Microsoft
7 33 (4.87%) Cloudera
8 26 (3.84%) Red Hat
9 17 (2.51%) Informatica
10 13 (1.92%) Hortonworks
11 12 (1.77%) Teradata
12 10 (1.48%) Talend
13 9 (1.33%) IBM
14 8 (1.18%) F5
14 8 (1.18%) Vertica
14 8 (1.18%) VMware
15 7 (1.03%) StreamSets
16 6 (0.89%) Ab Initio
16 6 (0.89%) SAP
17 5 (0.74%) MapR
157 Apache Spark job vacancies Nationwide