Period
to 15 October 2019

The following table provides summary statistics for permanent job vacancies advertised in the UK excluding London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 15 October 2019 with a comparison to the same period in the previous 2 years.

Apache Spark
UK > UK excluding London
6 months to
15 Oct 2019
Same period 2018 Same period 2017
Rank 336 344 403
Rank change year-on-year +8 +59 +91
Permanent jobs citing Apache Spark 676 791 676
As % of all permanent IT jobs advertised in the UK excluding London 0.86% 0.86% 0.70%
As % of the Application Platforms category 6.02% 5.48% 4.12%
Number of salaries quoted 540 625 508
UK excluding London median annual salary £59,750 £60,000 £55,000
Median salary % change year-on-year -0.42% +9.09% -
10th Percentile £42,450 £40,000 £32,500
90th Percentile £80,000 £87,500 £80,000
UK median annual salary £72,500 £70,000 £65,000
% change year-on-year +3.57% +7.69% -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in the UK excluding London with a requirement for application platform skills.

All Application Platform Skills
UK excluding London
Permanent vacancies with a requirement for application platform skills 11,225 14,439 16,411
As % of all permanent IT jobs advertised in the UK excluding London 14.24% 15.63% 17.08%
Number of salaries quoted 8,743 11,622 13,454
UK excluding London median annual salary £42,943 £43,692 £40,000
Median salary % change year-on-year -1.71% +9.23% -
10th Percentile £26,250 £26,000 £25,250
90th Percentile £68,750 £67,500 £65,000
UK median annual salary £50,000 £50,000 £47,500
% change year-on-year - +5.26% +5.56%

Apache Spark
Job Vacancy Trend in the UK excluding London

Job postings citing Apache Spark as a percentage of all IT jobs advertised in the UK excluding London.

Job vacancy trend for Apache Spark in the UK excluding London

Apache Spark
Salary Trend in the UK excluding London

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in the UK excluding London.

Salary trend for Apache Spark in the UK excluding London

Apache Spark
Salary Histogram in the UK excluding London

The salary distribution of IT jobs citing Apache Spark in the UK excluding London over the 6 months to 15 October 2019.

Salary histogram for Apache Spark in the UK excluding London

Apache Spark
Job Locations in the UK excluding London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the UK excluding London region over the 6 months to 15 October 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
South East -32 163 £60,000 - 20
South West +55 161 £55,000 -8.33% 13
North of England +32 137 £60,000 -7.69% 15
Midlands +19 86 £65,000 -10.34% 6
North West +17 70 £65,000 -7.14% 8
East Midlands +18 69 £57,500 -23.33% 4
Scotland +8 64 £57,500 +4.55% 14
East of England +28 61 £65,000 -1.52% 7
Yorkshire +32 56 £58,638 +16.11% 5
West Midlands +16 17 £70,000 +16.67% 2
North East +25 11 £53,638 -46.36% 2
Wales -7 3 £42,500 -26.72% 2
Channel Islands - 1 £37,500 -
Apache Spark
UK

For the 6 months to 15 October 2019, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the UK excluding London region with a requirement for Apache Spark.

1 449 (66.42%) Hadoop
2 441 (65.24%) Big Data
3 388 (57.40%) Python
4 316 (46.75%) Agile Software Development
5 264 (39.05%) Analytics
6 263 (38.91%) Java
7 257 (38.02%) Apache Hive
8 254 (37.57%) Scala
9 223 (32.99%) SQL
10 215 (31.80%) Data Science
11 198 (29.29%) Kafka
12 189 (27.96%) AWS
13 171 (25.30%) ETL
14 168 (24.85%) Git (software)
15 156 (23.08%) Machine Learning
16 146 (21.60%) Analytical Skills
17 144 (21.30%) Azure
18 143 (21.15%) Impala
19 140 (20.71%) Scrum
20 139 (20.56%) Linux
21 133 (19.67%) Docker
22 132 (19.53%) Software Engineering
23 131 (19.38%) Data Warehouse
24 129 (19.08%) Degree
24 129 (19.08%) Visualisation
25 127 (18.79%) R
26 126 (18.64%) MapReduce
27 125 (18.49%) NoSQL
28 115 (17.01%) Data Engineering
29 111 (16.42%) Apache

Apache Spark
Co-occurring IT Skills in the UK excluding London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 111 (16.42%) Apache
2 91 (13.46%) Elasticsearch
3 35 (5.18%) OpenStack
4 23 (3.40%) Apache Pig
4 23 (3.40%) Apache Solr
4 23 (3.40%) Hyperledger
5 18 (2.66%) SAS
6 12 (1.78%) Confluence
7 8 (1.18%) nginx
8 4 (0.59%) JBoss
8 4 (0.59%) Jupyter
9 2 (0.30%) Apache Airflow
9 2 (0.30%) Luigi
9 2 (0.30%) Tornado
9 2 (0.30%) WebLogic
9 2 (0.30%) WebSphere
10 1 (0.15%) CMS
10 1 (0.15%) Oracle SOA Suite
10 1 (0.15%) Oracle Workflow
10 1 (0.15%) WebLogic Portal
Applications
1 7 (1.04%) Gephi
2 6 (0.89%) Microsoft Office
3 2 (0.30%) Microsoft Excel
3 2 (0.30%) Microsoft PowerPoint
4 1 (0.15%) MS Visio
4 1 (0.15%) Weka
Business Applications
1 1 (0.15%) Magento
1 1 (0.15%) Sparx Enterprise Architect
Cloud Services
1 189 (27.96%) AWS
2 144 (21.30%) Azure
3 96 (14.20%) GCP
4 35 (5.18%) OpenShift
5 33 (4.88%) Power Platform
6 31 (4.59%) Amazon EMR
7 24 (3.55%) Azure Data Factory
8 23 (3.40%) Amazon S3
9 14 (2.07%) SaaS
10 13 (1.92%) Cloud Computing
11 11 (1.63%) Amazon Kinesis
11 11 (1.63%) GitHub
12 8 (1.18%) Azure Machine Learning
12 8 (1.18%) Serverless
13 7 (1.04%) Amazon EC2
13 7 (1.04%) Amazon SQS
13 7 (1.04%) PaaS
14 6 (0.89%) Data Lake Analytics
14 6 (0.89%) Dynamics 365
14 6 (0.89%) IaaS
Communications & Networking
1 35 (5.18%) Firewall
2 34 (5.03%) HTTPS
3 33 (4.88%) TCP/IP
4 13 (1.92%) HTTP
4 13 (1.92%) SSL
5 8 (1.18%) DNS
5 8 (1.18%) VPN
6 7 (1.04%) Ethernet
6 7 (1.04%) MPLS
7 5 (0.74%) Kerberos
8 4 (0.59%) Broadband
9 2 (0.30%) Internet
10 1 (0.15%) Network Security
10 1 (0.15%) VXLAN
Database & Business Intelligence
1 449 (66.42%) Hadoop
2 441 (65.24%) Big Data
3 257 (38.02%) Apache Hive
4 143 (21.15%) Impala
5 131 (19.38%) Data Warehouse
6 125 (18.49%) NoSQL
7 107 (15.83%) HBase
8 96 (14.20%) MongoDB
9 91 (13.46%) Apache Cassandra
10 82 (12.13%) Data Lake
11 54 (7.99%) SQL Server
12 52 (7.69%) Relational Database
13 42 (6.21%) PostgreSQL
14 34 (5.03%) Amazon Redshift
14 34 (5.03%) MySQL
15 33 (4.88%) Power BI
16 32 (4.73%) Sqoop
17 23 (3.40%) Blockchain
18 22 (3.25%) BigQuery
18 22 (3.25%) Redis
Development Applications
1 168 (24.85%) Git (software)
2 103 (15.24%) JIRA
3 95 (14.05%) Bitbucket
4 69 (10.21%) Jenkins
5 65 (9.62%) git-flow
6 32 (4.73%) Bitbucket Server
7 22 (3.25%) Cucumber
8 17 (2.51%) Selenium
9 16 (2.37%) Team Foundation Server
10 13 (1.92%) Eclipse
11 12 (1.78%) IntelliJ
12 11 (1.63%) Subversion
13 10 (1.48%) Maven
13 10 (1.48%) Visual Studio
14 9 (1.33%) Visual Studio Team System
15 8 (1.18%) ERwin
15 8 (1.18%) GitLab
15 8 (1.18%) Mercurial
16 7 (1.04%) NetBeans
17 6 (0.89%) HP UFT
General
1 106 (15.68%) Finance
2 43 (6.36%) Telecoms
3 35 (5.18%) Public Sector
4 28 (4.14%) Retail
5 22 (3.25%) Manufacturing
6 16 (2.37%) Games
7 15 (2.22%) Marketing
8 14 (2.07%) Electronics
9 4 (0.59%) Aerospace
9 4 (0.59%) Automotive
9 4 (0.59%) Banking
10 3 (0.44%) Advertising
10 3 (0.44%) Legal
10 3 (0.44%) Online Betting
11 1 (0.15%) Financial Institution
11 1 (0.15%) French Language
11 1 (0.15%) German Language
11 1 (0.15%) Investment Banking
11 1 (0.15%) Publishing
11 1 (0.15%) Retail Banking
Job Titles
1 197 (29.14%) Data Engineer
2 182 (26.92%) Developer
3 92 (13.61%) Java Developer
4 83 (12.28%) Big Data Engineer
5 77 (11.39%) Data Scientist
6 63 (9.32%) Senior Data Engineer
7 62 (9.17%) Consultant
8 61 (9.02%) Senior Developer
9 57 (8.43%) Software Engineer
10 47 (6.95%) Agile Developer
10 47 (6.95%) Agile Java Developer
11 46 (6.80%) Senior Java Developer
12 44 (6.51%) Architect
13 35 (5.18%) Big Data Developer
13 35 (5.18%) Data Architect
14 33 (4.88%) Data Consultant
15 30 (4.44%) Analyst
16 28 (4.14%) Python Engineer
17 27 (3.99%) Data Analyst
18 21 (3.11%) Technical Leader
Libraries, Frameworks & Software Standards
1 198 (29.29%) Kafka
2 92 (13.61%) Spring
3 69 (10.21%) Node.js
4 55 (8.14%) React
5 51 (7.54%) AngularJS
6 45 (6.66%) REST
7 42 (6.21%) Elastic Stack
8 41 (6.07%) .NET
9 36 (5.33%) RESTful
10 35 (5.18%) Pandas
11 34 (5.03%) ASP.NET
11 34 (5.03%) Spring Cloud
12 33 (4.88%) GraphQL
12 33 (4.88%) scikit-learn
12 33 (4.88%) Spring MVC
12 33 (4.88%) Vue.js
13 32 (4.73%) Akka
14 26 (3.85%) OAuth
14 26 (3.85%) RabbitMQ
15 25 (3.70%) TensorFlow
Miscellaneous
1 146 (21.60%) Analytical Skills
2 68 (10.06%) Algorithms
3 35 (5.18%) Analytical Mindset
3 35 (5.18%) Cyber Kill Chain
4 30 (4.44%) Private Cloud
5 29 (4.29%) Public Cloud
6 27 (3.99%) Management Information System
7 24 (3.55%) Cybercrime
8 22 (3.25%) Data Structures
9 19 (2.81%) Video on Demand
10 18 (2.66%) Greenfield Project
11 16 (2.37%) Cyberattack
12 10 (1.48%) Social Media
13 9 (1.33%) Self-Motivation
13 9 (1.33%) User Experience
14 8 (1.18%) Hybrid Cloud
15 7 (1.04%) Enterprise Software
16 6 (0.89%) CUDA
16 6 (0.89%) Linked Data
16 6 (0.89%) Mainframe
Operating Systems
1 139 (20.56%) Linux
2 54 (7.99%) Unix
3 33 (4.88%) Android
3 33 (4.88%) Apple iOS
4 29 (4.29%) Windows
5 6 (0.89%) Red Hat Enterprise Linux
6 3 (0.44%) CentOS
7 2 (0.30%) SUSE Linux Enterprise Server
8 1 (0.15%) CoreOS
Processes & Methodologies
1 316 (46.75%) Agile Software Development
2 264 (39.05%) Analytics
3 215 (31.80%) Data Science
4 171 (25.30%) ETL
5 156 (23.08%) Machine Learning
6 140 (20.71%) Scrum
7 132 (19.53%) Software Engineering
8 129 (19.08%) Visualisation
9 126 (18.64%) MapReduce
10 115 (17.01%) Data Engineering
11 100 (14.79%) Business Intelligence
12 88 (13.02%) Data Modelling
13 87 (12.87%) Kanban
14 86 (12.72%) Cybersecurity
15 85 (12.57%) DevOps
16 80 (11.83%) Mathematics
17 79 (11.69%) Artificial Intelligence
18 77 (11.39%) Data Analytics
19 74 (10.95%) Microservices
19 74 (10.95%) Problem-Solving
Programming Languages
1 388 (57.40%) Python
2 263 (38.91%) Java
3 254 (37.57%) Scala
4 223 (32.99%) SQL
5 127 (18.79%) R
6 93 (13.76%) C
7 77 (11.39%) C#
8 52 (7.69%) Bash Shell
9 50 (7.40%) JavaScript
10 43 (6.36%) Perl
11 38 (5.62%) C++
12 37 (5.47%) Shell Script
13 15 (2.22%) PowerShell
14 13 (1.92%) T-SQL
15 12 (1.78%) Go
15 12 (1.78%) PL/SQL
16 8 (1.18%) Ruby
16 8 (1.18%) TypeScript
17 7 (1.04%) Groovy
18 5 (0.74%) PHP
Qualifications
1 129 (19.08%) Degree
2 78 (11.54%) Security Cleared
3 36 (5.33%) DV Cleared
4 35 (5.18%) SC Cleared
5 33 (4.88%) CCNA
5 33 (4.88%) Cisco Certification
6 32 (4.73%) CISSP
6 32 (4.73%) CompTIA Security+
6 32 (4.73%) GIAC
6 32 (4.73%) Linux+ Certification
6 32 (4.73%) MCSE
6 32 (4.73%) Microsoft Certification
6 32 (4.73%) Network+ Certification
6 32 (4.73%) SANS
6 32 (4.73%) SSCP
7 25 (3.70%) Computer Science Degree
8 17 (2.51%) PhD
9 6 (0.89%) AWS Certification
10 3 (0.44%) Master's Degree
10 3 (0.44%) Numerate Degree
Quality Assurance & Compliance
1 38 (5.62%) Data Quality
2 23 (3.40%) QA
3 9 (1.33%) GDPR
4 3 (0.44%) Actionable Recommendations
5 2 (0.30%) PMO
6 1 (0.15%) IFRS
6 1 (0.15%) IFRS 9
6 1 (0.15%) ISO/IEC 27001
System Software
1 133 (19.67%) Docker
2 61 (9.02%) Apache Flume
2 61 (9.02%) HDFS
3 22 (3.25%) Apache ZooKeeper
4 3 (0.44%) Active Directory
5 1 (0.15%) Ceph
5 1 (0.15%) Hyper-V
5 1 (0.15%) Virtual Servers
5 1 (0.15%) VMware Infrastructure
Systems Management
1 76 (11.24%) Oozie
2 75 (11.09%) Kubernetes
3 36 (5.33%) Hadoop YARN
4 31 (4.59%) Ansible
5 18 (2.66%) Puppet
5 18 (2.66%) Terraform
6 10 (1.48%) Kibana
7 7 (1.04%) Docker Swarm
7 7 (1.04%) Opscode Chef
7 7 (1.04%) Salt
8 4 (0.59%) Ambari
9 3 (0.44%) Chukwa
9 3 (0.44%) HP Fortify
10 2 (0.30%) logstash
10 2 (0.30%) Single Sign-On
11 1 (0.15%) Grafana
11 1 (0.15%) Prometheus
Vendors
1 68 (10.06%) Informatica
2 66 (9.76%) Google
3 65 (9.62%) Oracle
4 43 (6.36%) Splunk
5 42 (6.21%) Teradata
6 40 (5.92%) Cloudera
7 38 (5.62%) Tableau
8 37 (5.47%) Microsoft
9 29 (4.29%) Databricks
9 29 (4.29%) Hortonworks
10 22 (3.25%) Ab Initio
11 19 (2.81%) SAP
12 16 (2.37%) Talend
13 15 (2.22%) IBM
14 13 (1.92%) Netezza
15 8 (1.18%) Red Hat
16 7 (1.04%) BT
16 7 (1.04%) Vertica
17 6 (0.89%) Alteryx
17 6 (0.89%) MapR