Period
to 25 February 2020

The following table provides summary statistics for permanent job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 25 February 2020 with a comparison to the same period in the previous 2 years.

Apache Spark
UK > England
6 months to
25 Feb 2020
Same period 2019 Same period 2018
Rank 231 238 239
Rank change year-on-year +7 +1 +69
Permanent jobs citing Apache Spark 1,622 2,060 2,404
As % of all permanent IT jobs advertised in England 1.42% 1.47% 1.47%
As % of the Application Platforms category 9.80% 9.38% 8.70%
Number of salaries quoted 1,320 1,618 1,994
England median annual salary £72,500 £75,000 £70,000
Median salary % change year-on-year -3.33% +7.14% +3.70%
10th Percentile £48,750 £47,500 £42,500
90th Percentile £105,000 £106,250 £105,000
UK median annual salary £72,500 £72,500 £70,000
% change year-on-year - +3.57% +3.70%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Permanent vacancies with a requirement for application platform skills 16,547 21,970 27,619
As % of all permanent IT jobs advertised in England 14.51% 15.73% 16.94%
Number of salaries quoted 13,379 17,640 22,995
England median annual salary £50,500 £50,000 £50,000
Median salary % change year-on-year +1.00% - +7.53%
10th Percentile £28,000 £28,298 £27,500
90th Percentile £85,000 £85,000 £80,000
UK median annual salary £50,000 £50,000 £47,500
% change year-on-year - +5.26% +5.56%

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a percentage of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Salary Trend in England

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in England.

Salary trend for Apache Spark in England

Apache Spark
Salary Histogram in England

The salary distribution of IT jobs citing Apache Spark in England over the 6 months to 25 February 2020.

Salary histogram for Apache Spark in England

Apache Spark
Job Locations in England

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the England region over the 6 months to 25 February 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
London -1 1,138 £76,500 -4.38% 214
South East +19 149 £60,000 -11.11% 32
North of England +15 118 £55,000 -15.38% 14
Midlands +60 111 £72,500 -3.33% 14
East Midlands +45 81 £72,500 -3.33% 5
North West +10 56 £57,500 -17.86% 8
East of England +56 55 £75,000 +3.45% 8
Yorkshire +38 55 £55,000 +10.73% 6
South West -58 43 £52,500 -4.98% 13
West Midlands +40 30 £60,000 +140.00% 9
North East +3 7 £54,671 -27.10%
Apache Spark
UK

For the 6 months to 25 February 2020, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the England region with a requirement for Apache Spark.

1 1,026 (63.26%) Python
2 998 (61.53%) Hadoop
3 989 (60.97%) Big Data
4 752 (46.36%) AWS
5 720 (44.39%) Java
6 674 (41.55%) Agile Software Development
7 655 (40.38%) SQL
8 638 (39.33%) Analytics
9 635 (39.15%) Scala
10 620 (38.22%) Data Science
11 576 (35.51%) Kafka
12 473 (29.16%) Machine Learning
13 459 (28.30%) Azure
14 395 (24.35%) NoSQL
15 391 (24.11%) Docker
16 389 (23.98%) GCP
17 377 (23.24%) Apache Hive
18 365 (22.50%) ETL
19 347 (21.39%) DevOps
20 334 (20.59%) Kubernetes
21 327 (20.16%) Data Engineering
22 305 (18.80%) Software Engineering
23 304 (18.74%) Finance
24 277 (17.08%) Degree
25 269 (16.58%) Data Warehouse
26 267 (16.46%) Business Intelligence
27 263 (16.21%) Data Modelling
28 262 (16.15%) Linux
29 260 (16.03%) Artificial Intelligence
30 259 (15.97%) Apache

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 259 (15.97%) Apache
2 161 (9.93%) Elasticsearch
3 94 (5.80%) Apache Airflow
4 48 (2.96%) Luigi
5 44 (2.71%) Apache Solr
6 32 (1.97%) SAS
7 27 (1.66%) Jupyter
8 24 (1.48%) Confluence
9 20 (1.23%) Apache Pig
10 15 (0.92%) nginx
11 13 (0.80%) OpenStack
12 6 (0.37%) IPython
13 5 (0.31%) Hyperledger
13 5 (0.31%) SharePoint
14 4 (0.25%) DataSynapse
14 4 (0.25%) JBoss
14 4 (0.25%) Oracle Workflow
14 4 (0.25%) WildFly
15 3 (0.18%) Oracle SOA Suite
15 3 (0.18%) Tomcat
Applications
1 8 (0.49%) SPSS
2 5 (0.31%) Gephi
2 5 (0.31%) Microsoft Excel
3 2 (0.12%) Chatbot
4 1 (0.062%) Microsoft Office
4 1 (0.062%) Microsoft PowerPoint
4 1 (0.062%) Weka
Business Applications
1 6 (0.37%) Sparx Enterprise Architect
2 2 (0.12%) Eikon
2 2 (0.12%) SAP BPC
2 2 (0.12%) SAP S/4HANA
3 1 (0.062%) Adobe Campaign
3 1 (0.062%) Magento
3 1 (0.062%) NetSuite
Cloud Services
1 752 (46.36%) AWS
2 459 (28.30%) Azure
3 389 (23.98%) GCP
4 137 (8.45%) Amazon EMR
5 131 (8.08%) Amazon S3
6 102 (6.29%) Power Platform
7 90 (5.55%) Amazon Kinesis
8 87 (5.36%) Serverless
9 64 (3.95%) AWS Lambda
10 52 (3.21%) PaaS
11 47 (2.90%) Amazon EC2
12 39 (2.40%) SaaS
13 38 (2.34%) Cloud Computing
13 38 (2.34%) IBM Cloud
14 34 (2.10%) OpenShift
15 30 (1.85%) GitHub
16 29 (1.79%) Azure Data Factory
17 27 (1.66%) Amazon SageMaker
17 27 (1.66%) Azure Machine Learning
18 16 (0.99%) Amazon ECS
Communications & Networking
1 32 (1.97%) HTTP
2 21 (1.29%) Firewall
3 14 (0.86%) HTTPS
4 12 (0.74%) TCP/IP
4 12 (0.74%) VPN
5 9 (0.55%) Internet
5 9 (0.55%) SSL
6 8 (0.49%) Kerberos
7 6 (0.37%) 5G
7 6 (0.37%) DNS
7 6 (0.37%) FTP
7 6 (0.37%) Intrusion Detection
8 5 (0.31%) 3G
8 5 (0.31%) 4G
8 5 (0.31%) Ethernet
8 5 (0.31%) MPLS
8 5 (0.31%) Wi-Fi
9 2 (0.12%) LTE
9 2 (0.12%) SAN
9 2 (0.12%) WLAN
Database & Business Intelligence
1 998 (61.53%) Hadoop
2 989 (60.97%) Big Data
3 395 (24.35%) NoSQL
4 377 (23.24%) Apache Hive
5 269 (16.58%) Data Warehouse
6 222 (13.69%) Amazon Redshift
7 208 (12.82%) PostgreSQL
8 194 (11.96%) Apache Cassandra
9 171 (10.54%) Relational Database
10 159 (9.80%) HBase
11 158 (9.74%) MongoDB
12 153 (9.43%) SQL Server
13 139 (8.57%) Impala
14 128 (7.89%) Data Lake
15 122 (7.52%) Neo4j
16 119 (7.34%) BigQuery
17 102 (6.29%) Power BI
18 99 (6.10%) Flink
19 86 (5.30%) Amazon Athena
20 75 (4.62%) Snowflake Schema
Development Applications
1 230 (14.18%) Git (software)
2 176 (10.85%) Jenkins
3 100 (6.17%) JIRA
4 69 (4.25%) Atlassian Bamboo
5 51 (3.14%) Vagrant
6 39 (2.40%) JUnit
7 32 (1.97%) Maven
8 21 (1.29%) Bitbucket
8 21 (1.29%) git-flow
9 19 (1.17%) Subversion
9 19 (1.17%) Visual Studio Team System
10 16 (0.99%) Visual Studio
11 14 (0.86%) TeamCity
12 13 (0.80%) Gradle
13 12 (0.74%) Mockito
14 11 (0.68%) Eclipse
14 11 (0.68%) IntelliJ
14 11 (0.68%) kappa
15 10 (0.62%) Mercurial
15 10 (0.62%) Sonatype Nexus
General
1 304 (18.74%) Finance
2 101 (6.23%) Retail
3 95 (5.86%) Banking
4 58 (3.58%) Marketing
5 47 (2.90%) Public Sector
6 43 (2.65%) Manufacturing
6 43 (2.65%) Telecoms
7 42 (2.59%) Automotive
8 41 (2.53%) Front Office
9 37 (2.28%) Legal
10 34 (2.10%) Games
11 17 (1.05%) Advertising
12 10 (0.62%) Investment Banking
13 9 (0.55%) Financial Institution
14 8 (0.49%) Billing
14 8 (0.49%) Electronics
15 6 (0.37%) Online Betting
15 6 (0.37%) Publishing
16 5 (0.31%) Back Office
16 5 (0.31%) Retail Banking
Job Titles
1 501 (30.89%) Data Engineer
2 374 (23.06%) Developer
3 209 (12.89%) Java Developer
4 205 (12.64%) Architect
5 154 (9.49%) Data Architect
6 145 (8.94%) Data Scientist
7 115 (7.09%) Senior Developer
8 96 (5.92%) Big Data Engineer
9 83 (5.12%) Senior Data Scientist
10 80 (4.93%) Senior Data Engineer
11 79 (4.87%) Senior Java Developer
12 78 (4.81%) Software Engineer
13 71 (4.38%) Scala Developer
14 65 (4.01%) Python Engineer
15 55 (3.39%) Java Engineer
16 48 (2.96%) Big Data Architect
16 48 (2.96%) Python Developer
17 46 (2.84%) Analyst
17 46 (2.84%) Big Data Developer
18 45 (2.77%) DevOps Engineer
Libraries, Frameworks & Software Standards
1 576 (35.51%) Kafka
2 120 (7.40%) TensorFlow
3 116 (7.15%) Natural Language Processing
4 113 (6.97%) REST
5 111 (6.84%) Node.js
6 104 (6.41%) Spring
7 102 (6.29%) Web Services
8 96 (5.92%) React
9 93 (5.73%) Spark Streaming
10 83 (5.12%) scikit-learn
11 81 (4.99%) AngularJS
12 72 (4.44%) RESTful
13 58 (3.58%) HTML
14 56 (3.45%) Pandas
15 54 (3.33%) Akka
16 52 (3.21%) Spring Boot
17 50 (3.08%) .NET
18 44 (2.71%) Dask
19 42 (2.59%) HTML5
19 42 (2.59%) RabbitMQ
Miscellaneous
1 234 (14.43%) Analytical Skills
2 169 (10.42%) Algorithms
3 92 (5.67%) Greenfield Project
4 90 (5.55%) Data Structures
5 63 (3.88%) Public Cloud
6 42 (2.59%) Blog
7 41 (2.53%) Management Information System
8 39 (2.40%) Self-Motivation
9 33 (2.03%) Acoustics
10 27 (1.66%) User Experience
11 26 (1.60%) Hedge funds
11 26 (1.60%) Mobile App
12 25 (1.54%) Cloud Native
12 25 (1.54%) Virtual Team
13 18 (1.11%) Video on Demand
14 16 (0.99%) Social Media
15 13 (0.80%) Enterprise Software
16 12 (0.74%) Cybercrime
16 12 (0.74%) Data Centre
17 11 (0.68%) Derivative
Operating Systems
1 262 (16.15%) Linux
2 70 (4.32%) Unix
3 27 (1.66%) Windows
4 13 (0.80%) Android
4 13 (0.80%) Apple iOS
5 7 (0.43%) VMS
6 3 (0.18%) Red Hat Enterprise Linux
7 2 (0.12%) Windows Server
8 1 (0.062%) CoreOS
8 1 (0.062%) Ubuntu
Processes & Methodologies
1 674 (41.55%) Agile Software Development
2 638 (39.33%) Analytics
3 620 (38.22%) Data Science
4 473 (29.16%) Machine Learning
5 365 (22.50%) ETL
6 347 (21.39%) DevOps
7 327 (20.16%) Data Engineering
8 305 (18.80%) Software Engineering
9 267 (16.46%) Business Intelligence
10 263 (16.21%) Data Modelling
11 260 (16.03%) Artificial Intelligence
12 239 (14.73%) Computer Science
12 239 (14.73%) Data Analytics
13 220 (13.56%) Microservices
14 217 (13.38%) Visualisation
15 165 (10.17%) Continuous Integration
16 151 (9.31%) Fintech
16 151 (9.31%) TDD
17 150 (9.25%) Scrum
18 145 (8.94%) Statistics
Programming Languages
1 1,026 (63.26%) Python
2 720 (44.39%) Java
3 655 (40.38%) SQL
4 635 (39.15%) Scala
5 191 (11.78%) JavaScript
6 187 (11.53%) R
7 122 (7.52%) C++
8 114 (7.03%) C#
9 80 (4.93%) Bash Shell
10 76 (4.69%) Go
11 62 (3.82%) C
12 41 (2.53%) Java 8
12 41 (2.53%) PHP
12 41 (2.53%) Shell Script
13 38 (2.34%) Kotlin
14 33 (2.03%) TypeScript
15 30 (1.85%) MATLAB
16 27 (1.66%) PL/SQL
17 22 (1.36%) ES6
18 18 (1.11%) Ruby
Qualifications
1 277 (17.08%) Degree
2 120 (7.40%) Computer Science Degree
3 72 (4.44%) PhD
4 59 (3.64%) Security Cleared
5 47 (2.90%) SC Cleared
6 39 (2.40%) DV Cleared
7 14 (0.86%) Numerate Degree
8 13 (0.80%) CCNA
8 13 (0.80%) Cisco Certification
8 13 (0.80%) CISSP
8 13 (0.80%) CompTIA Security+
8 13 (0.80%) GIAC
8 13 (0.80%) Linux+ Certification
8 13 (0.80%) Master's Degree
8 13 (0.80%) MCSE
8 13 (0.80%) Microsoft Certification
8 13 (0.80%) Network+ Certification
8 13 (0.80%) SANS
8 13 (0.80%) SSCP
9 12 (0.74%) AWS Certification
Quality Assurance & Compliance
1 93 (5.73%) Data Quality
2 58 (3.58%) QA
3 25 (1.54%) GDPR
4 8 (0.49%) ISO/IEC 27001
5 4 (0.25%) MiFID
6 3 (0.18%) CDISC
6 3 (0.18%) EMIR
6 3 (0.18%) HL7
6 3 (0.18%) ISO 26262
7 2 (0.12%) SLA
8 1 (0.062%) Actionable Recommendations
8 1 (0.062%) Sarbanes-Oxley
System Software
1 391 (24.11%) Docker
2 105 (6.47%) HDFS
3 51 (3.14%) Apache Flume
3 51 (3.14%) VirtualBox
4 41 (2.53%) Apache ZooKeeper
5 3 (0.18%) Varnish
6 2 (0.12%) Active Directory
7 1 (0.062%) Alluxio
7 1 (0.062%) Ceph
7 1 (0.062%) Cronacle
7 1 (0.062%) etcd
7 1 (0.062%) Gluster
7 1 (0.062%) Hyper-V
7 1 (0.062%) VMware Infrastructure
Systems Management
1 334 (20.59%) Kubernetes
2 183 (11.28%) Ansible
3 143 (8.82%) Terraform
4 83 (5.12%) Oozie
5 70 (4.32%) Puppet
6 57 (3.51%) Docker Swarm
7 44 (2.71%) Hadoop YARN
8 32 (1.97%) Salt
9 26 (1.60%) Opscode Chef
10 14 (0.86%) Mesos
11 13 (0.80%) Prometheus
12 10 (0.62%) Grafana
13 8 (0.49%) Ambari
13 8 (0.49%) Kibana
14 7 (0.43%) Single Sign-On
15 6 (0.37%) Chronos
15 6 (0.37%) Consul
15 6 (0.37%) Nagios
16 5 (0.31%) HP Quality Center
16 5 (0.31%) Tivoli
Vendors
1 254 (15.66%) Google
2 145 (8.94%) Tableau
3 125 (7.71%) Cloudera
4 116 (7.15%) Oracle
5 101 (6.23%) Microsoft
6 100 (6.17%) Hortonworks
7 89 (5.49%) Informatica
8 83 (5.12%) Databricks
9 81 (4.99%) Talend
10 66 (4.07%) IBM
11 34 (2.10%) Teradata
12 30 (1.85%) Splunk
13 25 (1.54%) WorldPay
14 20 (1.23%) FIS
15 19 (1.17%) Dataiku
16 18 (1.11%) Sybase
17 16 (0.99%) SAP
17 16 (0.99%) Vertica
18 15 (0.92%) MapR
19 14 (0.86%) TIBCO