Period
to 14 November 2019

The following table provides summary statistics for permanent job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 14 November 2019 with a comparison to the same period in the previous 2 years.

Apache Spark
UK > England
6 months to
14 Nov 2019
Same period 2018 Same period 2017
Rank 226 250 269
Rank change year-on-year +24 +19 +92
Permanent jobs citing Apache Spark 1,885 2,127 2,235
As % of all permanent IT jobs advertised in England 1.49% 1.40% 1.32%
As % of the Application Platforms category 10.17% 8.58% 7.47%
Number of salaries quoted 1,583 1,639 1,803
England median annual salary £72,500 £72,500 £65,000
Median salary % change year-on-year - +11.54% -3.70%
10th Percentile £48,050 £45,000 £41,250
90th Percentile £107,500 £102,500 £102,500
UK median annual salary £72,500 £70,000 £65,000
% change year-on-year +3.57% +7.69% -3.70%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Permanent vacancies with a requirement for application platform skills 18,534 24,776 29,931
As % of all permanent IT jobs advertised in England 14.63% 16.27% 17.66%
Number of salaries quoted 14,999 20,076 24,427
England median annual salary £50,000 £50,000 £47,500
Median salary % change year-on-year - +5.26% +3.26%
10th Percentile £27,500 £28,000 £27,000
90th Percentile £85,000 £83,750 £77,500
UK median annual salary £50,000 £50,000 £47,500
% change year-on-year - +5.26% +5.56%

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a percentage of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Salary Trend in England

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in England.

Salary trend for Apache Spark in England

Apache Spark
Salary Histogram in England

The salary distribution of IT jobs citing Apache Spark in England over the 6 months to 14 November 2019.

Salary histogram for Apache Spark in England

Apache Spark
Job Locations in England

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the England region over the 6 months to 14 November 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
London +27 1,271 £80,000 +3.23% 152
South East -41 158 £60,000 - 27
South West +27 143 £55,000 -8.33% 8
North of England +34 135 £60,000 -14.29% 17
Midlands +45 100 £70,000 -6.67% 12
East Midlands +22 86 £72,500 -3.33% 7
East of England +36 64 £65,000 -1.52% 16
Yorkshire +28 63 £60,000 +18.81% 4
North West -5 62 £65,000 -7.14% 11
West Midlands +20 14 £60,000 +14.29% 5
North East +23 10 £54,835 -45.16% 2
Apache Spark
UK

For the 6 months to 14 November 2019, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the England region with a requirement for Apache Spark.

1 1,180 (62.60%) Python
2 1,141 (60.53%) Big Data
3 1,114 (59.10%) Hadoop
4 821 (43.55%) Java
5 794 (42.12%) AWS
6 775 (41.11%) Agile Software Development
7 733 (38.89%) SQL
8 715 (37.93%) Analytics
9 706 (37.45%) Scala
10 704 (37.35%) Data Science
11 647 (34.32%) Kafka
12 593 (31.46%) Machine Learning
13 580 (30.77%) Apache Hive
14 493 (26.15%) Docker
15 483 (25.62%) Azure
16 391 (20.74%) Linux
17 387 (20.53%) ETL
18 376 (19.95%) NoSQL
19 372 (19.73%) Kubernetes
20 371 (19.68%) GCP
21 370 (19.63%) DevOps
22 367 (19.47%) Software Engineering
23 348 (18.46%) Finance
24 346 (18.36%) Data Engineering
25 340 (18.04%) Git (software)
26 324 (17.19%) Analytical Skills
27 307 (16.29%) Google
28 301 (15.97%) R
29 293 (15.54%) Apache
30 288 (15.28%) Degree

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 293 (15.54%) Apache
2 208 (11.03%) Elasticsearch
3 82 (4.35%) Apache Airflow
4 53 (2.81%) Apache Pig
5 47 (2.49%) Apache Solr
6 44 (2.33%) SAS
7 42 (2.23%) OpenStack
8 35 (1.86%) Luigi
9 23 (1.22%) Jupyter
10 21 (1.11%) Confluence
10 21 (1.11%) Hyperledger
11 15 (0.80%) nginx
12 5 (0.27%) SharePoint
13 4 (0.21%) JBoss
14 3 (0.16%) Cloud Foundry
14 3 (0.16%) CMS
14 3 (0.16%) WebSphere
15 2 (0.11%) ArcGIS
15 2 (0.11%) Tomcat
15 2 (0.11%) Tornado
Applications
1 7 (0.37%) Gephi
1 7 (0.37%) Microsoft Excel
2 3 (0.16%) GNU Octave
3 2 (0.11%) Chatbot
3 2 (0.11%) Microsoft Office
3 2 (0.11%) Microsoft PowerPoint
3 2 (0.11%) SPSS
4 1 (0.053%) Microsoft Project
4 1 (0.053%) MS Visio
4 1 (0.053%) Weka
Business Applications
1 2 (0.11%) Eikon
1 2 (0.11%) Sparx Enterprise Architect
2 1 (0.053%) Adobe Campaign
2 1 (0.053%) Magento
2 1 (0.053%) NetSuite
2 1 (0.053%) SAP Oil and Gas
Cloud Services
1 794 (42.12%) AWS
2 483 (25.62%) Azure
3 371 (19.68%) GCP
4 113 (5.99%) Amazon EMR
5 98 (5.20%) Amazon S3
6 89 (4.72%) Serverless
7 83 (4.40%) Power Platform
8 72 (3.82%) Amazon Kinesis
9 69 (3.66%) AWS Lambda
10 66 (3.50%) PaaS
11 57 (3.02%) OpenShift
12 51 (2.71%) Cloud Computing
13 48 (2.55%) SaaS
14 43 (2.28%) GitHub
15 38 (2.02%) Azure Data Factory
16 35 (1.86%) Amazon EC2
17 29 (1.54%) IaaS
18 26 (1.38%) Amazon SageMaker
19 22 (1.17%) Azure Machine Learning
20 21 (1.11%) AWS CloudFormation
Communications & Networking
1 53 (2.81%) Firewall
2 46 (2.44%) HTTPS
3 42 (2.23%) HTTP
4 41 (2.18%) TCP/IP
5 22 (1.17%) Internet
6 18 (0.95%) VPN
7 17 (0.90%) SSL
8 16 (0.85%) Kerberos
9 9 (0.48%) DNS
10 8 (0.42%) Ethernet
10 8 (0.42%) MPLS
11 3 (0.16%) FTP
12 2 (0.11%) SMS
12 2 (0.11%) Unified Communications
13 1 (0.053%) 3G
13 1 (0.053%) 4G
13 1 (0.053%) 5G
13 1 (0.053%) Broadband
13 1 (0.053%) Intrusion Detection
13 1 (0.053%) Wi-Fi
Database & Business Intelligence
1 1,141 (60.53%) Big Data
2 1,114 (59.10%) Hadoop
3 580 (30.77%) Apache Hive
4 376 (19.95%) NoSQL
5 260 (13.79%) Data Warehouse
6 229 (12.15%) Apache Cassandra
6 229 (12.15%) HBase
6 229 (12.15%) Impala
7 197 (10.45%) PostgreSQL
8 189 (10.03%) MongoDB
9 156 (8.28%) Amazon Redshift
10 147 (7.80%) Relational Database
11 127 (6.74%) SQL Server
12 121 (6.42%) Data Lake
13 120 (6.37%) BigQuery
14 112 (5.94%) Flink
15 96 (5.09%) Data Mining
16 88 (4.67%) Neo4j
17 86 (4.56%) Sqoop
18 84 (4.46%) MySQL
Development Applications
1 340 (18.04%) Git (software)
2 209 (11.09%) Jenkins
3 162 (8.59%) JIRA
4 116 (6.15%) Bitbucket
5 84 (4.46%) git-flow
6 45 (2.39%) Bitbucket Server
7 44 (2.33%) Atlassian Bamboo
8 34 (1.80%) Vagrant
9 24 (1.27%) Maven
10 20 (1.06%) Eclipse
10 20 (1.06%) Gradle
10 20 (1.06%) IntelliJ
10 20 (1.06%) JUnit
10 20 (1.06%) Selenium
11 18 (0.95%) Cucumber
12 17 (0.90%) Subversion
13 16 (0.85%) Team Foundation Server
14 14 (0.74%) Visual Studio Team System
15 13 (0.69%) GitLab
15 13 (0.69%) RStudio
General
1 348 (18.46%) Finance
2 127 (6.74%) Retail
3 107 (5.68%) Banking
4 69 (3.66%) Marketing
5 67 (3.55%) Telecoms
6 56 (2.97%) Public Sector
7 42 (2.23%) Front Office
8 39 (2.07%) Manufacturing
9 38 (2.02%) Games
10 31 (1.64%) Advertising
11 29 (1.54%) Automotive
12 26 (1.38%) Financial Institution
12 26 (1.38%) Law
13 25 (1.33%) Investment Banking
14 21 (1.11%) Legal
15 17 (0.90%) Publishing
16 12 (0.64%) Electronics
17 10 (0.53%) Retail Banking
18 6 (0.32%) Online Betting
19 3 (0.16%) Music Industry
Job Titles
1 568 (30.13%) Data Engineer
2 451 (23.93%) Developer
3 247 (13.10%) Java Developer
4 213 (11.30%) Data Scientist
5 175 (9.28%) Architect
6 128 (6.79%) Data Architect
7 127 (6.74%) Senior Data Engineer
8 118 (6.26%) Big Data Engineer
9 113 (5.99%) Software Engineer
10 106 (5.62%) Senior Developer
11 104 (5.52%) Senior Data Scientist
12 101 (5.36%) Consultant
13 98 (5.20%) Scala Developer
14 96 (5.09%) Python Engineer
15 81 (4.30%) Big Data Developer
16 72 (3.82%) Senior Java Developer
17 65 (3.45%) Data Consultant
18 63 (3.34%) Python Developer
19 52 (2.76%) Lead Developer
20 51 (2.71%) Agile Developer
Libraries, Frameworks & Software Standards
1 647 (34.32%) Kafka
2 179 (9.50%) REST
3 137 (7.27%) TensorFlow
4 136 (7.21%) Web Services
5 133 (7.06%) Spring
6 122 (6.47%) React
7 121 (6.42%) scikit-learn
8 120 (6.37%) Node.js
8 120 (6.37%) Pandas
9 109 (5.78%) AngularJS
10 93 (4.93%) Natural Language Processing
11 89 (4.72%) Akka
12 82 (4.35%) Elastic Stack
13 79 (4.19%) RESTful
14 71 (3.77%) GraphQL
15 68 (3.61%) NumPy
16 67 (3.55%) HTML
17 65 (3.45%) .NET
17 65 (3.45%) JSON
18 60 (3.18%) Spark Streaming
Miscellaneous
1 324 (17.19%) Analytical Skills
2 268 (14.22%) Algorithms
3 95 (5.04%) Data Structures
4 74 (3.93%) Greenfield Project
5 63 (3.34%) Public Cloud
6 46 (2.44%) Cyber Kill Chain
7 45 (2.39%) Analytical Mindset
8 38 (2.02%) Management Information System
9 36 (1.91%) Self-Motivation
10 33 (1.75%) Linux Command Line
11 32 (1.70%) User Experience
12 30 (1.59%) Private Cloud
12 30 (1.59%) Virtual Team
13 27 (1.43%) Cybercrime
14 26 (1.38%) Blog
15 25 (1.33%) Derivative
16 23 (1.22%) Social Media
17 21 (1.11%) Video on Demand
18 20 (1.06%) Cloud Native
18 20 (1.06%) Mobile App
Operating Systems
1 391 (20.74%) Linux
2 131 (6.95%) Unix
3 41 (2.18%) Android
3 41 (2.18%) Apple iOS
3 41 (2.18%) Windows
4 6 (0.32%) Red Hat Enterprise Linux
5 5 (0.27%) CentOS
6 2 (0.11%) SUSE Linux Enterprise Server
6 2 (0.11%) Ubuntu
7 1 (0.053%) CoreOS
7 1 (0.053%) Debian
Processes & Methodologies
1 775 (41.11%) Agile Software Development
2 715 (37.93%) Analytics
3 704 (37.35%) Data Science
4 593 (31.46%) Machine Learning
5 387 (20.53%) ETL
6 370 (19.63%) DevOps
7 367 (19.47%) Software Engineering
8 346 (18.36%) Data Engineering
9 279 (14.80%) Business Intelligence
10 278 (14.75%) Visualisation
11 268 (14.22%) Data Modelling
12 264 (14.01%) Data Analytics
13 259 (13.74%) Microservices
14 256 (13.58%) Artificial Intelligence
15 254 (13.47%) Scrum
16 247 (13.10%) Computer Science
17 196 (10.40%) Mathematics
18 185 (9.81%) MapReduce
19 175 (9.28%) Mentoring
19 175 (9.28%) Statistics
Programming Languages
1 1,180 (62.60%) Python
2 821 (43.55%) Java
3 733 (38.89%) SQL
4 706 (37.45%) Scala
5 301 (15.97%) R
6 194 (10.29%) JavaScript
7 141 (7.48%) C
8 129 (6.84%) Bash Shell
9 120 (6.37%) C#
10 110 (5.84%) C++
11 82 (4.35%) Go
12 53 (2.81%) Shell Script
13 50 (2.65%) Perl
14 49 (2.60%) Java 8
15 42 (2.23%) PL/SQL
16 41 (2.18%) TypeScript
17 30 (1.59%) PHP
18 27 (1.43%) Kotlin
19 20 (1.06%) Ruby
20 18 (0.95%) PowerShell
Qualifications
1 288 (15.28%) Degree
2 94 (4.99%) PhD
3 92 (4.88%) Computer Science Degree
4 87 (4.62%) Security Cleared
5 47 (2.49%) Master's Degree
6 43 (2.28%) DV Cleared
6 43 (2.28%) Microsoft Certification
7 41 (2.18%) CCNA
7 41 (2.18%) Cisco Certification
7 41 (2.18%) CISSP
7 41 (2.18%) CompTIA Security+
7 41 (2.18%) GIAC
7 41 (2.18%) Linux+ Certification
7 41 (2.18%) MCSE
7 41 (2.18%) Network+ Certification
7 41 (2.18%) SANS
7 41 (2.18%) SSCP
8 37 (1.96%) SC Cleared
9 31 (1.64%) Numerate Degree
10 7 (0.37%) PMI Certification
Quality Assurance & Compliance
1 102 (5.41%) Data Quality
2 81 (4.30%) QA
3 19 (1.01%) GDPR
4 11 (0.58%) ISO/IEC 27001
5 3 (0.16%) Actionable Recommendations
5 3 (0.16%) EMIR
5 3 (0.16%) MiFID
6 2 (0.11%) PMO
7 1 (0.053%) IFRS
7 1 (0.053%) IFRS 9
7 1 (0.053%) ISO 26262
System Software
1 493 (26.15%) Docker
2 152 (8.06%) HDFS
3 88 (4.67%) Apache Flume
4 36 (1.91%) Apache ZooKeeper
5 33 (1.75%) VirtualBox
6 10 (0.53%) Cronacle
7 3 (0.16%) Active Directory
7 3 (0.16%) Varnish
8 1 (0.053%) Hyper-V
8 1 (0.053%) Virtual Servers
8 1 (0.053%) VMware Infrastructure
Systems Management
1 372 (19.73%) Kubernetes
2 166 (8.81%) Ansible
3 153 (8.12%) Terraform
4 146 (7.75%) Oozie
5 112 (5.94%) Hadoop YARN
6 52 (2.76%) Puppet
7 40 (2.12%) Docker Swarm
8 24 (1.27%) Ambari
9 22 (1.17%) Opscode Chef
10 20 (1.06%) Consul
11 18 (0.95%) Kibana
12 15 (0.80%) Control-M
13 13 (0.69%) Mesos
14 10 (0.53%) Single Sign-On
15 8 (0.42%) logstash
15 8 (0.42%) Rancher
15 8 (0.42%) Salt
16 6 (0.32%) Marathon
16 6 (0.32%) Prometheus
17 5 (0.27%) HP Quality Center
Vendors
1 307 (16.29%) Google
2 135 (7.16%) Cloudera
3 130 (6.90%) Tableau
4 126 (6.68%) Hortonworks
5 109 (5.78%) Oracle
6 100 (5.31%) Informatica
7 93 (4.93%) Microsoft
8 85 (4.51%) Splunk
9 76 (4.03%) Databricks
10 67 (3.55%) Talend
11 51 (2.71%) Teradata
12 44 (2.33%) IBM
13 29 (1.54%) WorldPay
14 24 (1.27%) Ab Initio
14 24 (1.27%) SAP
15 20 (1.06%) Dataiku
16 18 (0.95%) MapR
16 18 (0.95%) Netezza
17 15 (0.80%) VMware
18 11 (0.58%) Alteryx