Period
to 17 August 2019

The following table provides summary statistics for permanent job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 17 August 2019 with a comparison to the same period in the previous 2 years.

Apache Spark
UK > England
6 months to
17 Aug 2019
Same period 2018 Same period 2017
Rank 213 235 286
Rank change year-on-year +22 +51 +99
Permanent jobs citing Apache Spark 2,117 2,336 2,061
As % of all permanent IT jobs advertised in England 1.60% 1.47% 1.24%
As % of the Application Platforms category 10.74% 8.83% 6.88%
Number of salaries quoted 1,726 1,827 1,652
England median annual salary £75,000 £70,000 £67,500
Median salary % change year-on-year +7.14% +3.70% +3.85%
10th Percentile £48,250 £44,050 £42,500
90th Percentile £110,000 £100,000 £101,250
UK median annual salary £75,000 £70,000 £67,500
% change year-on-year +7.14% +3.70% +3.85%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Permanent vacancies with a requirement for application platform skills 19,703 26,454 29,952
As % of all permanent IT jobs advertised in England 14.92% 16.63% 18.07%
Number of salaries quoted 15,631 21,288 24,379
England median annual salary £50,000 £50,000 £47,500
Median salary % change year-on-year - +5.26% +5.56%
10th Percentile £28,000 £28,500 £27,500
90th Percentile £85,000 £82,500 £77,500
UK median annual salary £50,000 £50,000 £47,500
% change year-on-year - +5.26% +5.56%

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a percentage of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Salary Trend in England

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in England.

Salary trend for Apache Spark in England

Apache Spark
Salary Histogram in England

The salary distribution of IT jobs citing Apache Spark in England over the 6 months to 17 August 2019.

Salary histogram for Apache Spark in England

Apache Spark
Job Locations in England

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the England region over the 6 months to 17 August 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
London +18 1,465 £80,000 +6.67% 160
South West +83 177 £55,000 -8.33% 11
South East -54 165 £64,000 +6.67% 12
North of England -8 131 £60,000 +9.09% 25
Midlands +49 98 £65,000 +8.33% 5
North West +2 77 £65,000 +18.18% 18
East Midlands +45 76 £57,500 -13.21%
East of England +18 67 £62,500 -5.30% 15
Yorkshire -8 41 £57,275 +4.14% 6
West Midlands +11 22 £70,000 +33.33% 5
North East +31 13 £53,638 +34.09% 1
Apache Spark
UK

For the 6 months to 17 August 2019, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the England region with a requirement for Apache Spark.

1 1,424 (67.26%) Big Data
2 1,394 (65.85%) Python
3 1,365 (64.48%) Hadoop
4 1,034 (48.84%) Java
5 961 (45.39%) Agile Software Development
6 888 (41.95%) Amazon AWS
7 870 (41.10%) Scala
8 865 (40.86%) SQL
9 860 (40.62%) Analytics
10 854 (40.34%) Kafka
11 746 (35.24%) Apache Hive
12 745 (35.19%) Data Science
13 680 (32.12%) Machine Learning
14 508 (24.00%) Microsoft Azure
15 502 (23.71%) Docker
16 488 (23.05%) DevOps
17 476 (22.48%) NoSQL
18 441 (20.83%) Software Engineering
19 411 (19.41%) ETL
20 407 (19.23%) Finance
21 402 (18.99%) Linux
22 396 (18.71%) Google Cloud Platform
23 376 (17.76%) Analytical Skills
24 369 (17.43%) Data Engineering
25 341 (16.11%) Apache
26 340 (16.06%) R
27 328 (15.49%) HBase
28 327 (15.45%) Degree
29 322 (15.21%) Git (software)
30 311 (14.69%) Data Modelling

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 341 (16.11%) Apache
2 280 (13.23%) Elasticsearch
3 204 (9.64%) Apache Pig
4 89 (4.20%) Apache Solr
5 55 (2.60%) Apache Airflow
6 54 (2.55%) SAS
7 39 (1.84%) OpenStack
8 26 (1.23%) Jupyter
9 25 (1.18%) Hyperledger
10 19 (0.90%) Confluence
11 14 (0.66%) Luigi
12 11 (0.52%) nginx
13 9 (0.43%) IPython
14 7 (0.33%) WebSphere
15 6 (0.28%) JBoss
16 5 (0.24%) ArcGIS
17 4 (0.19%) Tomcat
18 3 (0.14%) Cloud Foundry
18 3 (0.14%) WebLogic
19 2 (0.094%) Tornado
Applications
1 18 (0.85%) Microsoft Excel
2 7 (0.33%) Gephi
2 7 (0.33%) Microsoft Office
3 4 (0.19%) GNU Octave
3 4 (0.19%) Weka
4 3 (0.14%) Microsoft PowerPoint
5 2 (0.094%) Chatbot
5 2 (0.094%) SPSS
6 1 (0.047%) Microsoft Project
6 1 (0.047%) MS Visio
Business Applications
1 2 (0.094%) PolarLake
1 2 (0.094%) ReMetrica
1 2 (0.094%) SAP Oil and Gas
1 2 (0.094%) Towers Watson Igloo
2 1 (0.047%) Adobe Campaign
Cloud Services
1 888 (41.95%) Amazon AWS
2 508 (24.00%) Microsoft Azure
3 396 (18.71%) Google Cloud Platform
4 114 (5.38%) Amazon EMR
5 95 (4.49%) Power Platform
6 90 (4.25%) Serverless
7 85 (4.02%) Amazon Kinesis
7 85 (4.02%) Amazon S3
8 72 (3.40%) Azure Data Factory
9 70 (3.31%) Cloud Computing
9 70 (3.31%) PaaS
10 62 (2.93%) SaaS
11 55 (2.60%) Amazon EC2
12 51 (2.41%) AWS Lambda
13 44 (2.08%) OpenShift
14 39 (1.84%) GitHub
15 27 (1.28%) Cortana Intelligence Suite
16 26 (1.23%) Amazon SQS
16 26 (1.23%) AWS CloudFormation
17 22 (1.04%) IaaS
Communications & Networking
1 43 (2.03%) TCP/IP
2 42 (1.98%) Firewall
3 38 (1.79%) HTTPS
4 33 (1.56%) Internet
5 32 (1.51%) HTTP
6 14 (0.66%) SSL
7 12 (0.57%) VPN
7 12 (0.57%) Wi-Fi
8 10 (0.47%) DNS
8 10 (0.47%) Kerberos
9 8 (0.38%) Broadband
10 7 (0.33%) Ethernet
10 7 (0.33%) MPLS
11 6 (0.28%) LAN
11 6 (0.28%) Software-Defined Networking
11 6 (0.28%) WAN
12 3 (0.14%) FTP
12 3 (0.14%) Unified Communications
13 2 (0.094%) SMTP
14 1 (0.047%) SMS
Database & Business Intelligence
1 1,424 (67.26%) Big Data
2 1,365 (64.48%) Hadoop
3 746 (35.24%) Apache Hive
4 476 (22.48%) NoSQL
5 328 (15.49%) HBase
6 309 (14.60%) Data Warehouse
7 303 (14.31%) Impala
8 299 (14.12%) Apache Cassandra
9 272 (12.85%) MongoDB
10 183 (8.64%) Amazon Redshift
11 175 (8.27%) Data Lake
12 156 (7.37%) PostgreSQL
13 145 (6.85%) Relational Database
14 138 (6.52%) Sqoop
15 117 (5.53%) Flink
15 117 (5.53%) SQL Server
16 115 (5.43%) MySQL
17 97 (4.58%) Data Mining
18 95 (4.49%) Power BI
19 87 (4.11%) BigQuery
Development Applications
1 322 (15.21%) Git (software)
2 226 (10.68%) Jenkins
3 132 (6.24%) JIRA
4 111 (5.24%) Bitbucket
5 74 (3.50%) git-flow
6 49 (2.31%) Maven
7 45 (2.13%) Bitbucket Server
8 33 (1.56%) Gradle
9 31 (1.46%) Visual Studio
10 29 (1.37%) Atlassian Bamboo
11 28 (1.32%) Selenium
12 27 (1.28%) Cucumber
13 26 (1.23%) Artifactory
14 25 (1.18%) Subversion
15 22 (1.04%) Visual Studio Team System
16 19 (0.90%) Team Foundation Server
17 16 (0.76%) Ant
17 16 (0.76%) JUnit
17 16 (0.76%) RStudio
17 16 (0.76%) Vagrant
General
1 407 (19.23%) Finance
2 153 (7.23%) Retail
3 147 (6.94%) Banking
4 138 (6.52%) Law
5 91 (4.30%) Marketing
6 80 (3.78%) Telecoms
7 42 (1.98%) Games
8 40 (1.89%) Publishing
9 37 (1.75%) Front Office
10 33 (1.56%) Advertising
11 32 (1.51%) Financial Institution
11 32 (1.51%) Manufacturing
12 28 (1.32%) Investment Banking
13 18 (0.85%) Automotive
14 16 (0.76%) Electronics
15 9 (0.43%) Billing
16 7 (0.33%) Aerospace
16 7 (0.33%) Retail Banking
17 5 (0.24%) Legal
18 3 (0.14%) Pharmaceutical
Job Titles
1 676 (31.93%) Data Engineer
2 459 (21.68%) Developer
3 271 (12.80%) Big Data Engineer
4 242 (11.43%) Java Developer
5 233 (11.01%) Data Scientist
6 181 (8.55%) Senior Data Engineer
7 171 (8.08%) Architect
8 139 (6.57%) Python Engineer
9 134 (6.33%) Software Engineer
10 132 (6.24%) Data Architect
11 120 (5.67%) Big Data Developer
12 105 (4.96%) Senior Developer
13 96 (4.53%) Hadoop Engineer
14 95 (4.49%) Senior Data Scientist
15 93 (4.39%) Scala Developer
16 90 (4.25%) Consultant
17 74 (3.50%) DevOps Engineer
18 73 (3.45%) Big Data Architect
19 70 (3.31%) Java Engineer
20 68 (3.21%) Senior Java Developer
Libraries, Frameworks & Software Standards
1 854 (40.34%) Kafka
2 172 (8.12%) Web Services
3 167 (7.89%) Spring
4 156 (7.37%) REST
5 135 (6.38%) scikit-learn
6 129 (6.09%) Pandas
7 123 (5.81%) TensorFlow
8 112 (5.29%) Node.js
9 107 (5.05%) React
10 105 (4.96%) Elastic Stack
11 103 (4.87%) Natural Language Processing
12 102 (4.82%) Apache NiFi
13 97 (4.58%) RESTful
14 96 (4.53%) .NET
15 94 (4.44%) Spark Streaming
16 93 (4.39%) Akka
17 92 (4.35%) AngularJS
18 85 (4.02%) NumPy
19 78 (3.68%) JSON
20 61 (2.88%) HTML
Miscellaneous
1 376 (17.76%) Analytical Skills
2 299 (14.12%) Algorithms
3 126 (5.95%) Data Structures
4 76 (3.59%) Greenfield Project
5 55 (2.60%) Analytical Mindset
6 51 (2.41%) Public Cloud
7 41 (1.94%) Cyber Kill Chain
8 40 (1.89%) Management Information System
9 39 (1.84%) Social Media
10 36 (1.70%) Private Cloud
10 36 (1.70%) Self-Motivation
10 36 (1.70%) Virtual Team
11 32 (1.51%) Cloud Native
11 32 (1.51%) User Experience
12 30 (1.42%) Linux Command Line
13 26 (1.23%) Cybercrime
14 25 (1.18%) Derivative
15 24 (1.13%) Mobile App
16 20 (0.94%) Enterprise Software
17 19 (0.90%) Robotics
Operating Systems
1 402 (18.99%) Linux
2 180 (8.50%) Unix
3 58 (2.74%) Windows
4 37 (1.75%) Android
4 37 (1.75%) Apple iOS
5 17 (0.80%) CentOS
6 12 (0.57%) Ubuntu
7 8 (0.38%) Windows Server
8 5 (0.24%) Red Hat Enterprise Linux
9 2 (0.094%) Debian
9 2 (0.094%) Mac OS X
9 2 (0.094%) SUSE Linux Enterprise Server
Processes & Methodologies
1 961 (45.39%) Agile Software Development
2 860 (40.62%) Analytics
3 745 (35.19%) Data Science
4 680 (32.12%) Machine Learning
5 488 (23.05%) DevOps
6 441 (20.83%) Software Engineering
7 411 (19.41%) ETL
8 369 (17.43%) Data Engineering
9 311 (14.69%) Data Modelling
10 310 (14.64%) Data Analytics
11 304 (14.36%) Artificial Intelligence
12 297 (14.03%) Business Intelligence
13 296 (13.98%) Scrum
13 296 (13.98%) Visualisation
14 267 (12.61%) MapReduce
15 257 (12.14%) Microservices
16 229 (10.82%) Mathematics
17 223 (10.53%) Open Source
18 217 (10.25%) Computer Science
19 192 (9.07%) Problem-Solving
Programming Languages
1 1,394 (65.85%) Python
2 1,034 (48.84%) Java
3 870 (41.10%) Scala
4 865 (40.86%) SQL
5 340 (16.06%) R
6 179 (8.46%) JavaScript
7 136 (6.42%) C#
8 131 (6.19%) Bash Shell
9 128 (6.05%) C
10 77 (3.64%) C++
11 66 (3.12%) Perl
12 58 (2.74%) Java 8
13 57 (2.69%) PL/SQL
14 50 (2.36%) Shell Script
15 44 (2.08%) Go
16 31 (1.46%) T-SQL
17 28 (1.32%) Ruby
17 28 (1.32%) TypeScript
18 22 (1.04%) PHP
19 21 (0.99%) Groovy
Qualifications
1 327 (15.45%) Degree
2 111 (5.24%) Security Cleared
3 104 (4.91%) Computer Science Degree
4 98 (4.63%) PhD
5 59 (2.79%) Numerate Degree
6 55 (2.60%) SC Cleared
7 44 (2.08%) DV Cleared
8 41 (1.94%) Master's Degree
9 37 (1.75%) Microsoft Certification
10 36 (1.70%) CCNA
10 36 (1.70%) Cisco Certification
10 36 (1.70%) CISSP
10 36 (1.70%) CompTIA Security+
10 36 (1.70%) GIAC
10 36 (1.70%) Linux+ Certification
10 36 (1.70%) MCSE
10 36 (1.70%) Network+ Certification
10 36 (1.70%) SANS
10 36 (1.70%) SSCP
11 15 (0.71%) PMP
Quality Assurance & Compliance
1 78 (3.68%) Data Quality
2 56 (2.65%) QA
3 18 (0.85%) GDPR
4 4 (0.19%) ISO/IEC 27001
5 3 (0.14%) Actionable Recommendations
6 2 (0.094%) ISO 15022
6 2 (0.094%) ISO 20022
6 2 (0.094%) PCI DSS
6 2 (0.094%) PMO
7 1 (0.047%) IFRS
7 1 (0.047%) IFRS 9
System Software
1 502 (23.71%) Docker
2 156 (7.37%) HDFS
3 147 (6.94%) Apache Flume
4 85 (4.02%) Apache ZooKeeper
5 15 (0.71%) VirtualBox
6 9 (0.43%) Cronacle
7 6 (0.28%) Active Directory
8 4 (0.19%) VMware Infrastructure
9 2 (0.094%) Hyper-V
9 2 (0.094%) Virtual Machines
9 2 (0.094%) Virtual Servers
10 1 (0.047%) Ceph
Systems Management
1 303 (14.31%) Kubernetes
2 202 (9.54%) Oozie
3 193 (9.12%) Ansible
4 174 (8.22%) Hadoop YARN
5 164 (7.75%) Terraform
6 89 (4.20%) Puppet
7 80 (3.78%) Opscode Chef
8 46 (2.17%) Ambari
9 25 (1.18%) Kibana
9 25 (1.18%) Mesos
10 20 (0.94%) Docker Swarm
11 19 (0.90%) Marathon
12 16 (0.76%) Nagios
13 13 (0.61%) Consul
13 13 (0.61%) Control-M
14 10 (0.47%) logstash
15 9 (0.43%) Prometheus
16 8 (0.38%) Salt
17 7 (0.33%) Rancher
18 6 (0.28%) Microsoft Intune
Vendors
1 280 (13.23%) Google
2 241 (11.38%) Cloudera
3 166 (7.84%) Hortonworks
3 166 (7.84%) Oracle
4 163 (7.70%) Tableau
5 152 (7.18%) Informatica
6 136 (6.42%) Databricks
7 120 (5.67%) Microsoft
8 88 (4.16%) Splunk
9 80 (3.78%) Talend
10 58 (2.74%) Teradata
11 56 (2.65%) MapR
12 47 (2.22%) IBM
13 38 (1.79%) SAP
14 32 (1.51%) Ab Initio
15 28 (1.32%) Netezza
16 24 (1.13%) Dataiku
17 22 (1.04%) VMware
18 17 (0.80%) Greenplum
18 17 (0.80%) Vertica