Period
to 22 April 2019

The following table provides summary statistics for permanent job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 22 April 2019 with a comparison to the same period in the previous 2 years.

Apache Spark
UK > England
6 months to
22 Apr 2019
Same period 2018 Same period 2017
Rank 227 221 294
Rank change year-on-year -6 +73 +129
Permanent jobs citing Apache Spark 2,116 2,579 1,811
As % of all permanent IT jobs advertised in England 1.54% 1.55% 1.12%
As % of the Application Platforms category 9.95% 9.29% 6.29%
Number of salaries quoted 1,688 2,115 1,514
England median annual salary £75,000 £70,000 £67,500
Median salary % change year-on-year +7.14% +3.70% +3.85%
10th Percentile £48,375 £42,500 £44,125
90th Percentile £107,500 £105,000 £100,000
UK median annual salary £75,000 £70,000 £67,500
% change year-on-year +7.14% +3.70% +3.85%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Permanent vacancies with a requirement for application platform skills 21,264 27,758 28,781
As % of all permanent IT jobs advertised in England 15.46% 16.71% 17.84%
Number of salaries quoted 16,865 22,859 24,079
England median annual salary £50,000 £50,000 £47,500
Median salary % change year-on-year - +5.26% +5.56%
10th Percentile £28,000 £27,500 £27,000
90th Percentile £85,000 £80,000 £75,000
UK median annual salary £50,000 £50,000 £45,500
% change year-on-year - +9.89% +1.11%

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a percentage of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Salary Trend in England

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in England.

Salary trend for Apache Spark in England

Apache Spark
Salary Histogram in England

The salary distribution of IT jobs citing Apache Spark in England over the 6 months to 22 April 2019.

Salary histogram for Apache Spark in England

Apache Spark
Job Locations in England

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the England region over the 6 months to 22 April 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
London -4 1,529 £80,000 +6.67% 71
South East -25 160 £70,000 +16.67% 8
North of England +20 157 £63,354 +15.19% 13
South West +28 135 £55,000 +10.00% 6
North West +24 100 £65,000 +18.72% 11
Midlands +37 73 £57,500 +4.55% 13
East of England -9 52 £72,500 - 6
East Midlands +33 47 £57,500 +4.55% 6
Yorkshire -4 39 £49,337 -10.30% 2
West Midlands +17 26 £65,000 +8.33% 7
North East +42 19 £49,337 -
Apache Spark
UK

For the 6 months to 22 April 2019, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the England region with a requirement for Apache Spark.

1 1,531 (72.35%) Big Data
2 1,500 (70.89%) Hadoop
3 1,461 (69.05%) Python
4 1,188 (56.14%) Java
5 969 (45.79%) Agile Software Development
6 966 (45.65%) Scala
7 958 (45.27%) Kafka
8 933 (44.09%) Amazon AWS
9 866 (40.93%) Analytics
10 836 (39.51%) SQL
11 764 (36.11%) Machine Learning
12 734 (34.69%) Apache Hive
13 726 (34.31%) Data Science
14 661 (31.24%) NoSQL
15 591 (27.93%) DevOps
16 496 (23.44%) Microsoft Azure
17 495 (23.39%) Software Engineering
18 428 (20.23%) Data Modelling
19 427 (20.18%) Apache Cassandra
20 426 (20.13%) ETL
21 424 (20.04%) HBase
22 413 (19.52%) Apache
23 407 (19.23%) Google Cloud Platform
24 405 (19.14%) Finance
25 398 (18.81%) Artificial Intelligence
26 389 (18.38%) Data Engineering
27 367 (17.34%) Linux
28 356 (16.82%) Cloudera
29 340 (16.07%) Docker
30 339 (16.02%) Degree

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 413 (19.52%) Apache
2 324 (15.31%) Elasticsearch
3 302 (14.27%) Apache Pig
4 120 (5.67%) Apache Solr
5 85 (4.02%) SAS
6 37 (1.75%) Apache Airflow
7 23 (1.09%) Tomcat
8 22 (1.04%) GlassFish
9 18 (0.85%) WebSphere
10 16 (0.76%) JBoss
10 16 (0.76%) Jupyter
11 13 (0.61%) Apache Geronimo
11 13 (0.61%) Confluence
11 13 (0.61%) Luigi
12 9 (0.43%) nginx
13 7 (0.33%) Drupal
14 6 (0.28%) IPython
15 5 (0.24%) CMS
15 5 (0.24%) Sitecore CMS
16 4 (0.19%) OpenStack
Applications
1 15 (0.71%) Weka
2 12 (0.57%) Microsoft Excel
3 10 (0.47%) SPSS
4 5 (0.24%) Stata
5 4 (0.19%) Chatbot
5 4 (0.19%) Microsoft Office
6 3 (0.14%) Microsoft PowerPoint
7 2 (0.095%) CPLEX
7 2 (0.095%) Gephi
7 2 (0.095%) GNU Octave
7 2 (0.095%) MS Visio
7 2 (0.095%) S-PLUS
Business Applications
1 2 (0.095%) PolarLake
1 2 (0.095%) ReMetrica
1 2 (0.095%) Towers Watson Igloo
2 1 (0.047%) Adobe Campaign
2 1 (0.047%) SAP Oil and Gas
Cloud Services
1 933 (44.09%) Amazon AWS
2 496 (23.44%) Microsoft Azure
3 407 (19.23%) Google Cloud Platform
4 114 (5.39%) Amazon EMR
5 110 (5.20%) Cloud Computing
6 92 (4.35%) Serverless
7 88 (4.16%) GitHub
8 86 (4.06%) Amazon EC2
9 81 (3.83%) Amazon S3
10 69 (3.26%) AWS Lambda
11 62 (2.93%) Amazon Kinesis
12 54 (2.55%) PaaS
13 48 (2.27%) Azure Data Factory
14 39 (1.84%) Amazon SQS
14 39 (1.84%) SaaS
15 27 (1.28%) AWS CloudFormation
16 23 (1.09%) Cortana Intelligence Suite
17 18 (0.85%) Amazon ELB
18 16 (0.76%) Amazon SageMaker
18 16 (0.76%) Route 53
Communications & Networking
1 47 (2.22%) Internet
2 30 (1.42%) HTTP
3 11 (0.52%) TCP/IP
4 10 (0.47%) Wi-Fi
5 9 (0.43%) Firewall
5 9 (0.43%) LAN
5 9 (0.43%) Software-Defined Networking
5 9 (0.43%) WAN
6 8 (0.38%) AMQP
6 8 (0.38%) MQTT
6 8 (0.38%) VPN
7 7 (0.33%) DNS
8 6 (0.28%) Kerberos
9 5 (0.24%) Broadband
9 5 (0.24%) HTTPS
10 4 (0.19%) FTP
11 3 (0.14%) SSL
12 2 (0.095%) Ethernet
12 2 (0.095%) MPLS
12 2 (0.095%) SMTP
Database & Business Intelligence
1 1,531 (72.35%) Big Data
2 1,500 (70.89%) Hadoop
3 734 (34.69%) Apache Hive
4 661 (31.24%) NoSQL
5 427 (20.18%) Apache Cassandra
6 424 (20.04%) HBase
7 311 (14.70%) MongoDB
8 299 (14.13%) Impala
9 297 (14.04%) Amazon Redshift
10 247 (11.67%) Data Warehouse
11 191 (9.03%) PostgreSQL
12 174 (8.22%) Data Lake
13 171 (8.08%) Flink
14 163 (7.70%) Sqoop
15 142 (6.71%) MySQL
16 131 (6.19%) Relational Database
17 122 (5.77%) SQL Server
18 115 (5.43%) Power BI
19 111 (5.25%) Neo4j
20 106 (5.01%) CouchDB
Development Applications
1 183 (8.65%) Git (software)
2 166 (7.84%) Jenkins
3 81 (3.83%) JIRA
4 59 (2.79%) Maven
5 52 (2.46%) Gradle
6 43 (2.03%) Visual Studio
7 41 (1.94%) Ant
8 36 (1.70%) Selenium
9 29 (1.37%) Atlassian Bamboo
10 24 (1.13%) Mercurial
11 23 (1.09%) Visual Studio Team System
12 21 (0.99%) JMeter
13 19 (0.90%) Travis CI
14 18 (0.85%) Bitbucket Server
14 18 (0.85%) SoapUI
15 16 (0.76%) Cucumber
15 16 (0.76%) JUnit
16 15 (0.71%) Artifactory
16 15 (0.71%) IntelliJ
16 15 (0.71%) Subversion
General
1 405 (19.14%) Finance
2 248 (11.72%) Law
3 206 (9.74%) Military
4 182 (8.60%) Banking
5 154 (7.28%) Retail
6 110 (5.20%) Telecoms
7 62 (2.93%) Marketing
8 42 (1.98%) Investment Banking
9 35 (1.65%) Games
10 33 (1.56%) Publishing
11 27 (1.28%) Advertising
12 23 (1.09%) Manufacturing
13 17 (0.80%) Electronics
13 17 (0.80%) Financial Institution
14 12 (0.57%) Automotive
15 9 (0.43%) Billing
15 9 (0.43%) Front Office
16 8 (0.38%) Retail Banking
17 7 (0.33%) Pharmaceutical
18 6 (0.28%) Legal
Job Titles
1 750 (35.44%) Data Engineer
2 399 (18.86%) Big Data Engineer
3 379 (17.91%) Developer
4 233 (11.01%) Data Scientist
5 224 (10.59%) Architect
6 189 (8.93%) Java Developer
7 183 (8.65%) Data Architect
8 182 (8.60%) Senior Data Engineer
9 150 (7.09%) Python Engineer
10 132 (6.24%) Hadoop Engineer
11 116 (5.48%) Software Engineer
12 105 (4.96%) Big Data Architect
13 93 (4.40%) Big Data Developer
14 84 (3.97%) DevOps Engineer
15 75 (3.54%) Java Engineer
16 70 (3.31%) Consultant
17 68 (3.21%) Senior Data Scientist
17 68 (3.21%) Senior Developer
18 64 (3.02%) Scala Developer
19 60 (2.84%) AWS Engineer
Libraries, Frameworks & Software Standards
1 958 (45.27%) Kafka
2 219 (10.35%) Web Services
3 163 (7.70%) Spark Streaming
4 135 (6.38%) Natural Language Processing
5 127 (6.00%) Apache NiFi
6 124 (5.86%) REST
7 120 (5.67%) Spring
8 111 (5.25%) Node.js
9 110 (5.20%) Pandas
10 109 (5.15%) scikit-learn
11 98 (4.63%) RESTful
12 92 (4.35%) .NET
13 87 (4.11%) TensorFlow
14 81 (3.83%) AngularJS
14 81 (3.83%) NumPy
15 79 (3.73%) HTML
16 75 (3.54%) JSON
17 71 (3.36%) React
18 70 (3.31%) Akka
19 65 (3.07%) PySpark
Miscellaneous
1 269 (12.71%) Analytical Skills
2 251 (11.86%) Algorithms
3 158 (7.47%) Data Structures
4 101 (4.77%) Fintech
5 59 (2.79%) Greenfield Project
6 46 (2.17%) Virtual Team
7 44 (2.08%) Management Information System
8 39 (1.84%) Self-Motivation
9 36 (1.70%) Social Media
9 36 (1.70%) User Experience
10 28 (1.32%) Cloud Native
10 28 (1.32%) Public Cloud
11 22 (1.04%) Life Science
12 21 (0.99%) Mobile App
13 19 (0.90%) Social Network
14 18 (0.85%) Blog
14 18 (0.85%) Credit Risk
15 16 (0.76%) Robotics
16 15 (0.71%) Linux Command Line
16 15 (0.71%) Video on Demand
Operating Systems
1 367 (17.34%) Linux
2 185 (8.74%) Unix
3 59 (2.79%) Windows
4 19 (0.90%) CentOS
5 14 (0.66%) Android
5 14 (0.66%) Apple iOS
5 14 (0.66%) Ubuntu
6 10 (0.47%) Windows Server
7 4 (0.19%) Mac OS X
7 4 (0.19%) Oracle Linux
8 2 (0.095%) Debian
8 2 (0.095%) Red Hat Enterprise Linux
Processes & Methodologies
1 969 (45.79%) Agile Software Development
2 866 (40.93%) Analytics
3 764 (36.11%) Machine Learning
4 726 (34.31%) Data Science
5 591 (27.93%) DevOps
6 495 (23.39%) Software Engineering
7 428 (20.23%) Data Modelling
8 426 (20.13%) ETL
9 398 (18.81%) Artificial Intelligence
10 389 (18.38%) Data Engineering
11 335 (15.83%) Business Intelligence
12 331 (15.64%) Data Analytics
13 299 (14.13%) Visualisation
14 265 (12.52%) MapReduce
15 246 (11.63%) Open Source
16 242 (11.44%) Computer Science
17 240 (11.34%) Data Analysis
18 229 (10.82%) Scrum
19 226 (10.68%) Continuous Integration
20 215 (10.16%) Microservices
Programming Languages
1 1,461 (69.05%) Python
2 1,188 (56.14%) Java
3 966 (45.65%) Scala
4 836 (39.51%) SQL
5 255 (12.05%) R
6 158 (7.47%) JavaScript
7 126 (5.95%) C#
8 109 (5.15%) C++
9 73 (3.45%) C
10 66 (3.12%) Bash Shell
11 62 (2.93%) Shell Script
12 60 (2.84%) Java 8
13 56 (2.65%) Perl
14 53 (2.50%) PL/SQL
15 52 (2.46%) T-SQL
16 51 (2.41%) Ruby
17 34 (1.61%) Groovy
18 32 (1.51%) Go
19 29 (1.37%) PHP
20 27 (1.28%) Clojure
Qualifications
1 339 (16.02%) Degree
2 126 (5.95%) Computer Science Degree
3 83 (3.92%) PhD
4 74 (3.50%) Security Cleared
5 69 (3.26%) Numerate Degree
6 43 (2.03%) SC Cleared
7 25 (1.18%) DV Cleared
8 18 (0.85%) Master's Degree
9 9 (0.43%) Microsoft Certification
10 8 (0.38%) PMI Certification
11 6 (0.28%) PMP
11 6 (0.28%) TOGAF Certification
12 5 (0.24%) ISTQB
12 5 (0.24%) ISTQB Foundation Certification
13 4 (0.19%) AWS Certification
13 4 (0.19%) CCNA
13 4 (0.19%) Cisco Certification
13 4 (0.19%) CISSP
13 4 (0.19%) MCDBA
13 4 (0.19%) MCSE
Quality Assurance & Compliance
1 97 (4.58%) Data Quality
2 61 (2.88%) QA
3 21 (0.99%) GDPR
4 8 (0.38%) MiFID
5 6 (0.28%) PCI DSS
6 4 (0.19%) ISO/IEC 27001
7 1 (0.047%) COBIT
7 1 (0.047%) EMIR
7 1 (0.047%) ISO 15022
7 1 (0.047%) ISO 20022
7 1 (0.047%) ISO 9001
System Software
1 340 (16.07%) Docker
2 187 (8.84%) Apache Flume
3 162 (7.66%) HDFS
4 95 (4.49%) Apache ZooKeeper
5 9 (0.43%) VirtualBox
6 8 (0.38%) Active Directory
7 2 (0.095%) Hyper-V
7 2 (0.095%) Virtual Machines
7 2 (0.095%) VMware Infrastructure
8 1 (0.047%) Virtual Servers
Systems Management
1 268 (12.67%) Oozie
2 193 (9.12%) Hadoop YARN
3 192 (9.07%) Ansible
4 181 (8.55%) Kubernetes
5 143 (6.76%) Terraform
6 109 (5.15%) Opscode Chef
7 97 (4.58%) Puppet
8 50 (2.36%) Ambari
9 42 (1.98%) Kibana
10 40 (1.89%) Mesos
11 29 (1.37%) Nagios
12 25 (1.18%) Prometheus
13 22 (1.04%) Salt
14 17 (0.80%) logstash
15 15 (0.71%) Marathon
16 14 (0.66%) Docker Swarm
17 9 (0.43%) Microsoft Intune
17 9 (0.43%) SCCM
18 7 (0.33%) Oracle ZFS
19 4 (0.19%) Control-M
Vendors
1 356 (16.82%) Cloudera
2 245 (11.58%) Hortonworks
3 235 (11.11%) Google
4 233 (11.01%) Tableau
5 197 (9.31%) Oracle
6 164 (7.75%) Microsoft
7 157 (7.42%) Databricks
8 117 (5.53%) Informatica
9 108 (5.10%) MapR
10 88 (4.16%) Talend
11 72 (3.40%) IBM
12 55 (2.60%) SAP
13 40 (1.89%) CA Agile Central
13 40 (1.89%) Teradata
14 35 (1.65%) Splunk
15 25 (1.18%) Netezza
16 24 (1.13%) TIBCO
17 22 (1.04%) Alteryx
17 22 (1.04%) SaltStack
17 22 (1.04%) Vertica