Period
to 21 November 2019

The following table provides summary statistics for permanent job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 21 November 2019 with a comparison to the same period in the previous 2 years.

Apache Spark
England > London
6 months to
21 Nov 2019
Same period 2018 Same period 2017
Rank 146 173 170
Rank change year-on-year +27 -3 +77
Permanent jobs citing Apache Spark 1,262 1,402 1,625
As % of all permanent IT jobs advertised in London 2.32% 2.08% 2.15%
As % of the Application Platforms category 15.32% 11.98% 11.41%
Number of salaries quoted 1,051 1,082 1,342
London median annual salary £80,000 £77,500 £70,000
Median salary % change year-on-year +3.23% +10.71% -6.67%
10th Percentile £53,750 £50,000 £47,500
90th Percentile £112,500 £107,375 £106,000
England median annual salary £72,500 £72,500 £67,000
% change year-on-year - +8.21% -0.74%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Permanent vacancies with a requirement for application platform skills 8,237 11,700 14,237
As % of all permanent IT jobs advertised in London 15.13% 17.37% 18.83%
Number of salaries quoted 6,741 9,460 11,396
London median annual salary £62,500 £60,000 £57,500
Median salary % change year-on-year +4.17% +4.35% +4.55%
10th Percentile £32,500 £32,500 £32,500
90th Percentile £95,000 £92,500 £87,500
England median annual salary £50,000 £50,000 £47,500
% change year-on-year - +5.26% +3.26%

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a percentage of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Salary Trend in London

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in London.

Salary trend for Apache Spark in London

Apache Spark
Salary Histogram in London

The salary distribution of IT jobs citing Apache Spark in London over the 6 months to 21 November 2019.

Salary histogram for Apache Spark in London

Apache Spark
Job Locations in London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the London region over the 6 months to 21 November 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +37 269 £75,000 -6.25% 67
West London +32 28 £77,500 +14.81% 23
South London +28 17 £85,000 +6.25% 5
East London +8 16 £78,250 -17.63% 17
North London -3 1 £75,000 -6.25%
Apache Spark
England

For the 6 months to 21 November 2019, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the London region with a requirement for Apache Spark.

1 813 (64.42%) Python
2 730 (57.84%) Big Data
3 693 (54.91%) Hadoop
4 604 (47.86%) AWS
5 562 (44.53%) Java
6 514 (40.73%) SQL
7 506 (40.10%) Data Science
8 471 (37.32%) Agile Software Development
9 467 (37.00%) Scala
10 461 (36.53%) Analytics
11 455 (36.05%) Kafka
12 426 (33.76%) Machine Learning
13 362 (28.68%) Docker
14 336 (26.62%) Azure
15 327 (25.91%) Apache Hive
16 300 (23.77%) Kubernetes
17 285 (22.58%) DevOps
18 263 (20.84%) Finance
18 263 (20.84%) NoSQL
19 256 (20.29%) GCP
20 251 (19.89%) ETL
21 250 (19.81%) Linux
22 243 (19.26%) Software Engineering
23 234 (18.54%) Data Engineering
24 216 (17.12%) Google
25 209 (16.56%) Apache
26 206 (16.32%) Business Intelligence
27 205 (16.24%) Algorithms
28 201 (15.93%) Data Analytics
29 197 (15.61%) Microservices

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 209 (16.56%) Apache
2 108 (8.56%) Elasticsearch
3 80 (6.34%) Apache Airflow
4 33 (2.61%) Luigi
5 32 (2.54%) Apache Pig
6 27 (2.14%) SAS
7 22 (1.74%) Apache Solr
8 19 (1.51%) Jupyter
9 17 (1.35%) Confluence
10 9 (0.71%) OpenStack
11 7 (0.55%) nginx
12 5 (0.40%) SharePoint
13 2 (0.16%) Cloud Foundry
13 2 (0.16%) CMS
13 2 (0.16%) DataSynapse
13 2 (0.16%) Tomcat
14 1 (0.079%) ArcGIS
14 1 (0.079%) IPython
14 1 (0.079%) WebSphere
14 1 (0.079%) Zope
Applications
1 5 (0.40%) Microsoft Excel
2 3 (0.24%) GNU Octave
3 2 (0.16%) Chatbot
3 2 (0.16%) SPSS
4 1 (0.079%) Microsoft Project
Business Applications
1 2 (0.16%) Eikon
2 1 (0.079%) Adobe Campaign
2 1 (0.079%) SAP Oil and Gas
2 1 (0.079%) Sparx Enterprise Architect
Cloud Services
1 604 (47.86%) AWS
2 336 (26.62%) Azure
3 256 (20.29%) GCP
4 88 (6.97%) Amazon EMR
5 82 (6.50%) Amazon S3
6 81 (6.42%) Serverless
7 66 (5.23%) AWS Lambda
8 58 (4.60%) Amazon Kinesis
9 56 (4.44%) PaaS
10 46 (3.65%) Power Platform
11 38 (3.01%) SaaS
12 37 (2.93%) Cloud Computing
13 36 (2.85%) GitHub
14 27 (2.14%) Amazon EC2
15 24 (1.90%) OpenShift
16 23 (1.82%) Azure Data Factory
17 21 (1.66%) IBM Cloud
18 20 (1.58%) Amazon ECS
18 20 (1.58%) IaaS
19 15 (1.19%) AWS CloudFormation
Communications & Networking
1 33 (2.61%) HTTP
2 20 (1.58%) Internet
3 18 (1.43%) Firewall
4 13 (1.03%) HTTPS
4 13 (1.03%) Kerberos
5 10 (0.79%) VPN
6 8 (0.63%) TCP/IP
7 5 (0.40%) SSL
8 3 (0.24%) FTP
9 2 (0.16%) SMS
10 1 (0.079%) Unified Communications
Database & Business Intelligence
1 730 (57.84%) Big Data
2 693 (54.91%) Hadoop
3 327 (25.91%) Apache Hive
4 263 (20.84%) NoSQL
5 161 (12.76%) Data Warehouse
6 153 (12.12%) PostgreSQL
7 142 (11.25%) Apache Cassandra
8 127 (10.06%) Amazon Redshift
9 113 (8.95%) Relational Database
10 107 (8.48%) HBase
11 105 (8.32%) MongoDB
12 98 (7.77%) BigQuery
13 93 (7.37%) Flink
14 81 (6.42%) Data Mining
14 81 (6.42%) SQL Server
15 75 (5.94%) Impala
16 68 (5.39%) Neo4j
17 58 (4.60%) Data Lake
18 55 (4.36%) MySQL
19 52 (4.12%) Sqoop
Development Applications
1 178 (14.10%) Git (software)
2 147 (11.65%) Jenkins
3 62 (4.91%) JIRA
4 27 (2.14%) Atlassian Bamboo
5 24 (1.90%) Vagrant
6 19 (1.51%) Bitbucket
6 19 (1.51%) JUnit
6 19 (1.51%) Maven
7 17 (1.35%) git-flow
8 15 (1.19%) Gradle
9 13 (1.03%) RStudio
10 11 (0.87%) Eclipse
10 11 (0.87%) IntelliJ
11 10 (0.79%) TestNG
12 9 (0.71%) Bitbucket Server
12 9 (0.71%) kappa
13 8 (0.63%) Artifactory
14 7 (0.55%) Subversion
15 6 (0.48%) Visual Studio Code
16 5 (0.40%) CircleCI
General
1 263 (20.84%) Finance
2 107 (8.48%) Banking
3 101 (8.00%) Retail
4 63 (4.99%) Marketing
5 41 (3.25%) Front Office
6 26 (2.06%) Advertising
6 26 (2.06%) Telecoms
7 25 (1.98%) Investment Banking
7 25 (1.98%) Public Sector
8 24 (1.90%) Games
9 23 (1.82%) Financial Institution
10 21 (1.66%) Legal
10 21 (1.66%) Manufacturing
11 20 (1.58%) Automotive
12 16 (1.27%) Law
12 16 (1.27%) Publishing
13 9 (0.71%) Retail Banking
14 3 (0.24%) Back Office
14 3 (0.24%) Electronics
14 3 (0.24%) Music Industry
Job Titles
1 385 (30.51%) Data Engineer
2 283 (22.42%) Developer
3 150 (11.89%) Java Developer
4 149 (11.81%) Data Scientist
5 137 (10.86%) Architect
6 93 (7.37%) Data Architect
7 92 (7.29%) Senior Data Scientist
8 85 (6.74%) Scala Developer
9 73 (5.78%) Senior Data Engineer
10 68 (5.39%) Python Engineer
11 53 (4.20%) Software Engineer
12 52 (4.12%) Senior Developer
13 50 (3.96%) Big Data Developer
14 46 (3.65%) Python Developer
15 45 (3.57%) Big Data Engineer
16 43 (3.41%) Apache Spark Developer
17 39 (3.09%) Consultant
17 39 (3.09%) Machine Learning Engineer
18 38 (3.01%) Java Scala Developer
19 36 (2.85%) Lead Developer
Libraries, Frameworks & Software Standards
1 455 (36.05%) Kafka
2 141 (11.17%) REST
3 108 (8.56%) Web Services
4 103 (8.16%) TensorFlow
5 85 (6.74%) Pandas
6 77 (6.10%) Natural Language Processing
7 74 (5.86%) scikit-learn
8 68 (5.39%) React
9 62 (4.91%) NumPy
10 61 (4.83%) AngularJS
10 61 (4.83%) JSON
11 59 (4.68%) Spring
12 58 (4.60%) Akka
13 52 (4.12%) HTML
14 47 (3.72%) Node.js
14 47 (3.72%) RESTful
15 44 (3.49%) Apache Beam
15 44 (3.49%) Spark Streaming
16 42 (3.33%) Keras
17 38 (3.01%) GraphQL
Miscellaneous
1 205 (16.24%) Algorithms
2 188 (14.90%) Analytical Skills
3 70 (5.55%) Data Structures
4 61 (4.83%) Greenfield Project
5 32 (2.54%) Linux Command Line
6 28 (2.22%) Public Cloud
7 26 (2.06%) Virtual Team
8 23 (1.82%) Derivative
9 21 (1.66%) User Experience
10 20 (1.58%) Self-Motivation
11 19 (1.51%) Digital Media
11 19 (1.51%) Management Information System
12 18 (1.43%) Cloud Native
12 18 (1.43%) Data Centre
12 18 (1.43%) Equities
12 18 (1.43%) Mobile App
13 17 (1.35%) Social Media
14 16 (1.27%) Fixed Income
15 14 (1.11%) Hedge funds
15 14 (1.11%) Semantic Web
Operating Systems
1 250 (19.81%) Linux
2 76 (6.02%) Unix
3 12 (0.95%) Windows
4 8 (0.63%) Android
4 8 (0.63%) Apple iOS
5 2 (0.16%) CentOS
6 1 (0.079%) Debian
6 1 (0.079%) Ubuntu
Processes & Methodologies
1 506 (40.10%) Data Science
2 471 (37.32%) Agile Software Development
3 461 (36.53%) Analytics
4 426 (33.76%) Machine Learning
5 285 (22.58%) DevOps
6 251 (19.89%) ETL
7 243 (19.26%) Software Engineering
8 234 (18.54%) Data Engineering
9 206 (16.32%) Business Intelligence
10 201 (15.93%) Data Analytics
11 197 (15.61%) Microservices
12 195 (15.45%) Computer Science
13 192 (15.21%) Data Modelling
14 178 (14.10%) Artificial Intelligence
15 161 (12.76%) Visualisation
16 145 (11.49%) Statistics
17 131 (10.38%) Fintech
18 126 (9.98%) Scrum
19 124 (9.83%) Mentoring
20 123 (9.75%) Mathematics
Programming Languages
1 813 (64.42%) Python
2 562 (44.53%) Java
3 514 (40.73%) SQL
4 467 (37.00%) Scala
5 175 (13.87%) R
6 139 (11.01%) JavaScript
7 73 (5.78%) Bash Shell
8 72 (5.71%) Go
9 60 (4.75%) C++
10 48 (3.80%) C#
11 47 (3.72%) Java 8
12 45 (3.57%) C
13 39 (3.09%) TypeScript
14 27 (2.14%) PL/SQL
15 25 (1.98%) Kotlin
16 23 (1.82%) PHP
17 17 (1.35%) Shell Script
18 14 (1.11%) ES6
19 13 (1.03%) Ruby
20 9 (0.71%) Clojure
Qualifications
1 177 (14.03%) Degree
2 80 (6.34%) PhD
3 60 (4.75%) Computer Science Degree
4 45 (3.57%) Master's Degree
5 29 (2.30%) Numerate Degree
6 10 (0.79%) Microsoft Certification
6 10 (0.79%) Security Cleared
7 8 (0.63%) CCNA
7 8 (0.63%) Cisco Certification
7 8 (0.63%) CISSP
7 8 (0.63%) CompTIA Security+
7 8 (0.63%) GIAC
7 8 (0.63%) Linux+ Certification
7 8 (0.63%) MCSE
7 8 (0.63%) Network+ Certification
7 8 (0.63%) SANS
7 8 (0.63%) SSCP
8 5 (0.40%) PMI Certification
8 5 (0.40%) PMP
9 3 (0.24%) MBA
Quality Assurance & Compliance
1 79 (6.26%) Data Quality
2 62 (4.91%) QA
3 11 (0.87%) GDPR
3 11 (0.87%) ISO/IEC 27001
4 3 (0.24%) EMIR
4 3 (0.24%) MiFID
System Software
1 362 (28.68%) Docker
2 92 (7.29%) HDFS
3 29 (2.30%) Apache Flume
4 24 (1.90%) VirtualBox
5 10 (0.79%) Cronacle
6 8 (0.63%) Apache ZooKeeper
7 3 (0.24%) Varnish
8 1 (0.079%) Active Directory
Systems Management
1 300 (23.77%) Kubernetes
2 127 (10.06%) Ansible
3 126 (9.98%) Terraform
4 77 (6.10%) Hadoop YARN
5 59 (4.68%) Oozie
6 29 (2.30%) Puppet
7 26 (2.06%) Docker Swarm
8 20 (1.58%) Consul
9 18 (1.43%) Ambari
10 16 (1.27%) Control-M
11 14 (1.11%) Opscode Chef
12 11 (0.87%) Mesos
13 10 (0.79%) Single Sign-On
14 8 (0.63%) Rancher
15 7 (0.55%) Kibana
16 6 (0.48%) logstash
16 6 (0.48%) Marathon
17 5 (0.40%) BMC PATROL
17 5 (0.40%) HP Quality Center
17 5 (0.40%) Tivoli
Vendors
1 216 (17.12%) Google
2 98 (7.77%) Hortonworks
3 97 (7.69%) Tableau
4 83 (6.58%) Cloudera
5 66 (5.23%) Microsoft
6 56 (4.44%) Oracle
7 48 (3.80%) Informatica
8 46 (3.65%) Databricks
8 46 (3.65%) Talend
9 44 (3.49%) Splunk
10 33 (2.61%) IBM
11 29 (2.30%) WorldPay
12 20 (1.58%) Teradata
13 12 (0.95%) FIS
13 12 (0.95%) MapR
14 10 (0.79%) Dataiku
14 10 (0.79%) Netezza
15 8 (0.63%) Ab Initio
15 8 (0.63%) VMware
16 7 (0.55%) Sybase