Period
to 23 August 2019

The following table provides summary statistics for permanent job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 23 August 2019 with a comparison to the same period in the previous 2 years.

Apache Spark
England > London
6 months to
23 Aug 2019
Same period 2018 Same period 2017
Rank 128 149 200
Rank change year-on-year +21 +51 +72
Permanent jobs citing Apache Spark 1,453 1,578 1,390
As % of all permanent IT jobs advertised in London 2.58% 2.29% 1.88%
As % of the Application Platforms category 16.26% 12.84% 9.84%
Number of salaries quoted 1,199 1,205 1,137
London median annual salary £80,000 £75,000 £70,000
Median salary % change year-on-year +6.67% +7.14% -1.75%
10th Percentile £55,000 £51,250 £48,750
90th Percentile £115,000 £105,000 £107,500
England median annual salary £75,000 £70,000 £67,500
% change year-on-year +7.14% +3.70% +3.85%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Permanent vacancies with a requirement for application platform skills 8,936 12,292 14,121
As % of all permanent IT jobs advertised in London 15.87% 17.82% 19.08%
Number of salaries quoted 7,261 9,789 11,216
London median annual salary £62,500 £60,000 £57,500
Median salary % change year-on-year +4.17% +4.35% +4.55%
10th Percentile £33,750 £33,500 £32,500
90th Percentile £95,000 £90,000 £86,250
England median annual salary £50,000 £50,000 £47,500
% change year-on-year - +5.26% +5.56%

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a percentage of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Salary Trend in London

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in London.

Salary trend for Apache Spark in London

Apache Spark
Salary Histogram in London

The salary distribution of IT jobs citing Apache Spark in London over the 6 months to 23 August 2019.

Salary histogram for Apache Spark in London

Apache Spark
Job Locations in London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the London region over the 6 months to 23 August 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +5 257 £80,000 - 43
West London +42 34 £70,000 +5.66% 15
East London +17 19 £70,000 -26.32% 12
North London - 5 £77,500 - 1
South London -1 4 £80,000 +14.29% 3
Apache Spark
England

For the 6 months to 23 August 2019, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the London region with a requirement for Apache Spark.

1 1,019 (70.13%) Python
2 971 (66.83%) Big Data
3 909 (62.56%) Hadoop
4 751 (51.69%) Java
5 681 (46.87%) AWS
6 651 (44.80%) Kafka
7 642 (44.18%) SQL
8 624 (42.95%) Agile Software Development
9 605 (41.64%) Scala
10 589 (40.54%) Analytics
11 528 (36.34%) Data Science
12 515 (35.44%) Machine Learning
13 480 (33.04%) Apache Hive
14 383 (26.36%) DevOps
15 372 (25.60%) Docker
16 357 (24.57%) Azure
17 351 (24.16%) NoSQL
18 316 (21.75%) Finance
19 302 (20.78%) GCP
20 299 (20.58%) Software Engineering
21 262 (18.03%) Linux
22 259 (17.83%) Data Engineering
23 248 (17.07%) ETL
24 233 (16.04%) Kubernetes
25 231 (15.90%) Apache
26 223 (15.35%) Analytical Skills
27 222 (15.28%) Algorithms
28 221 (15.21%) Data Modelling
29 220 (15.14%) Data Analytics
30 219 (15.07%) HBase

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 231 (15.90%) Apache
2 184 (12.66%) Elasticsearch
3 147 (10.12%) Apache Pig
4 62 (4.27%) Apache Solr
5 53 (3.65%) Apache Airflow
6 37 (2.55%) SAS
7 25 (1.72%) Jupyter
8 12 (0.83%) Luigi
9 9 (0.62%) IPython
10 7 (0.48%) Confluence
11 6 (0.41%) nginx
11 6 (0.41%) OpenStack
12 5 (0.34%) ArcGIS
13 3 (0.21%) Cloud Foundry
14 2 (0.14%) Tomcat
14 2 (0.14%) WebSphere
15 1 (0.069%) CMS
15 1 (0.069%) Oracle Workflow
15 1 (0.069%) Zope
Applications
1 15 (1.03%) Microsoft Excel
2 4 (0.28%) GNU Octave
3 2 (0.14%) Chatbot
3 2 (0.14%) SPSS
4 1 (0.069%) Microsoft Project
Business Applications
1 2 (0.14%) ReMetrica
1 2 (0.14%) SAP Oil and Gas
1 2 (0.14%) Towers Watson Igloo
2 1 (0.069%) Adobe Campaign
2 1 (0.069%) PolarLake
Cloud Services
1 681 (46.87%) AWS
2 357 (24.57%) Azure
3 302 (20.78%) GCP
4 77 (5.30%) Serverless
5 74 (5.09%) Amazon EMR
6 67 (4.61%) Amazon Kinesis
7 61 (4.20%) PaaS
8 59 (4.06%) Cloud Computing
9 56 (3.85%) Amazon S3
10 55 (3.79%) Power Platform
11 52 (3.58%) SaaS
12 46 (3.17%) AWS Lambda
13 41 (2.82%) Amazon EC2
14 32 (2.20%) Azure Data Factory
15 29 (2.00%) GitHub
16 21 (1.45%) Amazon ECS
17 17 (1.17%) IaaS
18 16 (1.10%) Amazon SQS
19 15 (1.03%) AWS CloudFormation
19 15 (1.03%) Cortana Intelligence Suite
Communications & Networking
1 33 (2.27%) Internet
2 24 (1.65%) HTTP
3 11 (0.76%) Wi-Fi
4 10 (0.69%) Firewall
4 10 (0.69%) TCP/IP
5 7 (0.48%) HTTPS
6 6 (0.41%) LAN
6 6 (0.41%) Software-Defined Networking
6 6 (0.41%) WAN
7 5 (0.34%) Kerberos
7 5 (0.34%) VPN
8 3 (0.21%) Unified Communications
9 2 (0.14%) DNS
9 2 (0.14%) FTP
9 2 (0.14%) SMTP
9 2 (0.14%) SSL
10 1 (0.069%) SMS
Database & Business Intelligence
1 971 (66.83%) Big Data
2 909 (62.56%) Hadoop
3 480 (33.04%) Apache Hive
4 351 (24.16%) NoSQL
5 219 (15.07%) HBase
6 196 (13.49%) Apache Cassandra
7 170 (11.70%) Impala
8 163 (11.22%) Data Warehouse
8 163 (11.22%) MongoDB
9 152 (10.46%) Amazon Redshift
10 116 (7.98%) PostgreSQL
11 113 (7.78%) Flink
12 99 (6.81%) Sqoop
13 92 (6.33%) Relational Database
14 81 (5.57%) BigQuery
15 77 (5.30%) Data Mining
16 74 (5.09%) MySQL
17 69 (4.75%) Data Lake
18 66 (4.54%) SQL Server
19 55 (3.79%) Power BI
Development Applications
1 169 (11.63%) Git (software)
2 147 (10.12%) Jenkins
3 38 (2.62%) JIRA
4 31 (2.13%) Maven
5 26 (1.79%) Gradle
6 20 (1.38%) Artifactory
7 19 (1.31%) Bitbucket
8 17 (1.17%) Atlassian Bamboo
9 16 (1.10%) RStudio
9 16 (1.10%) Subversion
10 14 (0.96%) Bitbucket Server
10 14 (0.96%) Selenium
11 13 (0.89%) Visual Studio
12 12 (0.83%) Ant
12 12 (0.83%) git-flow
13 11 (0.76%) Travis CI
14 10 (0.69%) Visual Studio Team System
15 9 (0.62%) JUnit
16 7 (0.48%) Cucumber
17 6 (0.41%) WATIR
General
1 316 (21.75%) Finance
2 140 (9.64%) Banking
3 131 (9.02%) Law
4 115 (7.91%) Retail
5 75 (5.16%) Marketing
6 40 (2.75%) Telecoms
7 39 (2.68%) Publishing
8 37 (2.55%) Front Office
9 34 (2.34%) Advertising
10 28 (1.93%) Financial Institution
10 28 (1.93%) Games
10 28 (1.93%) Investment Banking
11 17 (1.17%) Automotive
12 15 (1.03%) Public Sector
13 9 (0.62%) Billing
14 8 (0.55%) Retail Banking
15 6 (0.41%) Manufacturing
16 4 (0.28%) Electronics
16 4 (0.28%) Legal
17 3 (0.21%) Pharmaceutical
Job Titles
1 480 (33.04%) Data Engineer
2 295 (20.30%) Developer
3 192 (13.21%) Big Data Engineer
4 161 (11.08%) Data Scientist
5 154 (10.60%) Java Developer
6 133 (9.15%) Architect
7 111 (7.64%) Python Engineer
8 103 (7.09%) Data Architect
9 101 (6.95%) Senior Data Engineer
10 90 (6.19%) Senior Data Scientist
11 88 (6.06%) Big Data Developer
12 85 (5.85%) Software Engineer
13 84 (5.78%) Scala Developer
14 78 (5.37%) Hadoop Engineer
15 60 (4.13%) Big Data Architect
16 59 (4.06%) DevOps Engineer
17 58 (3.99%) Java Engineer
18 52 (3.58%) Senior Developer
19 50 (3.44%) Python Developer
20 48 (3.30%) AWS Engineer
Libraries, Frameworks & Software Standards
1 651 (44.80%) Kafka
2 142 (9.77%) Web Services
3 121 (8.33%) REST
4 92 (6.33%) Pandas
4 92 (6.33%) TensorFlow
5 87 (5.99%) scikit-learn
6 78 (5.37%) NumPy
7 77 (5.30%) Natural Language Processing
8 75 (5.16%) Spring
9 69 (4.75%) Apache NiFi
10 66 (4.54%) Spark Streaming
11 65 (4.47%) React
12 60 (4.13%) Elastic Stack
12 60 (4.13%) JSON
13 55 (3.79%) RESTful
14 51 (3.51%) .NET
15 50 (3.44%) Akka
15 50 (3.44%) Keras
15 50 (3.44%) Node.js
16 47 (3.23%) Apache Beam
Miscellaneous
1 223 (15.35%) Analytical Skills
2 222 (15.28%) Algorithms
3 96 (6.61%) Data Structures
4 64 (4.40%) Greenfield Project
5 37 (2.55%) Virtual Team
6 29 (2.00%) Linux Command Line
7 27 (1.86%) Cloud Native
7 27 (1.86%) Self-Motivation
8 25 (1.72%) Social Media
9 23 (1.58%) Mobile App
10 22 (1.51%) Derivative
11 20 (1.38%) User Experience
12 19 (1.31%) Analytical Mindset
13 17 (1.17%) Public Cloud
14 16 (1.10%) Equities
15 15 (1.03%) Credit Risk
15 15 (1.03%) Robotics
16 14 (0.96%) Fixed Income
16 14 (0.96%) Market Risk
16 14 (0.96%) Semantic Web
Operating Systems
1 262 (18.03%) Linux
2 127 (8.74%) Unix
3 25 (1.72%) Windows
4 13 (0.89%) CentOS
5 11 (0.76%) Ubuntu
6 8 (0.55%) Windows Server
7 6 (0.41%) Android
7 6 (0.41%) Apple iOS
8 2 (0.14%) Debian
9 1 (0.069%) Mac OS X
9 1 (0.069%) Red Hat Enterprise Linux
Processes & Methodologies
1 624 (42.95%) Agile Software Development
2 589 (40.54%) Analytics
3 528 (36.34%) Data Science
4 515 (35.44%) Machine Learning
5 383 (26.36%) DevOps
6 299 (20.58%) Software Engineering
7 259 (17.83%) Data Engineering
8 248 (17.07%) ETL
9 221 (15.21%) Data Modelling
10 220 (15.14%) Data Analytics
11 201 (13.83%) Artificial Intelligence
12 199 (13.70%) Business Intelligence
13 182 (12.53%) Microservices
14 169 (11.63%) Computer Science
15 167 (11.49%) Visualisation
16 157 (10.81%) Scrum
17 153 (10.53%) Mathematics
18 140 (9.64%) MapReduce
19 132 (9.08%) Open Source
20 125 (8.60%) TDD
Programming Languages
1 1,019 (70.13%) Python
2 751 (51.69%) Java
3 642 (44.18%) SQL
4 605 (41.64%) Scala
5 205 (14.11%) R
6 117 (8.05%) JavaScript
7 82 (5.64%) Bash Shell
8 51 (3.51%) C#
8 51 (3.51%) Java 8
9 44 (3.03%) C++
10 39 (2.68%) PL/SQL
11 37 (2.55%) Go
12 32 (2.20%) C
13 25 (1.72%) Perl
14 24 (1.65%) Ruby
15 20 (1.38%) PHP
15 20 (1.38%) TypeScript
16 16 (1.10%) Shell Script
17 14 (0.96%) Clojure
17 14 (0.96%) Groovy
Qualifications
1 202 (13.90%) Degree
2 88 (6.06%) PhD
3 74 (5.09%) Computer Science Degree
4 55 (3.79%) Numerate Degree
5 43 (2.96%) Master's Degree
6 29 (2.00%) Security Cleared
7 15 (1.03%) PMI Certification
7 15 (1.03%) PMP
8 14 (0.96%) SC Cleared
9 12 (0.83%) DV Cleared
10 7 (0.48%) Microsoft Certification
11 6 (0.41%) ISTQB
11 6 (0.41%) ISTQB Foundation Certification
12 5 (0.34%) CCNA
12 5 (0.34%) Cisco Certification
12 5 (0.34%) CISSP
12 5 (0.34%) CompTIA Security+
12 5 (0.34%) MCSE
12 5 (0.34%) SANS
12 5 (0.34%) SSCP
Quality Assurance & Compliance
1 42 (2.89%) Data Quality
2 36 (2.48%) QA
3 7 (0.48%) GDPR
4 4 (0.28%) ISO/IEC 27001
5 2 (0.14%) ISO 15022
5 2 (0.14%) ISO 20022
5 2 (0.14%) PCI DSS
System Software
1 372 (25.60%) Docker
2 99 (6.81%) HDFS
3 81 (5.57%) Apache Flume
4 60 (4.13%) Apache ZooKeeper
5 9 (0.62%) Cronacle
6 6 (0.41%) VirtualBox
7 2 (0.14%) Active Directory
7 2 (0.14%) VMware Infrastructure
8 1 (0.069%) Virtual Machines
Systems Management
1 233 (16.04%) Kubernetes
2 149 (10.25%) Ansible
3 134 (9.22%) Hadoop YARN
4 129 (8.88%) Terraform
5 128 (8.81%) Oozie
6 62 (4.27%) Opscode Chef
7 60 (4.13%) Puppet
8 38 (2.62%) Ambari
9 25 (1.72%) Mesos
10 18 (1.24%) Marathon
11 14 (0.96%) Consul
11 14 (0.96%) Nagios
12 13 (0.89%) Control-M
12 13 (0.89%) Kibana
13 7 (0.48%) Docker Swarm
13 7 (0.48%) Rancher
14 6 (0.41%) logstash
14 6 (0.41%) Microsoft Intune
14 6 (0.41%) Prometheus
14 6 (0.41%) SCCM
Vendors
1 209 (14.38%) Google
2 184 (12.66%) Cloudera
3 129 (8.88%) Tableau
4 125 (8.60%) Hortonworks
5 95 (6.54%) Oracle
6 87 (5.99%) Databricks
7 71 (4.89%) Microsoft
8 63 (4.34%) Informatica
9 61 (4.20%) Talend
10 49 (3.37%) Splunk
11 46 (3.17%) MapR
12 28 (1.93%) IBM
13 23 (1.58%) SAP
14 20 (1.38%) VMware
15 17 (1.17%) WorldPay
16 16 (1.10%) Netezza
17 15 (1.03%) Greenplum
18 14 (0.96%) TIBCO
19 12 (0.83%) Atlassian
20 11 (0.76%) HP