Period
to 16 August 2018

The following table provides summary statistics for permanent job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 16 August 2018 with a comparison to the same period in the previous 2 years.

Apache Spark
England > London
6 months to
16 Aug 2018
Same period 2017 Same period 2016
Rank 149 200 262
Rank change year-on-year +51 +62 +261
Permanent jobs citing Apache Spark 1,601 1,383 1,070
As % of all permanent IT jobs advertised in London 2.31% 1.87% 1.32%
As % of the Application Platforms category 12.93% 9.77% 6.90%
Number of salaries quoted 1,215 1,136 933
London median annual salary £75,000 £70,000 £70,000
Median salary % change year-on-year +7.14% - -
10th Percentile £51,250 £48,750 £47,500
90th Percentile £106,250 £107,500 £110,000
England median annual salary £70,000 £67,500 £65,000
% change year-on-year +3.70% +3.85% -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Permanent vacancies with a requirement for application platform skills 12,379 14,161 15,513
As % of all permanent IT jobs advertised in London 17.88% 19.14% 19.15%
Number of salaries quoted 9,843 11,266 13,613
London median annual salary £60,000 £57,500 £55,000
Median salary % change year-on-year +4.35% +4.55% +4.76%
10th Percentile £33,750 £32,500 £31,250
90th Percentile £90,000 £86,875 £82,500
England median annual salary £50,000 £47,500 £45,000
% change year-on-year +5.26% +5.56% -

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a percentage of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Salary Trend in London

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in London.

Salary trend for Apache Spark in London

Apache Spark
Salary Histogram in London

The salary distribution of IT jobs citing Apache Spark in London over the 6 months to 16 August 2018.

Salary histogram for Apache Spark in London

Apache Spark
Job Locations in London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the London region over the 6 months to 16 August 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +51 293 £80,000 +10.34% 64
East London +31 34 £95,000 +52.00% 6
West London +21 20 £67,500 -5.26% 14
South London -4 3 £70,000 +14.29% 2
Apache Spark
England

For the 6 months to 16 August 2018, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the London region with a requirement for Apache Spark.

1 1,135 (70.89%) Hadoop
2 1,129 (70.52%) Big Data
3 974 (60.84%) Python
4 806 (50.34%) Java
5 757 (47.28%) Scala
6 648 (40.47%) Kafka
7 644 (40.22%) Analytics
8 626 (39.10%) SQL
9 574 (35.85%) Agile Software Development
10 543 (33.92%) Machine Learning
11 523 (32.67%) Amazon AWS
12 520 (32.48%) Data Science
13 469 (29.29%) Apache Hive
14 461 (28.79%) NoSQL
15 371 (23.17%) Finance
16 303 (18.93%) Apache Cassandra
17 298 (18.61%) ETL
18 289 (18.05%) Degree
19 272 (16.99%) R
20 268 (16.74%) Linux
21 254 (15.87%) DevOps
22 253 (15.80%) Analytical Skills
22 253 (15.80%) Problem-Solving
23 241 (15.05%) Software Engineering
24 240 (14.99%) Apache
25 238 (14.87%) HDFS
26 237 (14.80%) Algorithms
27 233 (14.55%) Computer Science
28 228 (14.24%) Docker
28 228 (14.24%) Microsoft Azure

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 240 (14.99%) Apache
2 144 (8.99%) Elasticsearch
3 84 (5.25%) Apache Pig
4 67 (4.18%) Apache Solr
5 55 (3.44%) SAS
6 22 (1.37%) Tomcat
7 14 (0.87%) Apache Airflow
8 13 (0.81%) OpenStack
9 10 (0.62%) nginx
10 9 (0.56%) CMS
10 9 (0.56%) Confluence
11 8 (0.50%) GemFire
11 8 (0.50%) Jupyter
12 7 (0.44%) NServiceBus
12 7 (0.44%) Oracle Workflow
13 6 (0.37%) JBoss
14 5 (0.31%) IIS
14 5 (0.31%) Jetty
14 5 (0.31%) Oracle SOA Suite
14 5 (0.31%) WebSphere
Applications
1 21 (1.31%) Microsoft Excel
2 14 (0.87%) SPSS
3 9 (0.56%) Chatbot
4 8 (0.50%) GNU Octave
5 6 (0.37%) MS Visio
6 5 (0.31%) Microsoft PowerPoint
7 3 (0.19%) Microsoft Office
8 2 (0.12%) CPLEX
8 2 (0.12%) Mathematica
9 1 (0.062%) Mingle
9 1 (0.062%) SPSS Modeler
9 1 (0.062%) Weka
Business Applications
1 3 (0.19%) Distributed Ledger
2 2 (0.12%) NetSuite
3 1 (0.062%) Actimize
3 1 (0.062%) Adobe Campaign
3 1 (0.062%) Eloqua
3 1 (0.062%) Oracle AML
3 1 (0.062%) WebTrends
Cloud Services
1 523 (32.67%) Amazon AWS
2 228 (14.24%) Microsoft Azure
3 116 (7.25%) Google Cloud Platform
4 96 (6.00%) Amazon EMR
5 66 (4.12%) Amazon Kinesis
6 64 (4.00%) Serverless
7 59 (3.69%) AWS Lambda
8 56 (3.50%) Amazon S3
9 33 (2.06%) Amazon EC2
10 31 (1.94%) IBM Cloud
11 29 (1.81%) GitHub
12 28 (1.75%) Cloud Computing
13 22 (1.37%) PaaS
14 19 (1.19%) Azure Machine Learning
14 19 (1.19%) SaaS
15 16 (1.00%) AWS CloudFormation
16 14 (0.87%) Google Analytics
16 14 (0.87%) IaaS
17 10 (0.62%) Amazon SageMaker
18 9 (0.56%) Slack
Communications & Networking
1 25 (1.56%) Sourcefire
2 24 (1.50%) HTTP
3 21 (1.31%) Kerberos
4 12 (0.75%) Internet
5 11 (0.69%) DNS
6 10 (0.62%) Network Security
7 9 (0.56%) Firewall
8 5 (0.31%) SMTP
8 5 (0.31%) TCP/IP
8 5 (0.31%) VPN
9 3 (0.19%) SMS
10 2 (0.12%) DAB
10 2 (0.12%) DSL
10 2 (0.12%) Intrusion Detection
10 2 (0.12%) SSH
11 1 (0.062%) 5G
11 1 (0.062%) IPFS
11 1 (0.062%) IPsec
11 1 (0.062%) Peer-to-Peer Networking
11 1 (0.062%) SSL
Database & Business Intelligence
1 1,135 (70.89%) Hadoop
2 1,129 (70.52%) Big Data
3 469 (29.29%) Apache Hive
4 461 (28.79%) NoSQL
5 303 (18.93%) Apache Cassandra
6 216 (13.49%) HBase
7 189 (11.81%) MongoDB
8 161 (10.06%) Relational Database
9 158 (9.87%) Amazon Redshift
10 144 (8.99%) Data Warehouse
11 141 (8.81%) Data Lake
12 140 (8.74%) Impala
13 100 (6.25%) MySQL
14 92 (5.75%) PostgreSQL
15 91 (5.68%) SQL Server
16 80 (5.00%) RDBMS
17 75 (4.68%) Flink
18 65 (4.06%) Redis
19 62 (3.87%) Power BI
20 60 (3.75%) Sqoop
Development Applications
1 134 (8.37%) Git (software)
2 130 (8.12%) Jenkins
3 70 (4.37%) JUnit
4 51 (3.19%) Maven
5 39 (2.44%) Gradle
6 36 (2.25%) Atlassian Bamboo
7 32 (2.00%) JIRA
7 32 (2.00%) Vagrant
8 24 (1.50%) Subversion
9 21 (1.31%) Cucumber
9 21 (1.31%) TeamCity
10 20 (1.25%) Selenium
11 17 (1.06%) Artifactory
11 17 (1.06%) Mockito
12 16 (1.00%) RStudio
13 15 (0.94%) GoCD
14 13 (0.81%) GitLab
15 12 (0.75%) gulp
16 11 (0.69%) ERwin
16 11 (0.69%) JMeter
General
1 371 (23.17%) Finance
2 223 (13.93%) Banking
3 139 (8.68%) Investment Banking
4 89 (5.56%) Front Office
5 83 (5.18%) Financial Institution
6 79 (4.93%) Games
7 76 (4.75%) Retail
8 64 (4.00%) Marketing
9 59 (3.69%) Telecoms
10 38 (2.37%) Pharmaceutical
11 36 (2.25%) Advertising
12 25 (1.56%) Law
13 21 (1.31%) Automotive
14 19 (1.19%) Manufacturing
15 18 (1.12%) Publishing
16 15 (0.94%) Electronics
17 10 (0.62%) Military
18 6 (0.37%) Legal
18 6 (0.37%) Music Industry
18 6 (0.37%) Retail Banking
Job Titles
1 428 (26.73%) Developer
2 392 (24.48%) Data Engineer
3 259 (16.18%) Data Scientist
4 234 (14.62%) Big Data Engineer
5 173 (10.81%) Java Developer
6 141 (8.81%) Architect
7 137 (8.56%) Big Data Developer
8 123 (7.68%) Senior Data Scientist
9 108 (6.75%) Scala Developer
10 100 (6.25%) Data Architect
11 73 (4.56%) Big Data Architect
12 63 (3.94%) Software Engineer
13 60 (3.75%) Java Engineer
14 59 (3.69%) Senior Developer
15 57 (3.56%) Hadoop Engineer
16 52 (3.25%) Hadoop Developer
17 51 (3.19%) DevOps Engineer
17 51 (3.19%) Python Engineer
18 43 (2.69%) AWS Engineer
19 42 (2.62%) Python Developer
Libraries, Frameworks & Software Standards
1 648 (40.47%) Kafka
2 136 (8.49%) Web Services
3 125 (7.81%) Natural Language Processing
4 111 (6.93%) Spring
5 105 (6.56%) RESTful
6 97 (6.06%) Spark Streaming
7 88 (5.50%) scikit-learn
8 87 (5.43%) Pandas
9 86 (5.37%) TensorFlow
10 83 (5.18%) AngularJS
11 81 (5.06%) Akka
12 80 (5.00%) REST
13 75 (4.68%) .NET
14 73 (4.56%) Node.js
15 71 (4.43%) RabbitMQ
16 64 (4.00%) HTML
17 61 (3.81%) NumPy
18 60 (3.75%) Elastic Stack
19 56 (3.50%) React
20 52 (3.25%) SciPy
Miscellaneous
1 253 (15.80%) Analytical Skills
2 237 (14.80%) Algorithms
3 233 (14.55%) Computer Science
4 139 (8.68%) Fintech
5 101 (6.31%) Data Structures
6 80 (5.00%) Greenfield Project
7 78 (4.87%) Cyberthreat
8 56 (3.50%) Self-Motivation
9 44 (2.75%) Management Information System
9 44 (2.75%) User Experience
10 43 (2.69%) Derivative
11 36 (2.25%) Virtual Team
12 32 (2.00%) Equities
12 32 (2.00%) Linux Command Line
13 25 (1.56%) Life Science
14 24 (1.50%) Fixed Income
14 24 (1.50%) Foreign Exchange (FX)
14 24 (1.50%) Market Risk
15 23 (1.44%) Hedge funds
16 20 (1.25%) Public Cloud
Operating Systems
1 268 (16.74%) Linux
2 164 (10.24%) Unix
3 24 (1.50%) Windows
4 17 (1.06%) Ubuntu
5 9 (0.56%) CentOS
6 6 (0.37%) CoreOS
7 4 (0.25%) Amazon Linux
7 4 (0.25%) Android
7 4 (0.25%) Red Hat Enterprise Linux
8 3 (0.19%) Apple iOS
8 3 (0.19%) Debian
9 2 (0.12%) AIX
9 2 (0.12%) Windows Server
10 1 (0.062%) Mesosphere
10 1 (0.062%) Solaris
10 1 (0.062%) Windows Server 2008
Processes & Methodologies
1 644 (40.22%) Analytics
2 574 (35.85%) Agile Software Development
3 543 (33.92%) Machine Learning
4 520 (32.48%) Data Science
5 298 (18.61%) ETL
6 254 (15.87%) DevOps
7 253 (15.80%) Problem-Solving
8 241 (15.05%) Software Engineering
9 224 (13.99%) TDD
10 205 (12.80%) Artificial Intelligence
11 204 (12.74%) Mathematics
12 185 (11.56%) Data Analytics
13 181 (11.31%) Data Engineering
14 175 (10.93%) Visualisation
15 168 (10.49%) Scrum
16 156 (9.74%) Open Source
17 154 (9.62%) Data Modelling
18 144 (8.99%) Continuous Integration
19 136 (8.49%) Statistics
20 129 (8.06%) Business Intelligence
Programming Languages
1 974 (60.84%) Python
2 806 (50.34%) Java
3 757 (47.28%) Scala
4 626 (39.10%) SQL
5 272 (16.99%) R
6 145 (9.06%) JavaScript
7 133 (8.31%) Java 8
8 122 (7.62%) C#
8 122 (7.62%) C++
9 98 (6.12%) Perl
10 75 (4.68%) Bash Shell
11 67 (4.18%) Shell Script
12 64 (4.00%) MATLAB
13 59 (3.69%) Ruby
14 56 (3.50%) C
15 37 (2.31%) T-SQL
16 36 (2.25%) Go
17 23 (1.44%) PHP
18 19 (1.19%) PL/SQL
19 18 (1.12%) HiveQL
Qualifications
1 289 (18.05%) Degree
2 102 (6.37%) Computer Science Degree
3 90 (5.62%) PhD
4 43 (2.69%) Master's Degree
5 41 (2.56%) Numerate Degree
6 37 (2.31%) Security Cleared
7 25 (1.56%) CEH
7 25 (1.56%) CISSP
7 25 (1.56%) GCIH
7 25 (1.56%) OSCP
8 19 (1.19%) DV Cleared
9 9 (0.56%) ISTQB
10 7 (0.44%) ISTQB Foundation Certification
11 3 (0.19%) ISEB
12 2 (0.12%) AWS Certification
12 2 (0.12%) MCSA
12 2 (0.12%) MCSD
12 2 (0.12%) Microsoft Certification
12 2 (0.12%) SC Cleared
13 1 (0.062%) Postgraduate
Quality Assurance & Compliance
1 54 (3.37%) QA
2 26 (1.62%) Data Quality
2 26 (1.62%) GCP
3 12 (0.75%) MiFID
4 11 (0.69%) ISO/IEC 27001
5 9 (0.56%) GDPR
5 9 (0.56%) GPG13
6 3 (0.19%) SLA
6 3 (0.19%) Solvency II
7 1 (0.062%) EMIR
7 1 (0.062%) HL7
7 1 (0.062%) PCI DSS
System Software
1 238 (14.87%) HDFS
2 228 (14.24%) Docker
3 121 (7.56%) Apache Flume
4 34 (2.12%) VirtualBox
5 25 (1.56%) Snort
6 21 (1.31%) Apache ZooKeeper
7 18 (1.12%) Active Directory
7 18 (1.12%) VMware Infrastructure
8 10 (0.62%) Hyper-V
9 6 (0.37%) Virtual Machines
10 4 (0.25%) Xen
11 3 (0.19%) Oracle RAC
12 2 (0.12%) KVM
12 2 (0.12%) LXC
13 1 (0.062%) Cronacle
Systems Management
1 137 (8.56%) Kubernetes
2 108 (6.75%) Oozie
3 92 (5.75%) Ansible
4 76 (4.75%) Terraform
5 71 (4.43%) Hadoop YARN
6 66 (4.12%) Mesos
7 39 (2.44%) Puppet
8 35 (2.19%) Prometheus
9 29 (1.81%) Opscode Chef
10 27 (1.69%) Docker Swarm
11 25 (1.56%) Kibana
11 25 (1.56%) Suricata
12 20 (1.25%) ITRS Geneos
13 17 (1.06%) Grafana
13 17 (1.06%) logstash
14 16 (1.00%) Consul
15 12 (0.75%) Salt
16 11 (0.69%) ZABBIX
17 9 (0.56%) QRadar
18 7 (0.44%) Ambari
Vendors
1 216 (13.49%) Cloudera
2 157 (9.81%) Google
3 144 (8.99%) Tableau
4 139 (8.68%) Hortonworks
5 134 (8.37%) Oracle
6 97 (6.06%) Informatica
7 96 (6.00%) Talend
8 83 (5.18%) Microsoft
9 66 (4.12%) IBM
10 52 (3.25%) MapR
11 50 (3.12%) Splunk
12 35 (2.19%) Sybase
13 34 (2.12%) Databricks
13 34 (2.12%) SAP
14 31 (1.94%) WorldPay
15 27 (1.69%) SnapLogic
16 26 (1.62%) EXASOL
17 25 (1.56%) TippingPoint
18 23 (1.44%) AppDynamics
19 22 (1.37%) VMware