Period
to 22 October 2018

The following table provides summary statistics for permanent job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 22 October 2018 with a comparison to the same period in the previous 2 years.

Apache Spark
England > London
6 months to
22 Oct 2018
Same period 2017 Same period 2016
Rank 171 179 259
Rank change year-on-year +8 +80 +188
Permanent jobs citing Apache Spark 1,461 1,512 1,009
As % of all permanent IT jobs advertised in London 2.15% 2.04% 1.32%
As % of the Application Platforms category 12.11% 10.69% 7.00%
Number of salaries quoted 1,101 1,234 864
London median annual salary £75,000 £70,000 £75,000
Median salary % change year-on-year +7.14% -6.67% +11.11%
10th Percentile £50,000 £47,500 £48,750
90th Percentile £105,000 £105,000 £107,500
England median annual salary £70,000 £67,000 £67,500
% change year-on-year +4.48% -0.74% +3.85%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Permanent vacancies with a requirement for application platform skills 12,067 14,141 14,418
As % of all permanent IT jobs advertised in London 17.73% 19.06% 18.89%
Number of salaries quoted 9,641 11,229 12,529
London median annual salary £60,000 £57,500 £55,000
Median salary % change year-on-year +4.35% +4.55% +4.76%
10th Percentile £32,500 £32,500 £31,490
90th Percentile £92,500 £86,250 £83,750
England median annual salary £50,000 £47,500 £45,000
% change year-on-year +5.26% +5.56% -

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a percentage of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Salary Trend in London

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in London.

Salary trend for Apache Spark in London

Apache Spark
Salary Histogram in London

The salary distribution of IT jobs citing Apache Spark in London over the 6 months to 22 October 2018.

Salary histogram for Apache Spark in London

Apache Spark
Job Locations in London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the London region over the 6 months to 22 October 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +17 279 £80,000 +10.34% 40
East London +23 32 £95,000 +26.67% 11
West London +21 20 £65,000 -13.33% 19
North London +7 5 £80,000 +36.17% 3
Apache Spark
England

For the 6 months to 22 October 2018, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the London region with a requirement for Apache Spark.

1 1,059 (72.48%) Hadoop
2 1,017 (69.61%) Big Data
3 875 (59.89%) Python
4 764 (52.29%) Java
5 689 (47.16%) Scala
6 611 (41.82%) Kafka
7 558 (38.19%) Agile Software Development
8 540 (36.96%) SQL
9 528 (36.14%) Analytics
10 515 (35.25%) Amazon AWS
11 477 (32.65%) Machine Learning
12 471 (32.24%) Apache Hive
13 447 (30.60%) Data Science
14 438 (29.98%) NoSQL
15 343 (23.48%) Finance
16 317 (21.70%) Apache Cassandra
17 273 (18.69%) ETL
18 248 (16.97%) HBase
19 247 (16.91%) Software Engineering
20 242 (16.56%) DevOps
21 237 (16.22%) Apache
22 235 (16.08%) Docker
23 232 (15.88%) Degree
24 228 (15.61%) Microsoft Azure
25 223 (15.26%) Linux
26 219 (14.99%) Computer Science
27 208 (14.24%) Analytical Skills
28 207 (14.17%) R
28 207 (14.17%) Cloudera
29 205 (14.03%) MongoDB

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 237 (16.22%) Apache
2 162 (11.09%) Elasticsearch
3 112 (7.67%) Apache Pig
4 58 (3.97%) Apache Solr
4 58 (3.97%) SAS
5 17 (1.16%) Apache Airflow
5 17 (1.16%) Tomcat
6 8 (0.55%) GlassFish
7 7 (0.48%) Confluence
7 7 (0.48%) Jupyter
7 7 (0.48%) OpenStack
7 7 (0.48%) Oracle Workflow
8 6 (0.41%) GemFire
8 6 (0.41%) nginx
8 6 (0.41%) NServiceBus
9 5 (0.34%) IIS
9 5 (0.34%) SharePoint Server
9 5 (0.34%) WebSphere
9 5 (0.34%) WildFly
10 4 (0.27%) Luigi
Applications
1 15 (1.03%) SPSS
2 10 (0.68%) GNU Octave
3 9 (0.62%) Chatbot
4 4 (0.27%) Microsoft Excel
5 3 (0.21%) Microsoft Office
5 3 (0.21%) Stata
6 2 (0.14%) Microsoft PowerPoint
7 1 (0.068%) CPLEX
7 1 (0.068%) Mingle
7 1 (0.068%) MS Visio
7 1 (0.068%) SPSS Modeler
7 1 (0.068%) Weka
Business Applications
1 2 (0.14%) NetSuite
2 1 (0.068%) Adobe Campaign
2 1 (0.068%) Eloqua
2 1 (0.068%) WebTrends
Cloud Services
1 515 (35.25%) Amazon AWS
2 228 (15.61%) Microsoft Azure
3 146 (9.99%) Google Cloud Platform
4 81 (5.54%) Amazon EMR
5 67 (4.59%) Amazon Kinesis
6 64 (4.38%) Serverless
7 51 (3.49%) AWS Lambda
8 35 (2.40%) Cloud Computing
9 34 (2.33%) GitHub
10 30 (2.05%) Amazon S3
11 28 (1.92%) Amazon EC2
12 24 (1.64%) IBM Cloud
12 24 (1.64%) PaaS
13 21 (1.44%) Cortana Intelligence Suite
13 21 (1.44%) SaaS
14 20 (1.37%) AWS CloudFormation
15 18 (1.23%) Amazon SageMaker
16 14 (0.96%) Azure Functions
16 14 (0.96%) Azure Machine Learning
17 12 (0.82%) Azure Data Factory
Communications & Networking
1 26 (1.78%) HTTP
2 17 (1.16%) Sourcefire
3 10 (0.68%) Internet
4 8 (0.55%) Kerberos
5 7 (0.48%) Network Security
6 6 (0.41%) DNS
7 4 (0.27%) Firewall
8 3 (0.21%) AMQP
9 2 (0.14%) DAB
9 2 (0.14%) DSL
9 2 (0.14%) Intrusion Detection
9 2 (0.14%) SMS
9 2 (0.14%) SSH
9 2 (0.14%) VPN
10 1 (0.068%) 5G
10 1 (0.068%) IPsec
10 1 (0.068%) NX-OS
10 1 (0.068%) Peer-to-Peer Networking
10 1 (0.068%) SSL
10 1 (0.068%) VLAN
Database & Business Intelligence
1 1,059 (72.48%) Hadoop
2 1,017 (69.61%) Big Data
3 471 (32.24%) Apache Hive
4 438 (29.98%) NoSQL
5 317 (21.70%) Apache Cassandra
6 248 (16.97%) HBase
7 205 (14.03%) MongoDB
8 184 (12.59%) Amazon Redshift
9 144 (9.86%) Data Warehouse
9 144 (9.86%) Impala
10 131 (8.97%) Relational Database
11 106 (7.26%) Data Lake
12 96 (6.57%) Flink
13 90 (6.16%) SQL Server
14 77 (5.27%) PostgreSQL
15 71 (4.86%) MySQL
16 65 (4.45%) Power BI
17 58 (3.97%) SQL Server Integration Services
18 51 (3.49%) Sqoop
19 50 (3.42%) RDBMS
Development Applications
1 144 (9.86%) Git (software)
2 128 (8.76%) Jenkins
3 62 (4.24%) JUnit
4 36 (2.46%) Maven
5 29 (1.98%) Gradle
5 29 (1.98%) TeamCity
6 28 (1.92%) JIRA
7 23 (1.57%) Atlassian Bamboo
8 22 (1.51%) Subversion
8 22 (1.51%) Vagrant
9 21 (1.44%) Visual Studio
10 18 (1.23%) Visual Studio Team System
11 17 (1.16%) GoCD
12 14 (0.96%) Cucumber
12 14 (0.96%) Mockito
12 14 (0.96%) Selenium
13 13 (0.89%) Octopus Deploy
14 12 (0.82%) Artifactory
14 12 (0.82%) CircleCI
15 11 (0.75%) gulp
General
1 343 (23.48%) Finance
2 205 (14.03%) Banking
3 141 (9.65%) Investment Banking
4 79 (5.41%) Retail
5 72 (4.93%) Front Office
6 67 (4.59%) Financial Institution
7 65 (4.45%) Games
8 57 (3.90%) Marketing
9 53 (3.63%) Telecoms
10 39 (2.67%) Law
11 36 (2.46%) Advertising
12 31 (2.12%) Automotive
13 23 (1.57%) Military
14 21 (1.44%) Pharmaceutical
15 19 (1.30%) Electronics
16 16 (1.10%) Manufacturing
17 10 (0.68%) Retail Banking
18 9 (0.62%) Publishing
19 6 (0.41%) Music Industry
20 5 (0.34%) Billing
Job Titles
1 442 (30.25%) Developer
2 392 (26.83%) Data Engineer
3 251 (17.18%) Big Data Engineer
4 194 (13.28%) Data Scientist
5 177 (12.11%) Java Developer
6 123 (8.42%) Architect
6 123 (8.42%) Big Data Developer
7 108 (7.39%) Scala Developer
8 100 (6.84%) Senior Data Scientist
9 85 (5.82%) Data Architect
10 72 (4.93%) Senior Developer
11 64 (4.38%) Java Engineer
12 62 (4.24%) Software Engineer
13 58 (3.97%) Hadoop Engineer
14 56 (3.83%) Python Engineer
15 55 (3.76%) Hadoop Developer
15 55 (3.76%) Python Developer
16 53 (3.63%) Big Data Architect
16 53 (3.63%) DevOps Engineer
17 49 (3.35%) Apache Spark Developer
Libraries, Frameworks & Software Standards
1 611 (41.82%) Kafka
2 131 (8.97%) Web Services
3 109 (7.46%) RESTful
4 96 (6.57%) Spring
5 95 (6.50%) AngularJS
6 92 (6.30%) Natural Language Processing
7 90 (6.16%) Spark Streaming
8 83 (5.68%) TensorFlow
9 79 (5.41%) Node.js
10 78 (5.34%) Akka
10 78 (5.34%) Pandas
10 78 (5.34%) scikit-learn
11 75 (5.13%) .NET
12 73 (5.00%) HTML
13 72 (4.93%) REST
14 65 (4.45%) RabbitMQ
15 64 (4.38%) React
16 55 (3.76%) NumPy
16 55 (3.76%) PySpark
17 52 (3.56%) MLlib
Miscellaneous
1 219 (14.99%) Computer Science
2 208 (14.24%) Analytical Skills
3 194 (13.28%) Algorithms
4 125 (8.56%) Fintech
5 100 (6.84%) Data Structures
6 81 (5.54%) Greenfield Project
7 67 (4.59%) Cyberthreat
7 67 (4.59%) Self-Motivation
8 53 (3.63%) Derivative
9 41 (2.81%) User Experience
10 32 (2.19%) Virtual Team
11 30 (2.05%) Management Information System
12 27 (1.85%) Market Risk
13 26 (1.78%) Life Science
14 24 (1.64%) Equities
15 20 (1.37%) Blog
16 19 (1.30%) Public Cloud
17 17 (1.16%) Foreign Exchange (FX)
18 16 (1.10%) Fixed Income
19 14 (0.96%) Linux Command Line
Operating Systems
1 223 (15.26%) Linux
2 133 (9.10%) Unix
3 27 (1.85%) Windows
4 12 (0.82%) Ubuntu
5 8 (0.55%) Android
5 8 (0.55%) Apple iOS
6 6 (0.41%) CentOS
7 4 (0.27%) Red Hat Enterprise Linux
8 3 (0.21%) CoreOS
9 2 (0.14%) DC/OS
9 2 (0.14%) Windows Server
10 1 (0.068%) AIX
10 1 (0.068%) Mac OS
10 1 (0.068%) Mesosphere
10 1 (0.068%) Windows Server 2012
Processes & Methodologies
1 558 (38.19%) Agile Software Development
2 528 (36.14%) Analytics
3 477 (32.65%) Machine Learning
4 447 (30.60%) Data Science
5 273 (18.69%) ETL
6 247 (16.91%) Software Engineering
7 242 (16.56%) DevOps
8 204 (13.96%) Problem-Solving
9 192 (13.14%) Artificial Intelligence
10 187 (12.80%) Data Analytics
11 186 (12.73%) TDD
12 173 (11.84%) Open Source
13 168 (11.50%) Data Modelling
14 162 (11.09%) Microservices
14 162 (11.09%) Visualisation
15 156 (10.68%) Scrum
16 147 (10.06%) Mathematics
17 138 (9.45%) Data Engineering
18 117 (8.01%) Continuous Integration
19 112 (7.67%) Business Intelligence
Programming Languages
1 875 (59.89%) Python
2 764 (52.29%) Java
3 689 (47.16%) Scala
4 540 (36.96%) SQL
5 207 (14.17%) R
6 154 (10.54%) JavaScript
7 145 (9.92%) C#
8 124 (8.49%) C++
9 110 (7.53%) Java 8
10 59 (4.04%) Shell Script
11 58 (3.97%) C
11 58 (3.97%) Perl
12 54 (3.70%) Bash Shell
13 45 (3.08%) Ruby
14 39 (2.67%) MATLAB
15 34 (2.33%) PHP
16 29 (1.98%) Go
16 29 (1.98%) T-SQL
17 16 (1.10%) PL/SQL
17 16 (1.10%) TypeScript
Qualifications
1 232 (15.88%) Degree
2 92 (6.30%) Computer Science Degree
3 69 (4.72%) PhD
4 42 (2.87%) Master's Degree
5 26 (1.78%) Numerate Degree
5 26 (1.78%) Security Cleared
6 17 (1.16%) CEH
6 17 (1.16%) CISSP
6 17 (1.16%) GCIH
6 17 (1.16%) OSCP
7 16 (1.10%) DV Cleared
8 6 (0.41%) SC Cleared
9 5 (0.34%) ISTQB
10 4 (0.27%) ISEB
11 2 (0.14%) ISTQB Foundation Certification
11 2 (0.14%) Postgraduate
12 1 (0.068%) AWS Certification
12 1 (0.068%) MCSA
12 1 (0.068%) MCSD
12 1 (0.068%) Microsoft Certification
Quality Assurance & Compliance
1 52 (3.56%) QA
2 38 (2.60%) Data Quality
3 14 (0.96%) GDPR
3 14 (0.96%) MiFID
4 9 (0.62%) ISO/IEC 27001
5 7 (0.48%) GPG13
6 3 (0.21%) Solvency II
7 2 (0.14%) PCI DSS
7 2 (0.14%) SLA
8 1 (0.068%) HL7
System Software
1 235 (16.08%) Docker
2 205 (14.03%) HDFS
3 113 (7.73%) Apache Flume
4 25 (1.71%) VirtualBox
5 21 (1.44%) Apache ZooKeeper
6 17 (1.16%) Snort
7 13 (0.89%) VMware Infrastructure
8 12 (0.82%) Active Directory
9 10 (0.68%) Virtual Machines
10 7 (0.48%) Hyper-V
11 3 (0.21%) Oracle RAC
11 3 (0.21%) Xen
12 2 (0.14%) KVM
13 1 (0.068%) Firmware
13 1 (0.068%) LXC
13 1 (0.068%) Platform Symphony
13 1 (0.068%) VMware ESXi
Systems Management
1 135 (9.24%) Kubernetes
2 113 (7.73%) Oozie
3 86 (5.89%) Terraform
4 84 (5.75%) Ansible
5 76 (5.20%) Hadoop YARN
6 65 (4.45%) Mesos
7 44 (3.01%) Prometheus
8 42 (2.87%) Opscode Chef
8 42 (2.87%) Puppet
9 41 (2.81%) Kibana
10 30 (2.05%) Grafana
10 30 (2.05%) logstash
11 22 (1.51%) Docker Swarm
12 17 (1.16%) Suricata
13 15 (1.03%) ITRS Geneos
14 9 (0.62%) Consul
15 8 (0.55%) ZABBIX
16 7 (0.48%) QRadar
17 6 (0.41%) Ambari
18 5 (0.34%) Packer
Vendors
1 207 (14.17%) Cloudera
2 160 (10.95%) Hortonworks
3 144 (9.86%) Google
4 129 (8.83%) Tableau
5 109 (7.46%) Oracle
6 92 (6.30%) Microsoft
7 83 (5.68%) Talend
8 81 (5.54%) Informatica
9 73 (5.00%) MapR
10 72 (4.93%) Databricks
11 66 (4.52%) IBM
12 46 (3.15%) Splunk
13 44 (3.01%) SAP
14 32 (2.19%) Sybase
15 31 (2.12%) TIBCO
16 24 (1.64%) Vertica
17 20 (1.37%) Greenplum
17 20 (1.37%) HP
18 17 (1.16%) AppDynamics
18 17 (1.16%) TippingPoint