Period
to 19 June 2018

The following table provides summary statistics for permanent job vacancies advertised in the City of London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 19 June 2018 with a comparison to the same period in the previous 2 years.

Apache Spark
Central London > City of London
6 months to
19 Jun 2018
Same period 2017 Same period 2016
Rank 134 201 204
Rank change year-on-year +67 +3 +210
Permanent jobs citing Apache Spark 258 131 192
As % of all permanent IT jobs advertised in the City of London 2.34% 1.29% 1.43%
As % of the Application Platforms category 13.93% 7.52% 7.67%
Number of salaries quoted 233 117 168
City of London median annual salary £80,000 £70,000 £65,000
Median salary % change year-on-year +14.29% +7.69% -10.34%
10th Percentile £60,000 £52,500 £45,000
90th Percentile £109,750 £102,000 £105,000
Central London median annual salary £80,000 £70,000 £65,000
% change year-on-year +14.29% +7.69% -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in the City of London with a requirement for application platform skills.

All Application Platform Skills
City of London
Permanent vacancies with a requirement for application platform skills 1,852 1,742 2,502
As % of all permanent IT jobs advertised in the City of London 16.79% 17.20% 18.69%
Number of salaries quoted 1,647 1,556 2,244
City of London median annual salary £60,000 £57,500 £55,000
Median salary % change year-on-year +4.35% +4.55% +4.76%
10th Percentile £36,250 £31,750 £31,250
90th Percentile £90,000 £82,500 £82,500
Central London median annual salary £57,500 £55,000 £55,000
% change year-on-year +4.55% - +4.76%

Apache Spark
Job Vacancy Trend in the City of London

Job postings citing Apache Spark as a percentage of all IT jobs advertised in the City of London.

Job vacancy trend for Apache Spark in the City of London

Apache Spark
Salary Trend in the City of London

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in the City of London.

Salary trend for Apache Spark in the City of London

Apache Spark
Salary Histogram in the City of London

The salary distribution of IT jobs citing Apache Spark in the City of London over the 6 months to 19 June 2018.

Salary histogram for Apache Spark in the City of London

Apache Spark
Job Locations in the City of London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the City of London region over the 6 months to 19 June 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
London Bridge - 4 £80,000 -
Apache Spark
Central London

For the 6 months to 19 June 2018, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the City of London region with a requirement for Apache Spark.

1 169 (65.50%) Big Data
2 166 (64.34%) Hadoop
3 163 (63.18%) Java
4 152 (58.91%) Python
5 145 (56.20%) Scala
6 97 (37.60%) Amazon AWS
7 95 (36.82%) Analytics
8 94 (36.43%) Machine Learning
9 89 (34.50%) SQL
10 88 (34.11%) Kafka
11 81 (31.40%) Finance
12 80 (31.01%) Data Science
13 77 (29.84%) NoSQL
14 71 (27.52%) Agile Software Development
15 60 (23.26%) Artificial Intelligence
16 57 (22.09%) Data Lake
17 52 (20.16%) Docker
18 46 (17.83%) Microsoft Azure
18 46 (17.83%) Apache Hive
19 44 (17.05%) Data Engineering
20 42 (16.28%) TDD
21 41 (15.89%) Apache Cassandra
22 40 (15.50%) Degree
22 40 (15.50%) Mathematics
22 40 (15.50%) ETL
22 40 (15.50%) Java 8
23 39 (15.12%) Linux
23 39 (15.12%) R
24 37 (14.34%) Unix
24 37 (14.34%) Software Engineering

Apache Spark
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 35 (13.57%) Elasticsearch
2 32 (12.40%) Apache
3 9 (3.49%) Apache Solr
4 7 (2.71%) nginx
4 7 (2.71%) SAS
5 6 (2.33%) Apache Pig
6 4 (1.55%) Oracle Workflow
7 3 (1.16%) Confluence
8 1 (0.39%) Apache Airflow
8 1 (0.39%) Cloud Foundry
8 1 (0.39%) CMS
8 1 (0.39%) NServiceBus
8 1 (0.39%) SharePoint Server
Applications
1 5 (1.94%) Microsoft Excel
2 1 (0.39%) Microsoft Office
2 1 (0.39%) Microsoft PowerPoint
2 1 (0.39%) MS Visio
2 1 (0.39%) SPSS
Cloud Services
1 97 (37.60%) Amazon AWS
2 46 (17.83%) Microsoft Azure
3 13 (5.04%) Amazon EMR
4 10 (3.88%) Amazon Kinesis
5 8 (3.10%) Amazon EC2
6 6 (2.33%) GitHub
6 6 (2.33%) Google Cloud Platform
7 5 (1.94%) OpenShift
8 4 (1.55%) Datadog
8 4 (1.55%) SaaS
9 3 (1.16%) Azure Data Factory
10 2 (0.78%) Amazon ElastiCache
10 2 (0.78%) Amazon S3
10 2 (0.78%) AWS Lambda
10 2 (0.78%) Cortana Intelligence Suite
10 2 (0.78%) Route 53
10 2 (0.78%) Serverless
11 1 (0.39%) Cloud Computing
11 1 (0.39%) Google Analytics
11 1 (0.39%) IaaS
Communications & Networking
1 4 (1.55%) DNS
1 4 (1.55%) Firewall
1 4 (1.55%) HTTP
1 4 (1.55%) SMTP
1 4 (1.55%) TCP/IP
1 4 (1.55%) VPN
2 1 (0.39%) Internet
Database & Business Intelligence
1 169 (65.50%) Big Data
2 166 (64.34%) Hadoop
3 77 (29.84%) NoSQL
4 57 (22.09%) Data Lake
5 46 (17.83%) Apache Hive
6 41 (15.89%) Apache Cassandra
7 24 (9.30%) Data Warehouse
8 22 (8.53%) HBase
8 22 (8.53%) Relational Database
9 20 (7.75%) MongoDB
10 19 (7.36%) SQL Server
11 16 (6.20%) Cosmos DB
11 16 (6.20%) MySQL
12 15 (5.81%) Amazon Redshift
12 15 (5.81%) Power BI
13 11 (4.26%) Sqoop
14 10 (3.88%) Neo4j
15 9 (3.49%) PostgreSQL
15 9 (3.49%) Redis
15 9 (3.49%) SQL Server Integration Services
Development Applications
1 15 (5.81%) Jenkins
2 13 (5.04%) Git (software)
3 10 (3.88%) RStudio
4 6 (2.33%) JIRA
5 5 (1.94%) Artifactory
5 5 (1.94%) TeamCity
5 5 (1.94%) Travis CI
6 4 (1.55%) Atlassian Bamboo
6 4 (1.55%) GoCD
6 4 (1.55%) Octopus Deploy
6 4 (1.55%) Sonatype Nexus
7 3 (1.16%) JUnit
7 3 (1.16%) Maven
8 2 (0.78%) Gatling
8 2 (0.78%) Mockito
8 2 (0.78%) Oracle APEX
8 2 (0.78%) Selenium
9 1 (0.39%) CA Release Automation
9 1 (0.39%) Cucumber
9 1 (0.39%) Visual Studio Team System
General
1 81 (31.40%) Finance
2 33 (12.79%) Banking
3 19 (7.36%) Publishing
4 11 (4.26%) Pharmaceutical
5 10 (3.88%) Games
6 9 (3.49%) Investment Banking
6 9 (3.49%) Retail
7 7 (2.71%) Marketing
8 4 (1.55%) Front Office
9 3 (1.16%) Advertising
9 3 (1.16%) Retail Banking
10 2 (0.78%) Telecoms
11 1 (0.39%) Financial Institution
11 1 (0.39%) Law
Job Titles
1 108 (41.86%) Developer
2 57 (22.09%) Data Engineer
3 46 (17.83%) Java Developer
4 37 (14.34%) Big Data Developer
5 35 (13.57%) Scala Developer
6 33 (12.79%) Data Scientist
7 30 (11.63%) Big Data Engineer
8 19 (7.36%) Senior Data Scientist
9 14 (5.43%) Apache Spark Developer
9 14 (5.43%) Hadoop Developer
10 13 (5.04%) Java Scala Developer
11 12 (4.65%) Java Engineer
12 11 (4.26%) Artificial Intelligence Developer
12 11 (4.26%) Artificial Intelligence Engineer
12 11 (4.26%) Machine Learning Engineer
12 11 (4.26%) Senior Developer
13 10 (3.88%) Architect
13 10 (3.88%) C++ Developer
13 10 (3.88%) Java Hadoop Developer
14 9 (3.49%) Hadoop Engineer
Libraries, Frameworks & Software Standards
1 88 (34.11%) Kafka
2 26 (10.08%) Natural Language Processing
3 20 (7.75%) Akka
3 20 (7.75%) Web Services
4 12 (4.65%) .NET
4 12 (4.65%) REST
5 10 (3.88%) Elastic Stack
5 10 (3.88%) RabbitMQ
6 9 (3.49%) scikit-learn
6 9 (3.49%) Spring
7 8 (3.10%) Pandas
7 8 (3.10%) Spark Streaming
8 6 (2.33%) D3.js
9 5 (1.94%) Node.js
9 5 (1.94%) React
10 4 (1.55%) AngularJS
10 4 (1.55%) LDAP
10 4 (1.55%) NumPy
10 4 (1.55%) RESTful
10 4 (1.55%) SciPy
Miscellaneous
1 36 (13.95%) Fintech
2 34 (13.18%) Algorithms
3 24 (9.30%) Greenfield Project
4 20 (7.75%) Analytical Skills
5 18 (6.98%) Computer Science
6 12 (4.65%) Self-Motivation
7 10 (3.88%) Data Structures
7 10 (3.88%) Linux Command Line
8 6 (2.33%) Derivative
9 5 (1.94%) Life Science
9 5 (1.94%) Tandem
10 4 (1.55%) Blog
10 4 (1.55%) Public Cloud
11 3 (1.16%) Hedge funds
11 3 (1.16%) Market Risk
11 3 (1.16%) Semantic Web
12 2 (0.78%) Mobile App
12 2 (0.78%) Reinsurance
12 2 (0.78%) Social Media
12 2 (0.78%) Virtual Team
Operating Systems
1 39 (15.12%) Linux
2 37 (14.34%) Unix
3 2 (0.78%) Ubuntu
4 1 (0.39%) DC/OS
Processes & Methodologies
1 95 (36.82%) Analytics
2 94 (36.43%) Machine Learning
3 80 (31.01%) Data Science
4 71 (27.52%) Agile Software Development
5 60 (23.26%) Artificial Intelligence
6 44 (17.05%) Data Engineering
7 42 (16.28%) TDD
8 40 (15.50%) ETL
8 40 (15.50%) Mathematics
9 37 (14.34%) Software Engineering
10 36 (13.95%) DevOps
11 34 (13.18%) Open Source
12 28 (10.85%) BDD
13 27 (10.47%) Business Intelligence
14 24 (9.30%) Deep Learning
15 22 (8.53%) Prototyping
16 21 (8.14%) Statistics
17 19 (7.36%) Microservices
17 19 (7.36%) SDLC
18 18 (6.98%) Data Analytics
Programming Languages
1 163 (63.18%) Java
2 152 (58.91%) Python
3 145 (56.20%) Scala
4 89 (34.50%) SQL
5 40 (15.50%) Java 8
6 39 (15.12%) R
7 24 (9.30%) C#
7 24 (9.30%) JavaScript
8 21 (8.14%) C++
9 9 (3.49%) Go
10 7 (2.71%) MATLAB
10 7 (2.71%) Perl
10 7 (2.71%) Shell Script
11 6 (2.33%) C
11 6 (2.33%) T-SQL
12 5 (1.94%) Bash Shell
12 5 (1.94%) Ruby
13 4 (1.55%) Haskell
13 4 (1.55%) Korn
13 4 (1.55%) VBA
Qualifications
1 40 (15.50%) Degree
2 9 (3.49%) PhD
3 8 (3.10%) Computer Science Degree
4 5 (1.94%) Security Cleared
5 4 (1.55%) Master's Degree
6 3 (1.16%) MBA
7 1 (0.39%) Microsoft Certification
7 1 (0.39%) Numerate Degree
Quality Assurance & Compliance
1 9 (3.49%) Data Quality
2 4 (1.55%) Solvency II
3 3 (1.16%) GDPR
3 3 (1.16%) QA
4 2 (0.78%) EMIR
4 2 (0.78%) MiFID
5 1 (0.39%) Dodd-Frank
5 1 (0.39%) FATCA
5 1 (0.39%) GCP
System Software
1 52 (20.16%) Docker
2 19 (7.36%) HDFS
3 18 (6.98%) Apache Flume
4 5 (1.94%) KVM
4 5 (1.94%) Virtual Machines
4 5 (1.94%) VMware Infrastructure
4 5 (1.94%) Xen
5 4 (1.55%) VirtualBox
6 3 (1.16%) Apache ZooKeeper
6 3 (1.16%) Oracle RAC
7 2 (0.78%) LXC
8 1 (0.39%) Active Directory
Systems Management
1 32 (12.40%) Kubernetes
2 25 (9.69%) Ansible
3 23 (8.91%) Mesos
4 15 (5.81%) Opscode Chef
5 13 (5.04%) Puppet
6 12 (4.65%) Terraform
7 10 (3.88%) Hadoop YARN
8 9 (3.49%) Oozie
9 8 (3.10%) Docker Swarm
10 5 (1.94%) Consul
10 5 (1.94%) Red Hat Spacewalk
10 5 (1.94%) Salt
11 4 (1.55%) Prometheus
12 3 (1.16%) Data Guard
12 3 (1.16%) Oracle RMAN
13 2 (0.78%) Simian Army
14 1 (0.39%) Graphite
14 1 (0.39%) Kibana
14 1 (0.39%) logstash
14 1 (0.39%) rkt
Vendors
1 24 (9.30%) Oracle
2 23 (8.91%) Tableau
3 22 (8.53%) Google
4 19 (7.36%) Cloudera
5 17 (6.59%) Hortonworks
6 15 (5.81%) Microsoft
7 13 (5.04%) MapR
8 9 (3.49%) Informatica
8 9 (3.49%) VMware
9 7 (2.71%) Alteryx
9 7 (2.71%) Facebook
10 6 (2.33%) Atlassian
11 4 (1.55%) Datameer
11 4 (1.55%) IBM
11 4 (1.55%) New Relic
11 4 (1.55%) SaltStack
11 4 (1.55%) Sun
12 3 (1.16%) Talend
13 2 (0.78%) Apple
14 1 (0.39%) Greenplum