Period
to 21 May 2018

The following table provides summary statistics for permanent job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 21 May 2018 with a comparison to the same period in the previous 2 years.

Apache Spark
England > London
6 months to
21 May 2018
Same period 2017 Same period 2016
Rank 135 192 258
Rank change year-on-year +57 +66 +349
Permanent jobs citing Apache Spark 1,784 1,255 1,117
As % of all permanent IT jobs advertised in London 2.59% 1.84% 1.36%
As % of the Application Platforms category 14.52% 9.82% 7.20%
Number of salaries quoted 1,419 1,073 970
London median annual salary £75,000 £70,000 £70,000
Median salary % change year-on-year +7.14% - -
10th Percentile £51,250 £50,000 £45,000
90th Percentile £110,000 £107,250 £105,000
England median annual salary £70,000 £67,500 £65,000
% change year-on-year +3.70% +3.85% -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Permanent vacancies with a requirement for application platform skills 12,289 12,775 15,522
As % of all permanent IT jobs advertised in London 17.84% 18.76% 18.84%
Number of salaries quoted 9,984 10,641 13,558
London median annual salary £60,000 £57,500 £54,580
Median salary % change year-on-year +4.35% +5.35% +3.96%
10th Percentile £35,000 £32,500 £31,250
90th Percentile £90,000 £85,000 £81,250
England median annual salary £50,000 £47,500 £45,000
% change year-on-year +5.26% +5.56% -

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a percentage of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Salary Trend in London

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in London.

Salary trend for Apache Spark in London

Apache Spark
Salary Histogram in London

The salary distribution of IT jobs citing Apache Spark in London over the 6 months to 21 May 2018.

Salary histogram for Apache Spark in London

Apache Spark
Job Locations in London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the London region over the 6 months to 21 May 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +70 306 £77,500 +10.71% 69
East London +26 28 £95,000 +35.71% 13
West London +36 27 £67,500 +22.73% 24
South London -4 9 £100,000 +63.27%
North London - 1 - - 1
Apache Spark
England

For the 6 months to 21 May 2018, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the London region with a requirement for Apache Spark.

1 1,252 (70.18%) Hadoop
2 1,220 (68.39%) Big Data
3 1,104 (61.88%) Python
4 880 (49.33%) Java
5 828 (46.41%) Scala
6 746 (41.82%) Analytics
7 663 (37.16%) Machine Learning
8 655 (36.72%) SQL
9 640 (35.87%) Data Science
10 626 (35.09%) Kafka
11 603 (33.80%) Agile Software Development
12 543 (30.44%) Amazon AWS
13 461 (25.84%) NoSQL
14 453 (25.39%) Apache Hive
15 377 (21.13%) Finance
16 362 (20.29%) R
17 355 (19.90%) Apache Cassandra
18 313 (17.54%) Algorithms
19 304 (17.04%) Degree
20 302 (16.93%) ETL
21 288 (16.14%) Software Engineering
22 276 (15.47%) Analytical Skills
23 274 (15.36%) Linux
23 274 (15.36%) Apache
24 267 (14.97%) DevOps
25 251 (14.07%) Docker
26 247 (13.85%) Microsoft Azure
27 246 (13.79%) HBase
28 245 (13.73%) Artificial Intelligence
29 242 (13.57%) TDD

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 274 (15.36%) Apache
2 156 (8.74%) Elasticsearch
3 96 (5.38%) Apache Solr
4 76 (4.26%) SAS
5 74 (4.15%) Apache Pig
6 30 (1.68%) CMS
7 24 (1.35%) Apache Airflow
8 22 (1.23%) Tomcat
9 18 (1.01%) OpenStack
10 16 (0.90%) nginx
11 15 (0.84%) Confluence
12 13 (0.73%) Luigi
13 9 (0.50%) JBoss
13 9 (0.50%) Jupyter
14 8 (0.45%) Jetty
15 6 (0.34%) Oracle SOA Suite
16 5 (0.28%) GemFire
16 5 (0.28%) NServiceBus
16 5 (0.28%) Oracle Coherence
17 4 (0.22%) SharePoint Server
Applications
1 34 (1.91%) Microsoft Excel
2 15 (0.84%) SPSS
3 6 (0.34%) Chatbot
3 6 (0.34%) GNU Octave
3 6 (0.34%) MS Visio
4 4 (0.22%) Mathematica
4 4 (0.22%) Microsoft PowerPoint
5 1 (0.056%) CPLEX
5 1 (0.056%) Mingle
5 1 (0.056%) SPSS Modeler
5 1 (0.056%) Weka
Business Applications
1 3 (0.17%) Distributed Ledger
2 1 (0.056%) Actimize
2 1 (0.056%) Oracle AML
2 1 (0.056%) WebTrends
Cloud Services
1 543 (30.44%) Amazon AWS
2 247 (13.85%) Microsoft Azure
3 112 (6.28%) Google Cloud Platform
4 92 (5.16%) Amazon EMR
5 73 (4.09%) Amazon S3
6 52 (2.91%) Serverless
7 51 (2.86%) Amazon Kinesis
8 49 (2.75%) AWS Lambda
9 44 (2.47%) GitHub
10 31 (1.74%) Amazon EC2
10 31 (1.74%) Azure Machine Learning
11 28 (1.57%) SaaS
12 24 (1.35%) IBM Cloud
12 24 (1.35%) PaaS
13 23 (1.29%) Cloud Computing
14 21 (1.18%) IaaS
15 16 (0.90%) OpenShift
16 14 (0.78%) Datadog
17 13 (0.73%) Google Analytics
18 9 (0.50%) Slack
Communications & Networking
1 21 (1.18%) HTTP
1 21 (1.18%) Kerberos
1 21 (1.18%) Sourcefire
2 19 (1.07%) DNS
3 18 (1.01%) Internet
4 14 (0.78%) Firewall
5 11 (0.62%) TCP/IP
5 11 (0.62%) VPN
6 10 (0.56%) Network Security
7 9 (0.50%) SMTP
8 5 (0.28%) DAB
9 2 (0.11%) GPRS
9 2 (0.11%) GSM
9 2 (0.11%) ICMP
9 2 (0.11%) LTE
9 2 (0.11%) Radio Access Network
9 2 (0.11%) UMTS
9 2 (0.11%) Wireless
10 1 (0.056%) LAN
10 1 (0.056%) Software-Defined Networking
Database & Business Intelligence
1 1,252 (70.18%) Hadoop
2 1,220 (68.39%) Big Data
3 461 (25.84%) NoSQL
4 453 (25.39%) Apache Hive
5 355 (19.90%) Apache Cassandra
6 246 (13.79%) HBase
7 225 (12.61%) MongoDB
8 182 (10.20%) Data Lake
9 180 (10.09%) Data Warehouse
10 167 (9.36%) Relational Database
11 150 (8.41%) Impala
12 131 (7.34%) Amazon Redshift
13 116 (6.50%) MySQL
14 105 (5.89%) SQL Server
15 98 (5.49%) Data Mining
16 93 (5.21%) PostgreSQL
17 83 (4.65%) Flink
18 82 (4.60%) RDBMS
19 73 (4.09%) Redis
20 66 (3.70%) Power BI
Development Applications
1 132 (7.40%) Jenkins
2 131 (7.34%) Git (software)
3 68 (3.81%) Maven
4 53 (2.97%) JIRA
5 42 (2.35%) Gradle
6 38 (2.13%) Atlassian Bamboo
7 37 (2.07%) JUnit
8 29 (1.63%) Vagrant
9 28 (1.57%) TeamCity
10 25 (1.40%) RStudio
11 23 (1.29%) Cucumber
12 21 (1.18%) Selenium
13 17 (0.95%) Sonatype Nexus
14 15 (0.84%) Artifactory
14 15 (0.84%) GoCD
14 15 (0.84%) Subversion
15 12 (0.67%) GitLab
15 12 (0.67%) JMeter
16 11 (0.62%) Mockito
17 10 (0.56%) ERwin
General
1 377 (21.13%) Finance
2 176 (9.87%) Banking
3 106 (5.94%) Investment Banking
3 106 (5.94%) Retail
4 90 (5.04%) Marketing
5 88 (4.93%) Telecoms
6 78 (4.37%) Front Office
7 64 (3.59%) Financial Institution
8 60 (3.36%) Games
9 38 (2.13%) Advertising
10 28 (1.57%) Pharmaceutical
11 26 (1.46%) Publishing
12 24 (1.35%) Manufacturing
13 23 (1.29%) Law
14 15 (0.84%) Electronics
15 12 (0.67%) Legal
16 7 (0.39%) Military
17 4 (0.22%) Automotive
17 4 (0.22%) Retail Banking
18 3 (0.17%) Music Industry
Job Titles
1 418 (23.43%) Data Engineer
2 387 (21.69%) Developer
3 311 (17.43%) Data Scientist
4 229 (12.84%) Big Data Engineer
5 188 (10.54%) Architect
6 175 (9.81%) Java Developer
7 140 (7.85%) Data Architect
8 132 (7.40%) Big Data Developer
9 122 (6.84%) Senior Data Scientist
10 104 (5.83%) Scala Developer
11 98 (5.49%) Big Data Architect
12 87 (4.88%) Software Engineer
13 82 (4.60%) Java Engineer
14 64 (3.59%) Python Engineer
15 56 (3.14%) Machine Learning Engineer
16 55 (3.08%) DevOps Engineer
17 54 (3.03%) Hadoop Engineer
18 53 (2.97%) Senior Developer
19 49 (2.75%) Senior Software Engineer
20 42 (2.35%) Senior Data Engineer
Libraries, Frameworks & Software Standards
1 626 (35.09%) Kafka
2 148 (8.30%) Natural Language Processing
3 129 (7.23%) Web Services
4 111 (6.22%) Spark Streaming
5 106 (5.94%) Spring
6 90 (5.04%) Pandas
7 85 (4.76%) .NET
8 83 (4.65%) scikit-learn
9 82 (4.60%) REST
10 80 (4.48%) Akka
11 78 (4.37%) Node.js
12 75 (4.20%) AngularJS
12 75 (4.20%) Elastic Stack
13 71 (3.98%) NumPy
13 71 (3.98%) RabbitMQ
14 65 (3.64%) TensorFlow
15 62 (3.48%) HTML
16 61 (3.42%) SciPy
17 53 (2.97%) RESTful
18 52 (2.91%) React
Miscellaneous
1 313 (17.54%) Algorithms
2 276 (15.47%) Analytical Skills
3 242 (13.57%) Computer Science
4 128 (7.17%) Fintech
5 94 (5.27%) Data Structures
6 64 (3.59%) Greenfield Project
7 63 (3.53%) Cyberthreat
8 43 (2.41%) Self-Motivation
9 40 (2.24%) Management Information System
10 38 (2.13%) User Experience
11 37 (2.07%) Hedge funds
11 37 (2.07%) Linux Command Line
12 30 (1.68%) Derivative
13 27 (1.51%) Equities
14 26 (1.46%) Fixed Income
15 25 (1.40%) Foreign Exchange (FX)
15 25 (1.40%) Market Risk
16 21 (1.18%) Virtual Team
17 18 (1.01%) Life Science
18 17 (0.95%) Enterprise Software
Operating Systems
1 274 (15.36%) Linux
2 149 (8.35%) Unix
3 26 (1.46%) Windows
4 16 (0.90%) Ubuntu
5 7 (0.39%) CoreOS
6 6 (0.34%) CentOS
7 5 (0.28%) Amazon Linux
8 3 (0.17%) Android
8 3 (0.17%) Debian
9 2 (0.11%) Apple iOS
9 2 (0.11%) Red Hat Enterprise Linux
9 2 (0.11%) Windows Server
10 1 (0.056%) AIX
10 1 (0.056%) DC/OS
10 1 (0.056%) Solaris
10 1 (0.056%) Windows Server 2008
Processes & Methodologies
1 746 (41.82%) Analytics
2 663 (37.16%) Machine Learning
3 640 (35.87%) Data Science
4 603 (33.80%) Agile Software Development
5 302 (16.93%) ETL
6 288 (16.14%) Software Engineering
7 267 (14.97%) DevOps
8 245 (13.73%) Artificial Intelligence
9 242 (13.57%) TDD
10 240 (13.45%) Mathematics
11 233 (13.06%) Problem-Solving
12 206 (11.55%) Visualisation
13 205 (11.49%) Data Engineering
14 201 (11.27%) Data Analytics
15 180 (10.09%) Data Modelling
16 175 (9.81%) Business Intelligence
17 171 (9.59%) Open Source
18 167 (9.36%) Statistics
19 154 (8.63%) Scrum
20 151 (8.46%) Continuous Integration
Programming Languages
1 1,104 (61.88%) Python
2 880 (49.33%) Java
3 828 (46.41%) Scala
4 655 (36.72%) SQL
5 362 (20.29%) R
6 163 (9.14%) Java 8
7 134 (7.51%) C#
7 134 (7.51%) JavaScript
8 125 (7.01%) C++
9 101 (5.66%) Bash Shell
10 90 (5.04%) Perl
11 81 (4.54%) MATLAB
12 69 (3.87%) Ruby
13 67 (3.76%) Shell Script
14 55 (3.08%) Go
15 47 (2.63%) C
16 33 (1.85%) T-SQL
17 21 (1.18%) PHP
18 16 (0.90%) PL/SQL
19 15 (0.84%) VBA
Qualifications
1 304 (17.04%) Degree
2 118 (6.61%) PhD
3 100 (5.61%) Computer Science Degree
4 51 (2.86%) Master's Degree
5 45 (2.52%) Security Cleared
6 36 (2.02%) Numerate Degree
7 21 (1.18%) CEH
7 21 (1.18%) CISSP
7 21 (1.18%) GCIH
7 21 (1.18%) OSCP
8 17 (0.95%) DV Cleared
9 9 (0.50%) ISTQB
9 9 (0.50%) ISTQB Foundation Certification
10 6 (0.34%) SC Cleared
11 4 (0.22%) MBA
12 3 (0.17%) Microsoft Certification
13 2 (0.11%) EMC Certification
13 2 (0.11%) MCSA
13 2 (0.11%) MCSD
13 2 (0.11%) TOGAF Certification
Quality Assurance & Compliance
1 56 (3.14%) Data Quality
2 49 (2.75%) QA
3 13 (0.73%) GCP
4 12 (0.67%) GPG13
4 12 (0.67%) ISO/IEC 27001
5 11 (0.62%) GDPR
6 7 (0.39%) Solvency II
7 6 (0.34%) MiFID
8 3 (0.17%) EMIR
9 2 (0.11%) PMO
9 2 (0.11%) SLA
10 1 (0.056%) Dodd-Frank
10 1 (0.056%) FATCA
10 1 (0.056%) HIPAA
10 1 (0.056%) Sarbanes-Oxley
System Software
1 251 (14.07%) Docker
2 189 (10.59%) HDFS
3 120 (6.73%) Apache Flume
4 26 (1.46%) Apache ZooKeeper
5 25 (1.40%) VirtualBox
6 21 (1.18%) Snort
7 18 (1.01%) Active Directory
8 17 (0.95%) VMware Infrastructure
9 14 (0.78%) Virtual Machines
10 7 (0.39%) Hyper-V
11 5 (0.28%) Xen
12 3 (0.17%) KVM
12 3 (0.17%) Oracle RAC
13 2 (0.11%) LXC
14 1 (0.056%) Cronacle
Systems Management
1 139 (7.79%) Kubernetes
2 123 (6.89%) Ansible
3 97 (5.44%) Hadoop YARN
4 86 (4.82%) Terraform
5 79 (4.43%) Oozie
6 74 (4.15%) Mesos
6 74 (4.15%) Puppet
7 53 (2.97%) Opscode Chef
8 24 (1.35%) Docker Swarm
9 21 (1.18%) Suricata
10 20 (1.12%) Salt
11 16 (0.90%) Consul
12 14 (0.78%) Kibana
12 14 (0.78%) Prometheus
13 12 (0.67%) Ambari
14 10 (0.56%) Nagios
15 9 (0.50%) ITRS Geneos
16 8 (0.45%) QRadar
17 7 (0.39%) ZABBIX
18 5 (0.28%) logstash
Vendors
1 233 (13.06%) Cloudera
2 189 (10.59%) Google
3 178 (9.98%) Tableau
4 144 (8.07%) Oracle
5 117 (6.56%) Hortonworks
6 109 (6.11%) Microsoft
7 108 (6.05%) Talend
8 97 (5.44%) Informatica
9 66 (3.70%) IBM
10 43 (2.41%) Teradata
11 40 (2.24%) MapR
11 40 (2.24%) SAP
12 35 (1.96%) Splunk
13 34 (1.91%) Alteryx
14 31 (1.74%) WorldPay
15 30 (1.68%) Sybase
16 28 (1.57%) TIBCO
17 27 (1.51%) EXASOL
18 26 (1.46%) VMware
19 23 (1.29%) Atlassian