Period
to 21 February 2019

The following table provides summary statistics for permanent job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 21 February 2019 with a comparison to the same period in the previous 2 years.

Apache Spark
England > London
6 months to
21 Feb 2019
Same period 2018 Same period 2017
Rank 156 142 201
Rank change year-on-year -14 +59 +126
Permanent jobs citing Apache Spark 1,404 1,758 1,235
As % of all permanent IT jobs advertised in London 2.24% 2.50% 1.76%
As % of the Application Platforms category 13.20% 13.85% 9.66%
Number of salaries quoted 1,131 1,472 1,056
London median annual salary £80,000 £75,000 £72,500
Median salary % change year-on-year +6.67% +3.45% +7.41%
10th Percentile £53,750 £50,000 £48,750
90th Percentile £107,500 £110,000 £105,000
England median annual salary £75,000 £70,000 £67,500
% change year-on-year +7.14% +3.70% +3.85%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Permanent vacancies with a requirement for application platform skills 10,638 12,694 12,790
As % of all permanent IT jobs advertised in London 16.97% 18.07% 18.26%
Number of salaries quoted 8,577 10,624 10,872
London median annual salary £60,000 £60,000 £55,000
Median salary % change year-on-year - +9.09% +4.76%
10th Percentile £32,500 £33,750 £32,000
90th Percentile £95,000 £88,750 £85,000
England median annual salary £50,000 £50,000 £46,500
% change year-on-year - +7.53% +3.33%

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a percentage of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Salary Trend in London

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in London.

Salary trend for Apache Spark in London

Apache Spark
Salary Histogram in London

The salary distribution of IT jobs citing Apache Spark in London over the 6 months to 21 February 2019.

Salary histogram for Apache Spark in London

Apache Spark
Job Locations in London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the London region over the 6 months to 21 February 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London -14 230 £77,500 +3.33% 80
West London -6 26 £65,000 -3.70% 18
East London +10 11 £95,000 +2.70% 18
North London +25 7 £77,500 - 2
South London -15 3 £80,000 -3.03%
Apache Spark
England

For the 6 months to 21 February 2019, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the London region with a requirement for Apache Spark.

1 1,002 (71.37%) Hadoop
2 978 (69.66%) Big Data
3 973 (69.30%) Python
4 778 (55.41%) Java
5 667 (47.51%) Scala
6 636 (45.30%) Amazon AWS
7 608 (43.30%) Kafka
8 588 (41.88%) Agile Software Development
9 553 (39.39%) Machine Learning
10 512 (36.47%) Analytics
11 491 (34.97%) Data Science
12 489 (34.83%) NoSQL
13 483 (34.40%) SQL
14 475 (33.83%) Apache Hive
15 358 (25.50%) Apache Cassandra
16 328 (23.36%) HBase
16 328 (23.36%) Software Engineering
17 324 (23.08%) Microsoft Azure
18 307 (21.87%) DevOps
19 296 (21.08%) Finance
20 285 (20.30%) Data Modelling
21 278 (19.80%) ETL
21 278 (19.80%) Artificial Intelligence
22 266 (18.95%) Google Cloud Platform
23 254 (18.09%) Cloudera
24 249 (17.74%) Apache
25 235 (16.74%) Linux
26 234 (16.67%) Docker
27 225 (16.03%) Data Analytics
28 222 (15.81%) Amazon Redshift

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 249 (17.74%) Apache
2 215 (15.31%) Elasticsearch
3 190 (13.53%) Apache Pig
4 86 (6.13%) Apache Solr
5 45 (3.21%) SAS
6 23 (1.64%) Apache Airflow
7 16 (1.14%) GlassFish
7 16 (1.14%) Tomcat
8 10 (0.71%) Luigi
9 7 (0.50%) Confluence
10 6 (0.43%) Jupyter
11 4 (0.28%) nginx
11 4 (0.28%) WebSphere
12 1 (0.071%) Hyperledger
12 1 (0.071%) Mule
12 1 (0.071%) OpenStack
12 1 (0.071%) WebSphere Application Server
Applications
1 12 (0.85%) SPSS
2 6 (0.43%) Stata
3 5 (0.36%) Microsoft Excel
4 4 (0.28%) Chatbot
4 4 (0.28%) GNU Octave
5 2 (0.14%) Microsoft Office
5 2 (0.14%) Microsoft PowerPoint
5 2 (0.14%) MS Visio
Business Applications
1 1 (0.071%) Adobe Campaign
1 1 (0.071%) PolarLake
Cloud Services
1 636 (45.30%) Amazon AWS
2 324 (23.08%) Microsoft Azure
3 266 (18.95%) Google Cloud Platform
4 79 (5.63%) Serverless
5 71 (5.06%) Amazon EMR
6 69 (4.91%) Cloud Computing
7 67 (4.77%) AWS Lambda
8 58 (4.13%) Amazon S3
9 55 (3.92%) GitHub
10 38 (2.71%) Amazon EC2
11 37 (2.64%) PaaS
12 33 (2.35%) Amazon Kinesis
13 30 (2.14%) SaaS
14 29 (2.07%) Amazon SQS
15 23 (1.64%) AWS CloudFormation
16 22 (1.57%) Amazon SageMaker
17 21 (1.50%) Cortana Intelligence Suite
18 20 (1.42%) Azure Data Factory
19 18 (1.28%) Azure Machine Learning
20 14 (1.00%) AWS Data Pipeline
Communications & Networking
1 27 (1.92%) HTTP
1 27 (1.92%) Internet
2 11 (0.78%) AMQP
3 6 (0.43%) DNS
3 6 (0.43%) MQTT
3 6 (0.43%) VPN
4 5 (0.36%) Kerberos
5 4 (0.28%) Firewall
5 4 (0.28%) LAN
5 4 (0.28%) Software-Defined Networking
5 4 (0.28%) WAN
6 2 (0.14%) FTP
6 2 (0.14%) Network Security
6 2 (0.14%) Sourcefire
6 2 (0.14%) TCP/IP
7 1 (0.071%) HTTPS
7 1 (0.071%) NX-OS
7 1 (0.071%) SSL
7 1 (0.071%) VLAN
7 1 (0.071%) Wi-Fi
Database & Business Intelligence
1 1,002 (71.37%) Hadoop
2 978 (69.66%) Big Data
3 489 (34.83%) NoSQL
4 475 (33.83%) Apache Hive
5 358 (25.50%) Apache Cassandra
6 328 (23.36%) HBase
7 222 (15.81%) Amazon Redshift
8 206 (14.67%) Impala
9 173 (12.32%) MongoDB
10 154 (10.97%) Data Warehouse
11 130 (9.26%) PostgreSQL
12 125 (8.90%) Flink
13 90 (6.41%) MySQL
14 86 (6.13%) Data Lake
15 84 (5.98%) Relational Database
16 83 (5.91%) Sqoop
17 82 (5.84%) SQL Server
18 76 (5.41%) Couchbase
19 74 (5.27%) QlikView
20 72 (5.13%) CouchDB
Development Applications
1 124 (8.83%) Git (software)
2 117 (8.33%) Jenkins
3 36 (2.56%) Visual Studio
4 33 (2.35%) Maven
5 29 (2.07%) Gradle
6 26 (1.85%) JIRA
7 21 (1.50%) Ant
8 18 (1.28%) Visual Studio Team System
9 17 (1.21%) Atlassian Bamboo
10 15 (1.07%) TeamCity
11 13 (0.93%) CircleCI
11 13 (0.93%) Selenium
12 11 (0.78%) GoCD
12 11 (0.78%) JUnit
12 11 (0.78%) Vagrant
13 10 (0.71%) GitLab
14 9 (0.64%) Travis CI
15 8 (0.57%) Artifactory
15 8 (0.57%) Octopus Deploy
16 6 (0.43%) Sonatype Nexus
General
1 296 (21.08%) Finance
2 173 (12.32%) Law
3 154 (10.97%) Military
4 147 (10.47%) Banking
5 79 (5.63%) Retail
6 59 (4.20%) Telecoms
7 58 (4.13%) Investment Banking
8 44 (3.13%) Games
9 39 (2.78%) Marketing
10 21 (1.50%) Advertising
11 18 (1.28%) Automotive
12 14 (1.00%) Retail Banking
13 12 (0.85%) Financial Institution
13 12 (0.85%) Front Office
14 10 (0.71%) Manufacturing
14 10 (0.71%) Publishing
15 8 (0.57%) Pharmaceutical
16 6 (0.43%) Billing
16 6 (0.43%) Electronics
17 5 (0.36%) Legal
Job Titles
1 510 (36.32%) Data Engineer
2 278 (19.80%) Big Data Engineer
3 270 (19.23%) Developer
4 173 (12.32%) Architect
5 157 (11.18%) Data Scientist
6 151 (10.75%) Data Architect
7 130 (9.26%) Java Developer
8 100 (7.12%) Python Engineer
8 100 (7.12%) Senior Data Engineer
9 96 (6.84%) Hadoop Engineer
10 86 (6.13%) Big Data Architect
11 64 (4.56%) Scala Developer
12 58 (4.13%) Consultant
13 55 (3.92%) Java Engineer
14 53 (3.77%) Big Data Developer
15 50 (3.56%) DevOps Engineer
16 49 (3.49%) Software Engineer
17 43 (3.06%) Python Developer
17 43 (3.06%) Senior Data Scientist
18 41 (2.92%) Senior Developer
Libraries, Frameworks & Software Standards
1 608 (43.30%) Kafka
2 108 (7.69%) Web Services
3 91 (6.48%) Natural Language Processing
4 81 (5.77%) Apache NiFi
5 80 (5.70%) Spark Streaming
6 76 (5.41%) Pandas
7 71 (5.06%) RESTful
7 71 (5.06%) scikit-learn
8 65 (4.63%) NumPy
9 63 (4.49%) PySpark
10 62 (4.42%) REST
11 60 (4.27%) TensorFlow
12 59 (4.20%) Node.js
13 58 (4.13%) AngularJS
14 55 (3.92%) Akka
15 52 (3.70%) .NET
16 49 (3.49%) HTML
17 44 (3.13%) React
17 44 (3.13%) Spring
18 37 (2.64%) SciPy
Miscellaneous
1 166 (11.82%) Algorithms
2 138 (9.83%) Analytical Skills
3 124 (8.83%) Data Structures
4 88 (6.27%) Fintech
5 54 (3.85%) Greenfield Project
6 44 (3.13%) Self-Motivation
7 26 (1.85%) Virtual Team
8 24 (1.71%) Derivative
8 24 (1.71%) User Experience
9 22 (1.57%) Life Science
10 19 (1.35%) Management Information System
11 18 (1.28%) Market Risk
11 18 (1.28%) Social Media
12 15 (1.07%) Blog
13 13 (0.93%) Public Cloud
14 12 (0.85%) Enterprise Software
15 11 (0.78%) Linux Command Line
16 10 (0.71%) Spread Betting
17 9 (0.64%) Mobile App
18 8 (0.57%) Cloud Native
Operating Systems
1 235 (16.74%) Linux
2 92 (6.55%) Unix
3 33 (2.35%) Windows
4 16 (1.14%) Android
4 16 (1.14%) Apple iOS
5 10 (0.71%) Ubuntu
6 7 (0.50%) CentOS
7 5 (0.36%) Windows Server
8 4 (0.28%) Oracle Linux
9 3 (0.21%) Mac OS X
10 2 (0.14%) DC/OS
11 1 (0.071%) CoreOS
11 1 (0.071%) Debian
11 1 (0.071%) Mac OS
11 1 (0.071%) Red Hat Enterprise Linux
11 1 (0.071%) Windows Server 2012
Processes & Methodologies
1 588 (41.88%) Agile Software Development
2 553 (39.39%) Machine Learning
3 512 (36.47%) Analytics
4 491 (34.97%) Data Science
5 328 (23.36%) Software Engineering
6 307 (21.87%) DevOps
7 285 (20.30%) Data Modelling
8 278 (19.80%) Artificial Intelligence
8 278 (19.80%) ETL
9 225 (16.03%) Data Analytics
10 192 (13.68%) Visualisation
11 183 (13.03%) Data Engineering
12 177 (12.61%) Business Intelligence
13 158 (11.25%) MapReduce
14 152 (10.83%) Computer Science
14 152 (10.83%) Open Source
15 150 (10.68%) Scrum
16 141 (10.04%) Microservices
17 127 (9.05%) Data Analysis
18 118 (8.40%) Continuous Integration
Programming Languages
1 973 (69.30%) Python
2 778 (55.41%) Java
3 667 (47.51%) Scala
4 483 (34.40%) SQL
5 158 (11.25%) R
6 124 (8.83%) C++
7 112 (7.98%) C#
8 92 (6.55%) JavaScript
9 72 (5.13%) Java 8
10 50 (3.56%) Shell Script
11 48 (3.42%) Bash Shell
12 47 (3.35%) C
13 35 (2.49%) Perl
13 35 (2.49%) PHP
14 28 (1.99%) Ruby
15 27 (1.92%) Groovy
16 24 (1.71%) T-SQL
17 22 (1.57%) PL/SQL
18 21 (1.50%) Go
19 17 (1.21%) Kotlin
Qualifications
1 160 (11.40%) Degree
2 58 (4.13%) Computer Science Degree
3 51 (3.63%) PhD
4 22 (1.57%) Numerate Degree
4 22 (1.57%) Security Cleared
5 17 (1.21%) Master's Degree
6 14 (1.00%) SC Cleared
7 11 (0.78%) DV Cleared
8 5 (0.36%) TOGAF Certification
9 4 (0.28%) CEH
9 4 (0.28%) MCAD
9 4 (0.28%) MCDBA
9 4 (0.28%) MCSD
9 4 (0.28%) Microsoft Certification
10 3 (0.21%) ISTQB
11 2 (0.14%) CISSP
11 2 (0.14%) GCIH
11 2 (0.14%) OSCP
11 2 (0.14%) PMI Certification
11 2 (0.14%) Postgraduate
Quality Assurance & Compliance
1 80 (5.70%) Data Quality
2 30 (2.14%) QA
3 21 (1.50%) GDPR
4 11 (0.78%) MiFID
5 5 (0.36%) PCI DSS
6 4 (0.28%) ISO/IEC 27001
7 2 (0.14%) GPG13
8 1 (0.071%) COBIT
8 1 (0.071%) EMIR
8 1 (0.071%) Solvency II
System Software
1 234 (16.67%) Docker
2 125 (8.90%) Apache Flume
3 104 (7.41%) HDFS
4 61 (4.34%) Apache ZooKeeper
5 13 (0.93%) Active Directory
6 9 (0.64%) VirtualBox
7 6 (0.43%) Virtual Machines
8 4 (0.28%) VMware Infrastructure
9 2 (0.14%) Hyper-V
9 2 (0.14%) Snort
10 1 (0.071%) Firmware
10 1 (0.071%) Platform Symphony
10 1 (0.071%) VMware ESXi
Systems Management
1 179 (12.75%) Oozie
2 131 (9.33%) Kubernetes
3 122 (8.69%) Hadoop YARN
4 118 (8.40%) Ansible
5 92 (6.55%) Terraform
6 78 (5.56%) Opscode Chef
7 57 (4.06%) Puppet
8 47 (3.35%) Mesos
9 44 (3.13%) Kibana
10 34 (2.42%) Prometheus
11 30 (2.14%) Ambari
12 26 (1.85%) logstash
13 22 (1.57%) Nagios
14 20 (1.42%) Salt
15 16 (1.14%) Grafana
16 12 (0.85%) Docker Swarm
17 7 (0.50%) Marathon
18 4 (0.28%) Control-M
18 4 (0.28%) Microsoft Intune
18 4 (0.28%) SCCM
Vendors
1 254 (18.09%) Cloudera
2 184 (13.11%) Hortonworks
3 165 (11.75%) Google
4 159 (11.32%) Tableau
5 118 (8.40%) Databricks
6 116 (8.26%) Microsoft
7 93 (6.62%) MapR
8 91 (6.48%) Oracle
9 75 (5.34%) Talend
10 69 (4.91%) IBM
11 61 (4.34%) Informatica
12 32 (2.28%) SAP
12 32 (2.28%) Splunk
13 31 (2.21%) TIBCO
14 21 (1.50%) Greenplum
15 18 (1.28%) HP
15 18 (1.28%) Vertica
16 16 (1.14%) WorldPay
17 14 (1.00%) SaltStack
17 14 (1.00%) Teradata