Period
to 24 February 2018

The following table provides summary statistics for permanent job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 24 February 2018 with a comparison to the same period in the previous 2 years.

Apache Spark
England > London
6 months to
24 Feb 2018
Same period 2017 Same period 2016
Rank 135 193 312
Rank change year-on-year +58 +119 +423
Permanent jobs citing Apache Spark 1,774 1,241 928
As % of all permanent IT jobs advertised in London 2.55% 1.78% 1.08%
As % of the Application Platforms category 14.17% 9.72% 5.68%
Number of salaries quoted 1,484 1,065 787
London median annual salary £75,000 £72,500 £67,500
Median salary % change year-on-year +3.45% +7.41% -6.90%
10th Percentile £50,000 £48,750 £45,000
90th Percentile £110,000 £105,000 £98,500
England median annual salary £70,000 £67,500 £65,000
% change year-on-year +3.70% +3.85% -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Permanent vacancies with a requirement for application platform skills 12,523 12,765 16,331
As % of all permanent IT jobs advertised in London 17.99% 18.27% 19.01%
Number of salaries quoted 10,500 10,845 14,045
London median annual salary £60,000 £55,000 £52,500
Median salary % change year-on-year +9.09% +4.76% +5.00%
10th Percentile £33,750 £32,000 £30,000
90th Percentile £90,000 £85,000 £80,000
England median annual salary £50,000 £46,500 £45,000
% change year-on-year +7.53% +3.33% +5.88%

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a percentage of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Salary Trend in London

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in London.

Salary trend for Apache Spark in London

Apache Spark
Salary Histogram in London

The salary distribution of IT jobs citing Apache Spark in London over the 6 months to 24 February 2018.

Salary histogram for Apache Spark in London

Apache Spark
Job Locations in London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the London region over the 6 months to 24 February 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
Central London +44 310 £75,000 +3.45% 68
West London +44 46 £67,500 - 30
East London +1 24 £92,500 +37.04% 8
South London +1 10 £82,500 +29.81% 2
North London -2 1 - - 4
Apache Spark
England

For the 6 months to 24 February 2018, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the London region with a requirement for Apache Spark.

1 1,218 (68.66%) Hadoop
2 1,159 (65.33%) Big Data
3 1,069 (60.26%) Python
4 843 (47.52%) Java
5 789 (44.48%) Scala
6 710 (40.02%) Analytics
7 680 (38.33%) Machine Learning
8 659 (37.15%) Data Science
9 649 (36.58%) Agile Software Development
10 584 (32.92%) SQL
11 554 (31.23%) Kafka
12 528 (29.76%) Amazon AWS
13 445 (25.08%) Finance
14 415 (23.39%) Apache Hive
15 367 (20.69%) NoSQL
16 352 (19.84%) Apache Cassandra
17 331 (18.66%) Algorithms
18 311 (17.53%) Docker
19 310 (17.47%) Software Engineering
20 309 (17.42%) R
21 308 (17.36%) DevOps
22 285 (16.07%) Degree
23 284 (16.01%) Apache
24 282 (15.90%) Linux
25 266 (14.99%) MongoDB
26 261 (14.71%) Computer Science
27 260 (14.66%) Analytical Skills
28 257 (14.49%) HBase
29 239 (13.47%) ETL
30 223 (12.57%) TDD

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 284 (16.01%) Apache
2 164 (9.24%) Elasticsearch
3 86 (4.85%) SAS
4 82 (4.62%) Apache Solr
5 75 (4.23%) Apache Pig
6 38 (2.14%) Apache Airflow
7 36 (2.03%) CMS
8 29 (1.63%) OpenStack
9 21 (1.18%) Confluence
10 18 (1.01%) Tomcat
11 15 (0.85%) nginx
12 14 (0.79%) Luigi
13 10 (0.56%) Jetty
14 7 (0.39%) IPython
15 6 (0.34%) JBoss
15 6 (0.34%) Oracle Coherence
16 5 (0.28%) WebSphere
17 4 (0.23%) Jupyter
17 4 (0.23%) Oracle SOA Suite
17 4 (0.23%) WebLogic
Applications
1 33 (1.86%) Microsoft Excel
2 21 (1.18%) SPSS
3 6 (0.34%) Chatbot
4 5 (0.28%) GNU Octave
5 3 (0.17%) Microsoft PowerPoint
6 2 (0.11%) HipChat
6 2 (0.11%) Mathematica
7 1 (0.056%) MS Visio
7 1 (0.056%) Weka
Business Applications
1 1 (0.056%) Capstone
1 1 (0.056%) Distributed Ledger
1 1 (0.056%) Dynamics CRM
1 1 (0.056%) Summit
1 1 (0.056%) WebTrends
Cloud Services
1 528 (29.76%) Amazon AWS
2 188 (10.60%) Microsoft Azure
3 101 (5.69%) Google Cloud Platform
4 59 (3.33%) GitHub
5 53 (2.99%) Amazon S3
6 52 (2.93%) Amazon Kinesis
7 49 (2.76%) Amazon EMR
7 49 (2.76%) SaaS
8 30 (1.69%) Cloud Computing
9 29 (1.63%) AWS Lambda
10 28 (1.58%) Amazon EC2
10 28 (1.58%) Serverless
11 25 (1.41%) Azure Machine Learning
12 20 (1.13%) IaaS
12 20 (1.13%) PaaS
13 17 (0.96%) Datadog
14 15 (0.85%) OpenShift
15 13 (0.73%) Virtual Private Cloud
16 12 (0.68%) AWS CloudFormation
17 8 (0.45%) CloudFront
Communications & Networking
1 22 (1.24%) DNS
1 22 (1.24%) Kerberos
2 20 (1.13%) HTTP
3 19 (1.07%) Internet
4 18 (1.01%) Firewall
5 17 (0.96%) TCP/IP
6 16 (0.90%) VPN
7 9 (0.51%) ICMP
8 8 (0.45%) DAB
9 7 (0.39%) SMTP
10 5 (0.28%) Network Security
11 3 (0.17%) SSL
12 2 (0.11%) GPRS
12 2 (0.11%) GSM
12 2 (0.11%) LTE
12 2 (0.11%) Radio Access Network
12 2 (0.11%) UMTS
12 2 (0.11%) Wireless
13 1 (0.056%) LAN
13 1 (0.056%) WAN
Database & Business Intelligence
1 1,218 (68.66%) Hadoop
2 1,159 (65.33%) Big Data
3 415 (23.39%) Apache Hive
4 367 (20.69%) NoSQL
5 352 (19.84%) Apache Cassandra
6 266 (14.99%) MongoDB
7 257 (14.49%) HBase
8 198 (11.16%) Data Warehouse
9 182 (10.26%) Relational Database
10 155 (8.74%) SQL Server
11 136 (7.67%) Impala
12 133 (7.50%) PostgreSQL
13 120 (6.76%) Data Lake
14 112 (6.31%) MySQL
15 97 (5.47%) Amazon Redshift
16 91 (5.13%) Data Mining
17 87 (4.90%) RDBMS
18 82 (4.62%) Flink
19 74 (4.17%) Redis
20 63 (3.55%) Power BI
Development Applications
1 156 (8.79%) Jenkins
2 140 (7.89%) Git (software)
3 80 (4.51%) JIRA
4 58 (3.27%) Maven
5 41 (2.31%) Atlassian Bamboo
6 35 (1.97%) Selenium
7 27 (1.52%) Gradle
7 27 (1.52%) JUnit
8 25 (1.41%) Team Foundation Server
9 23 (1.30%) Bitbucket
9 23 (1.30%) TeamCity
10 22 (1.24%) Sonatype Nexus
11 20 (1.13%) Cucumber
12 19 (1.07%) Artifactory
12 19 (1.07%) Vagrant
13 12 (0.68%) Travis CI
14 10 (0.56%) GitLab
14 10 (0.56%) IntelliJ
15 9 (0.51%) GoCD
15 9 (0.51%) VSS/SourceSafe
General
1 445 (25.08%) Finance
2 119 (6.71%) Marketing
3 117 (6.60%) Banking
4 110 (6.20%) Retail
5 98 (5.52%) Telecoms
6 57 (3.21%) Investment Banking
6 57 (3.21%) Legal
7 46 (2.59%) Front Office
8 37 (2.09%) Advertising
8 37 (2.09%) Financial Institution
9 29 (1.63%) Manufacturing
10 28 (1.58%) Games
11 23 (1.30%) Pharmaceutical
12 20 (1.13%) Electronics
13 15 (0.85%) Publishing
14 14 (0.79%) Law
15 12 (0.68%) Retail Banking
16 9 (0.51%) Automotive
17 6 (0.34%) Aerospace
18 2 (0.11%) Billing
Job Titles
1 392 (22.10%) Developer
2 369 (20.80%) Data Engineer
3 269 (15.16%) Data Scientist
4 223 (12.57%) Big Data Engineer
5 173 (9.75%) Architect
6 152 (8.57%) Java Developer
7 117 (6.60%) Data Architect
8 116 (6.54%) Software Engineer
9 107 (6.03%) Big Data Developer
10 99 (5.58%) Scala Developer
11 95 (5.36%) Senior Data Scientist
12 91 (5.13%) Java Engineer
13 79 (4.45%) DevOps Engineer
14 77 (4.34%) Machine Learning Engineer
15 75 (4.23%) Big Data Architect
16 59 (3.33%) Senior Developer
16 59 (3.33%) Senior Software Engineer
17 53 (2.99%) Hadoop Engineer
17 53 (2.99%) Java Software Engineer
18 47 (2.65%) Java Scala Developer
Libraries, Frameworks & Software Standards
1 554 (31.23%) Kafka
2 184 (10.37%) Natural Language Processing
3 132 (7.44%) Web Services
4 112 (6.31%) Node.js
5 103 (5.81%) Spring
6 95 (5.36%) Akka
7 93 (5.24%) AngularJS
7 93 (5.24%) REST
8 91 (5.13%) Spark Streaming
9 85 (4.79%) HTML
10 75 (4.23%) NumPy
10 75 (4.23%) Pandas
11 70 (3.95%) React
12 68 (3.83%) Elastic Stack
13 67 (3.78%) RabbitMQ
14 62 (3.49%) scikit-learn
15 61 (3.44%) CSS
16 56 (3.16%) .NET
16 56 (3.16%) HTML5
17 55 (3.10%) SciPy
Miscellaneous
1 331 (18.66%) Algorithms
2 261 (14.71%) Computer Science
3 260 (14.66%) Analytical Skills
4 117 (6.60%) Fintech
5 79 (4.45%) Greenfield Project
6 68 (3.83%) Data Structures
7 58 (3.27%) Cyberthreat
8 43 (2.42%) Hedge funds
9 30 (1.69%) User Experience
10 29 (1.63%) Mobile App
11 27 (1.52%) Fixed Income
12 26 (1.47%) Data Centre
13 25 (1.41%) Derivative
13 25 (1.41%) Market Risk
14 23 (1.30%) Clustering
14 23 (1.30%) Enterprise Software
15 22 (1.24%) Equities
16 20 (1.13%) Linux Command Line
17 16 (0.90%) Management Information System
17 16 (0.90%) Public Cloud
Operating Systems
1 282 (15.90%) Linux
2 116 (6.54%) Unix
3 18 (1.01%) Windows
4 17 (0.96%) Ubuntu
5 6 (0.34%) Android
5 6 (0.34%) Red Hat Enterprise Linux
6 5 (0.28%) Apple iOS
6 5 (0.28%) CoreOS
7 4 (0.23%) CentOS
8 2 (0.11%) Amazon Linux
8 2 (0.11%) Mesosphere
8 2 (0.11%) Solaris
9 1 (0.056%) AIX
9 1 (0.056%) DC/OS
9 1 (0.056%) Debian
9 1 (0.056%) Mac OS X
9 1 (0.056%) Windows Server
Processes & Methodologies
1 710 (40.02%) Analytics
2 680 (38.33%) Machine Learning
3 659 (37.15%) Data Science
4 649 (36.58%) Agile Software Development
5 310 (17.47%) Software Engineering
6 308 (17.36%) DevOps
7 239 (13.47%) ETL
8 223 (12.57%) TDD
9 211 (11.89%) Business Intelligence
10 205 (11.56%) Data Analytics
11 198 (11.16%) Visualisation
12 194 (10.94%) Open Source
13 192 (10.82%) Mathematics
14 187 (10.54%) Continuous Integration
15 164 (9.24%) Data Modelling
16 161 (9.08%) Microservices
16 161 (9.08%) Statistics
17 156 (8.79%) Artificial Intelligence
18 153 (8.62%) Scrum
19 130 (7.33%) BDD
Programming Languages
1 1,069 (60.26%) Python
2 843 (47.52%) Java
3 789 (44.48%) Scala
4 584 (32.92%) SQL
5 309 (17.42%) R
6 175 (9.86%) JavaScript
7 156 (8.79%) C#
8 119 (6.71%) C++
9 113 (6.37%) Ruby
10 104 (5.86%) Bash Shell
11 83 (4.68%) Go
12 63 (3.55%) MATLAB
13 59 (3.33%) Shell Script
14 51 (2.87%) C
15 38 (2.14%) Perl
16 22 (1.24%) PowerShell
17 21 (1.18%) PHP
18 13 (0.73%) ES6
18 13 (0.73%) T-SQL
19 11 (0.62%) PL/SQL
Qualifications
1 285 (16.07%) Degree
2 117 (6.60%) PhD
3 101 (5.69%) Computer Science Degree
4 48 (2.71%) Master's Degree
5 46 (2.59%) Security Cleared
6 27 (1.52%) DV Cleared
7 15 (0.85%) Numerate Degree
8 13 (0.73%) CISSP
8 13 (0.73%) ISTQB
8 13 (0.73%) SC Cleared
9 12 (0.68%) CEH
9 12 (0.68%) GCIH
9 12 (0.68%) MBA
9 12 (0.68%) OSCP
10 11 (0.62%) ISEB
11 6 (0.34%) AWS Certification
11 6 (0.34%) ISTQB Foundation Certification
12 3 (0.17%) Microsoft Certification
13 2 (0.11%) EMC Certification
14 1 (0.056%) ACCA
Quality Assurance & Compliance
1 84 (4.74%) Data Quality
2 59 (3.33%) QA
3 13 (0.73%) ISO/IEC 27001
4 12 (0.68%) GPG13
5 11 (0.62%) Solvency II
6 10 (0.56%) GCP
6 10 (0.56%) GDPR
7 6 (0.34%) SLA
8 5 (0.28%) Sarbanes-Oxley
9 3 (0.17%) MiFID
10 2 (0.11%) EMIR
10 2 (0.11%) PMO
11 1 (0.056%) Dodd-Frank
11 1 (0.056%) FATCA
11 1 (0.056%) HIPAA
System Software
1 311 (17.53%) Docker
2 141 (7.95%) HDFS
3 95 (5.36%) Apache Flume
4 34 (1.92%) Apache ZooKeeper
5 17 (0.96%) Virtual Machines
6 12 (0.68%) Snort
7 11 (0.62%) VMware Infrastructure
8 5 (0.28%) Active Directory
8 5 (0.28%) VirtualBox
9 4 (0.23%) Xen
10 3 (0.17%) KVM
11 2 (0.11%) MapR-FS
12 1 (0.056%) GridGain
12 1 (0.056%) Hyper-V
Systems Management
1 158 (8.91%) Kubernetes
2 140 (7.89%) Ansible
3 102 (5.75%) Hadoop YARN
4 97 (5.47%) Terraform
5 91 (5.13%) Puppet
6 79 (4.45%) Mesos
7 62 (3.49%) Opscode Chef
8 61 (3.44%) Oozie
9 21 (1.18%) Kibana
10 18 (1.01%) Salt
11 14 (0.79%) Ambari
12 12 (0.68%) Suricata
13 10 (0.56%) logstash
14 9 (0.51%) Marathon
15 8 (0.45%) Consul
15 8 (0.45%) Docker Swarm
15 8 (0.45%) ITRS Geneos
15 8 (0.45%) Nagios
16 7 (0.39%) Prometheus
17 6 (0.34%) QRadar
Vendors
1 186 (10.48%) Cloudera
2 169 (9.53%) Google
3 155 (8.74%) Oracle
4 143 (8.06%) Tableau
5 129 (7.27%) Microsoft
6 105 (5.92%) Hortonworks
7 62 (3.49%) Talend
8 54 (3.04%) Teradata
9 51 (2.87%) Informatica
10 46 (2.59%) MapR
11 41 (2.31%) SAP
12 39 (2.20%) Alteryx
13 32 (1.80%) VMware
14 31 (1.75%) IBM
15 30 (1.69%) Splunk
16 26 (1.47%) TIBCO
17 21 (1.18%) RedGate
18 20 (1.13%) Netezza
19 19 (1.07%) Twitter
20 18 (1.01%) EXASOL