Period
to 21 July 2018

The following table provides summary statistics for permanent job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 21 July 2018 with a comparison to the same period in the previous 2 years.

Apache Spark
UK > England
6 months to
21 Jul 2018
Same period 2017 Same period 2016
Rank 228 289 385
Rank change year-on-year +61 +96 +424
Permanent jobs citing Apache Spark 2,462 2,021 1,585
As % of all permanent IT jobs advertised in England 1.51% 1.20% 0.83%
As % of the Application Platforms category 9.05% 6.65% 4.44%
Number of salaries quoted 1,900 1,612 1,319
England median annual salary £70,000 £67,500 £65,000
Median salary % change year-on-year +3.70% +3.85% -
10th Percentile £43,750 £42,250 £41,250
90th Percentile £102,500 £100,000 £105,000
UK median annual salary £70,000 £67,000 £65,000
% change year-on-year +4.48% +3.08% -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Permanent vacancies with a requirement for application platform skills 27,203 30,403 35,714
As % of all permanent IT jobs advertised in England 16.65% 18.09% 18.71%
Number of salaries quoted 21,920 24,909 30,635
England median annual salary £50,000 £47,500 £45,000
Median salary % change year-on-year +5.26% +5.56% -
10th Percentile £28,000 £27,000 £26,500
90th Percentile £81,250 £76,250 £73,750
UK median annual salary £50,000 £47,500 £45,000
% change year-on-year +5.26% +5.56% -

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a percentage of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Salary Trend in England

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in England.

Salary trend for Apache Spark in England

Apache Spark
Salary Histogram in England

The salary distribution of IT jobs citing Apache Spark in England over the 6 months to 21 July 2018.

Salary histogram for Apache Spark in England

Apache Spark
Job Locations in England

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the England region over the 6 months to 21 July 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
London +62 1,722 £75,000 +7.14% 191
South East +35 259 £60,000 -7.69% 30
North of England -11 170 £55,000 - 27
East of England +18 135 £66,000 +32.00% 10
North West -21 107 £55,000 -8.33% 18
South West +19 89 £55,000 +4.76% 13
Midlands +4 80 £60,000 +23.71% 8
Yorkshire +31 58 £55,000 - 9
East Midlands +1 42 £60,000 +41.18% 6
West Midlands +37 38 £60,000 - 2
North East -27 5 £40,000 +14.29%
Apache Spark
UK

For the 6 months to 21 July 2018, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the England region with a requirement for Apache Spark.

1 1,757 (71.36%) Hadoop
2 1,654 (67.18%) Big Data
3 1,470 (59.71%) Python
4 1,204 (48.90%) Java
5 1,021 (41.47%) Scala
6 1,003 (40.74%) Analytics
7 942 (38.26%) Agile Software Development
8 939 (38.14%) SQL
9 870 (35.34%) Kafka
10 830 (33.71%) Machine Learning
11 829 (33.67%) Data Science
12 743 (30.18%) Amazon AWS
13 685 (27.82%) Apache Hive
14 613 (24.90%) NoSQL
15 497 (20.19%) R
16 470 (19.09%) Finance
17 448 (18.20%) ETL
18 434 (17.63%) Degree
19 429 (17.42%) Apache Cassandra
20 424 (17.22%) Linux
21 396 (16.08%) Algorithms
22 388 (15.76%) DevOps
23 385 (15.64%) Analytical Skills
24 376 (15.27%) Software Engineering
25 364 (14.78%) Computer Science
26 362 (14.70%) Problem-Solving
27 357 (14.50%) Microsoft Azure
28 356 (14.46%) Artificial Intelligence
28 356 (14.46%) TDD
28 356 (14.46%) Apache

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 356 (14.46%) Apache
2 239 (9.71%) Elasticsearch
3 167 (6.78%) Apache Pig
4 105 (4.26%) Apache Solr
5 101 (4.10%) SAS
6 35 (1.42%) Confluence
7 29 (1.18%) Tomcat
8 26 (1.06%) CMS
9 18 (0.73%) Jupyter
10 16 (0.65%) OpenStack
11 15 (0.61%) Apache Airflow
11 15 (0.61%) nginx
12 14 (0.57%) JBoss
13 13 (0.53%) Oracle SOA Suite
14 11 (0.45%) WildFly
15 8 (0.32%) GemFire
15 8 (0.32%) Jetty
16 7 (0.28%) NServiceBus
17 6 (0.24%) Oracle Workflow
18 5 (0.20%) WebLogic
Applications
1 38 (1.54%) Microsoft Excel
2 21 (0.85%) Chatbot
3 16 (0.65%) GNU Octave
4 12 (0.49%) SPSS
5 7 (0.28%) MS Visio
6 6 (0.24%) Microsoft PowerPoint
6 6 (0.24%) Weka
7 3 (0.12%) Mathematica
8 2 (0.081%) Microsoft Office
8 2 (0.081%) Microsoft Project
9 1 (0.041%) CPLEX
9 1 (0.041%) Mingle
9 1 (0.041%) SPSS Modeler
Business Applications
1 3 (0.12%) Distributed Ledger
2 1 (0.041%) Actimize
2 1 (0.041%) Adobe Campaign
2 1 (0.041%) Eloqua
2 1 (0.041%) NetSuite
2 1 (0.041%) Oracle 11i
2 1 (0.041%) Oracle AML
2 1 (0.041%) Oracle Assets
2 1 (0.041%) Oracle EBS
2 1 (0.041%) Oracle EBS R12
2 1 (0.041%) Oracle General Ledger
2 1 (0.041%) Oracle iProcurement
2 1 (0.041%) Oracle Payables
2 1 (0.041%) Oracle Receivables
2 1 (0.041%) WebTrends
Cloud Services
1 743 (30.18%) Amazon AWS
2 357 (14.50%) Microsoft Azure
3 159 (6.46%) Google Cloud Platform
4 154 (6.26%) Amazon EMR
5 121 (4.91%) Amazon S3
6 110 (4.47%) Amazon Kinesis
7 96 (3.90%) Serverless
8 74 (3.01%) AWS Lambda
9 67 (2.72%) GitHub
10 52 (2.11%) Amazon EC2
11 51 (2.07%) Cloud Computing
12 44 (1.79%) SaaS
13 38 (1.54%) IBM Cloud
14 33 (1.34%) PaaS
15 26 (1.06%) Azure Machine Learning
16 24 (0.97%) IaaS
17 17 (0.69%) Google Analytics
18 16 (0.65%) AWS CloudFormation
19 15 (0.61%) OpenShift
19 15 (0.61%) Slack
Communications & Networking
1 46 (1.87%) HTTP
2 30 (1.22%) Kerberos
3 20 (0.81%) DNS
3 20 (0.81%) Firewall
3 20 (0.81%) Internet
3 20 (0.81%) Sourcefire
4 18 (0.73%) TCP/IP
5 12 (0.49%) Network Security
5 12 (0.49%) SMS
5 12 (0.49%) VPN
6 9 (0.37%) SMTP
7 5 (0.20%) IPsec
7 5 (0.20%) MQTT
8 4 (0.16%) DAB
8 4 (0.16%) SSL
9 3 (0.12%) DHCP
9 3 (0.12%) DSL
9 3 (0.12%) FTP
10 2 (0.081%) UMTS
10 2 (0.081%) Wireless
Database & Business Intelligence
1 1,757 (71.36%) Hadoop
2 1,654 (67.18%) Big Data
3 685 (27.82%) Apache Hive
4 613 (24.90%) NoSQL
5 429 (17.42%) Apache Cassandra
6 342 (13.89%) HBase
7 326 (13.24%) MongoDB
8 260 (10.56%) Impala
9 248 (10.07%) Relational Database
10 234 (9.50%) Data Warehouse
11 224 (9.10%) Amazon Redshift
12 185 (7.51%) Data Lake
13 154 (6.26%) SQL Server
14 140 (5.69%) MySQL
15 118 (4.79%) PostgreSQL
15 118 (4.79%) RDBMS
16 109 (4.43%) Power BI
17 98 (3.98%) Sqoop
18 97 (3.94%) Data Mining
19 92 (3.74%) Redis
Development Applications
1 193 (7.84%) Jenkins
2 168 (6.82%) Git (software)
3 92 (3.74%) JIRA
4 73 (2.97%) Maven
5 68 (2.76%) JUnit
6 55 (2.23%) Gradle
7 54 (2.19%) Atlassian Bamboo
8 45 (1.83%) Selenium
9 37 (1.50%) Cucumber
10 31 (1.26%) Subversion
10 31 (1.26%) TeamCity
11 30 (1.22%) Vagrant
12 26 (1.06%) RStudio
12 26 (1.06%) Sonatype Nexus
13 21 (0.85%) Mockito
14 17 (0.69%) Artifactory
14 17 (0.69%) Bitbucket
14 17 (0.69%) gulp
14 17 (0.69%) JMeter
15 16 (0.65%) GitLab
General
1 470 (19.09%) Finance
2 250 (10.15%) Banking
3 183 (7.43%) Retail
4 152 (6.17%) Investment Banking
5 137 (5.56%) Marketing
6 102 (4.14%) Telecoms
7 95 (3.86%) Financial Institution
7 95 (3.86%) Front Office
8 93 (3.78%) Games
9 52 (2.11%) Electronics
10 49 (1.99%) Advertising
11 43 (1.75%) Manufacturing
12 39 (1.58%) Pharmaceutical
13 29 (1.18%) Law
14 22 (0.89%) Publishing
15 10 (0.41%) Military
16 9 (0.37%) Automotive
17 6 (0.24%) Legal
18 5 (0.20%) Music Industry
19 4 (0.16%) Retail Banking
Job Titles
1 580 (23.56%) Developer
2 562 (22.83%) Data Engineer
3 421 (17.10%) Data Scientist
4 301 (12.23%) Big Data Engineer
5 289 (11.74%) Architect
6 234 (9.50%) Java Developer
7 190 (7.72%) Data Architect
8 163 (6.62%) Senior Data Scientist
9 151 (6.13%) Big Data Developer
10 127 (5.16%) Scala Developer
11 126 (5.12%) Big Data Architect
12 109 (4.43%) Senior Developer
13 100 (4.06%) Software Engineer
14 95 (3.86%) Hadoop Engineer
15 86 (3.49%) Java Engineer
16 81 (3.29%) Senior Data Engineer
17 70 (2.84%) Python Engineer
18 68 (2.76%) Machine Learning Scientist
19 64 (2.60%) AWS Engineer
20 62 (2.52%) DevOps Engineer
Libraries, Frameworks & Software Standards
1 870 (35.34%) Kafka
2 189 (7.68%) Web Services
3 178 (7.23%) Natural Language Processing
4 164 (6.66%) .NET
5 160 (6.50%) AngularJS
6 153 (6.21%) Spring
7 135 (5.48%) Pandas
8 128 (5.20%) TensorFlow
9 123 (5.00%) Node.js
9 123 (5.00%) RESTful
10 122 (4.96%) REST
11 117 (4.75%) HTML
12 111 (4.51%) Spark Streaming
13 103 (4.18%) scikit-learn
14 99 (4.02%) Akka
15 91 (3.70%) React
16 89 (3.61%) NumPy
17 87 (3.53%) RabbitMQ
18 81 (3.29%) HTML5
19 79 (3.21%) JSON
Miscellaneous
1 396 (16.08%) Algorithms
2 385 (15.64%) Analytical Skills
3 364 (14.78%) Computer Science
4 161 (6.54%) Fintech
5 154 (6.26%) Data Structures
6 122 (4.96%) Greenfield Project
7 85 (3.45%) Cyberthreat
8 74 (3.01%) Self-Motivation
9 58 (2.36%) User Experience
10 57 (2.32%) Management Information System
11 43 (1.75%) Virtual Team
12 39 (1.58%) Linux Command Line
13 38 (1.54%) Derivative
14 33 (1.34%) Equities
15 30 (1.22%) Fixed Income
15 30 (1.22%) Hedge funds
16 28 (1.14%) Mobile App
17 26 (1.06%) Foreign Exchange (FX)
18 24 (0.97%) Clustering
18 24 (0.97%) Market Risk
Operating Systems
1 424 (17.22%) Linux
2 174 (7.07%) Unix
3 70 (2.84%) Windows
4 23 (0.93%) Ubuntu
5 17 (0.69%) Android
6 16 (0.65%) Apple iOS
7 15 (0.61%) CentOS
7 15 (0.61%) Red Hat Enterprise Linux
8 6 (0.24%) CoreOS
9 5 (0.20%) Amazon Linux
9 5 (0.20%) VMS
10 4 (0.16%) Windows Server
11 3 (0.12%) Debian
11 3 (0.12%) Windows 10
12 2 (0.081%) Mac OS
13 1 (0.041%) AIX
13 1 (0.041%) Mac OS X
13 1 (0.041%) Mesosphere
13 1 (0.041%) Solaris
13 1 (0.041%) Windows Server 2008
Processes & Methodologies
1 1,003 (40.74%) Analytics
2 942 (38.26%) Agile Software Development
3 830 (33.71%) Machine Learning
4 829 (33.67%) Data Science
5 448 (18.20%) ETL
6 388 (15.76%) DevOps
7 376 (15.27%) Software Engineering
8 362 (14.70%) Problem-Solving
9 356 (14.46%) Artificial Intelligence
9 356 (14.46%) TDD
10 327 (13.28%) Mathematics
11 303 (12.31%) Visualisation
12 299 (12.14%) Data Analytics
13 296 (12.02%) Scrum
14 287 (11.66%) Business Intelligence
15 271 (11.01%) Open Source
16 265 (10.76%) Data Modelling
17 258 (10.48%) Continuous Integration
18 244 (9.91%) Data Engineering
19 189 (7.68%) Statistics
Programming Languages
1 1,470 (59.71%) Python
2 1,204 (48.90%) Java
3 1,021 (41.47%) Scala
4 939 (38.14%) SQL
5 497 (20.19%) R
6 246 (9.99%) JavaScript
7 235 (9.55%) C#
8 227 (9.22%) C++
9 155 (6.30%) Java 8
10 128 (5.20%) Perl
11 120 (4.87%) C
12 114 (4.63%) Bash Shell
13 99 (4.02%) MATLAB
14 94 (3.82%) Ruby
15 88 (3.57%) Shell Script
16 62 (2.52%) Go
17 58 (2.36%) T-SQL
18 51 (2.07%) HiveQL
19 38 (1.54%) PL/SQL
20 29 (1.18%) PHP
Qualifications
1 434 (17.63%) Degree
2 168 (6.82%) PhD
3 131 (5.32%) Computer Science Degree
4 91 (3.70%) Security Cleared
5 64 (2.60%) Master's Degree
6 53 (2.15%) DV Cleared
7 48 (1.95%) Numerate Degree
8 21 (0.85%) CEH
8 21 (0.85%) CISSP
9 20 (0.81%) GCIH
9 20 (0.81%) OSCP
10 16 (0.65%) ISTQB
11 12 (0.49%) ISTQB Foundation Certification
12 10 (0.41%) SC Cleared
13 4 (0.16%) AWS Certification
13 4 (0.16%) ISEB
13 4 (0.16%) MBA
13 4 (0.16%) Postgraduate
14 2 (0.081%) MCSA
14 2 (0.081%) TOGAF Certification
Quality Assurance & Compliance
1 99 (4.02%) QA
2 56 (2.27%) Data Quality
3 39 (1.58%) GCP
4 14 (0.57%) GDPR
5 13 (0.53%) MiFID
6 9 (0.37%) ISO/IEC 27001
7 8 (0.32%) GPG13
8 5 (0.20%) SLA
8 5 (0.20%) Solvency II
9 2 (0.081%) EMIR
9 2 (0.081%) PMO
10 1 (0.041%) HIPAA
10 1 (0.041%) HL7
10 1 (0.041%) Sarbanes-Oxley
System Software
1 330 (13.40%) Docker
2 317 (12.88%) HDFS
3 175 (7.11%) Apache Flume
4 42 (1.71%) Apache ZooKeeper
5 33 (1.34%) VirtualBox
6 30 (1.22%) VMware Infrastructure
7 24 (0.97%) Virtual Machines
8 20 (0.81%) Active Directory
8 20 (0.81%) Snort
9 13 (0.53%) Hyper-V
10 7 (0.28%) LXC
10 7 (0.28%) Xen
11 5 (0.20%) KVM
12 3 (0.12%) Oracle RAC
13 2 (0.081%) vCloud
13 2 (0.081%) Virtual Servers
13 2 (0.081%) vSphere
14 1 (0.041%) Cronacle
Systems Management
1 171 (6.95%) Kubernetes
2 152 (6.17%) Ansible
3 132 (5.36%) Oozie
4 95 (3.86%) Hadoop YARN
5 93 (3.78%) Puppet
6 81 (3.29%) Terraform
7 68 (2.76%) Mesos
8 61 (2.48%) Opscode Chef
9 49 (1.99%) Kibana
10 34 (1.38%) Docker Swarm
11 31 (1.26%) Prometheus
12 24 (0.97%) logstash
13 20 (0.81%) Suricata
14 19 (0.77%) Ambari
14 19 (0.77%) Consul
14 19 (0.77%) Salt
15 18 (0.73%) Grafana
16 17 (0.69%) ITRS Geneos
17 12 (0.49%) Nagios
17 12 (0.49%) ZABBIX
Vendors
1 319 (12.96%) Cloudera
2 223 (9.06%) Google
3 221 (8.98%) Tableau
4 196 (7.96%) Oracle
5 170 (6.90%) Hortonworks
6 154 (6.26%) Talend
7 136 (5.52%) Microsoft
8 124 (5.04%) Informatica
9 71 (2.88%) IBM
9 71 (2.88%) SAP
10 60 (2.44%) Atlassian
11 58 (2.36%) MapR
12 51 (2.07%) Splunk
13 36 (1.46%) Sybase
13 36 (1.46%) VMware
14 35 (1.42%) Alteryx
15 32 (1.30%) Databricks
15 32 (1.30%) EXASOL
16 31 (1.26%) WorldPay
17 30 (1.22%) SnapLogic