Period
to 20 October 2018

The following table provides summary statistics for permanent job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 20 October 2018 with a comparison to the same period in the previous 2 years.

Apache Spark
UK > England
6 months to
20 Oct 2018
Same period 2017 Same period 2016
Rank 243 274 368
Rank change year-on-year +31 +94 +282
Permanent jobs citing Apache Spark 2,158 2,111 1,535
As % of all permanent IT jobs advertised in England 1.42% 1.29% 0.85%
As % of the Application Platforms category 8.57% 7.18% 4.62%
Number of salaries quoted 1,642 1,691 1,269
England median annual salary £70,000 £67,000 £66,250
Median salary % change year-on-year +4.48% +1.13% +1.92%
10th Percentile £45,000 £41,250 £40,000
90th Percentile £102,500 £101,250 £100,000
UK median annual salary £70,000 £65,000 £65,000
% change year-on-year +7.69% - -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Permanent vacancies with a requirement for application platform skills 25,176 29,416 33,245
As % of all permanent IT jobs advertised in England 16.53% 17.98% 18.45%
Number of salaries quoted 20,302 23,921 28,278
England median annual salary £50,000 £47,500 £45,000
Median salary % change year-on-year +5.26% +5.56% -
10th Percentile £28,000 £27,500 £27,000
90th Percentile £82,500 £77,500 £75,000
UK median annual salary £50,000 £47,500 £45,000
% change year-on-year +5.26% +5.56% -

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a percentage of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Salary Trend in England

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in England.

Salary trend for Apache Spark in England

Apache Spark
Salary Histogram in England

The salary distribution of IT jobs citing Apache Spark in England over the 6 months to 20 October 2018.

Salary histogram for Apache Spark in England

Apache Spark
Job Locations in England

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the England region over the 6 months to 20 October 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
London +8 1,446 £75,000 +7.14% 155
South East +21 262 £58,000 -3.33% 41
North of England +25 123 £65,000 +30.00% 28
Midlands +92 111 £72,500 +70.59% 10
South West +57 107 £60,000 +20.00% 18
East of England +20 104 £66,000 +10.00% 8
North West -15 88 £70,000 +33.33% 12
East Midlands +55 81 £75,000 +76.47% 7
Yorkshire +26 32 £50,500 +1.00% 12
West Midlands +41 30 £60,000 +41.18% 3
North East -7 5 £100,000 +185.71% 4
Apache Spark
UK

For the 6 months to 20 October 2018, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the England region with a requirement for Apache Spark.

1 1,582 (73.31%) Hadoop
2 1,468 (68.03%) Big Data
3 1,260 (58.39%) Python
4 1,105 (51.20%) Java
5 887 (41.10%) Agile Software Development
6 878 (40.69%) Scala
7 842 (39.02%) Kafka
8 840 (38.92%) SQL
9 796 (36.89%) Analytics
10 720 (33.36%) Amazon AWS
11 694 (32.16%) Apache Hive
12 669 (31.00%) Machine Learning
13 667 (30.91%) Data Science
14 599 (27.76%) NoSQL
15 465 (21.55%) Finance
16 399 (18.49%) Microsoft Azure
17 398 (18.44%) ETL
18 397 (18.40%) Apache Cassandra
19 396 (18.35%) Degree
20 379 (17.56%) R
21 375 (17.38%) Software Engineering
22 367 (17.01%) Apache
23 363 (16.82%) DevOps
24 346 (16.03%) HBase
25 342 (15.85%) Linux
26 335 (15.52%) Computer Science
27 323 (14.97%) Data Modelling
28 311 (14.41%) MongoDB
29 309 (14.32%) TDD
30 301 (13.95%) Visualisation

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 367 (17.01%) Apache
2 252 (11.68%) Elasticsearch
3 157 (7.28%) Apache Pig
4 99 (4.59%) SAS
5 72 (3.34%) Apache Solr
6 36 (1.67%) Confluence
7 29 (1.34%) Tomcat
8 24 (1.11%) Jupyter
9 17 (0.79%) Apache Airflow
9 17 (0.79%) GlassFish
9 17 (0.79%) nginx
10 11 (0.51%) JBoss
11 10 (0.46%) Cloud Foundry
11 10 (0.46%) GemFire
12 9 (0.42%) WebSphere
13 8 (0.37%) OpenStack
14 7 (0.32%) Oracle Workflow
15 6 (0.28%) IIS
15 6 (0.28%) NServiceBus
15 6 (0.28%) WildFly
Applications
1 17 (0.79%) Chatbot
2 14 (0.65%) GNU Octave
2 14 (0.65%) SPSS
3 11 (0.51%) Microsoft Excel
4 5 (0.23%) Weka
5 4 (0.19%) Microsoft Office
6 3 (0.14%) Microsoft PowerPoint
6 3 (0.14%) Microsoft Project
6 3 (0.14%) MS Visio
6 3 (0.14%) Stata
7 1 (0.046%) CPLEX
7 1 (0.046%) Mingle
7 1 (0.046%) SPSS Modeler
Business Applications
1 2 (0.093%) NetSuite
2 1 (0.046%) Adobe Campaign
2 1 (0.046%) Eloqua
2 1 (0.046%) WebTrends
Cloud Services
1 720 (33.36%) Amazon AWS
2 399 (18.49%) Microsoft Azure
3 214 (9.92%) Google Cloud Platform
4 118 (5.47%) Amazon EMR
5 95 (4.40%) Serverless
6 87 (4.03%) Amazon Kinesis
7 74 (3.43%) Cloud Computing
8 65 (3.01%) GitHub
9 56 (2.59%) AWS Lambda
10 53 (2.46%) Amazon EC2
11 48 (2.22%) Amazon S3
12 41 (1.90%) PaaS
13 37 (1.71%) SaaS
14 36 (1.67%) IBM Cloud
15 26 (1.20%) Azure Data Factory
15 26 (1.20%) Cortana Intelligence Suite
16 23 (1.07%) AWS CloudFormation
17 22 (1.02%) Amazon SageMaker
17 22 (1.02%) IaaS
18 19 (0.88%) Slack
Communications & Networking
1 50 (2.32%) HTTP
2 26 (1.20%) Internet
3 18 (0.83%) Kerberos
4 17 (0.79%) Sourcefire
5 12 (0.56%) DNS
6 11 (0.51%) Firewall
7 8 (0.37%) Network Security
7 8 (0.37%) SSL
7 8 (0.37%) TCP/IP
8 7 (0.32%) DAB
8 7 (0.32%) SMS
9 3 (0.14%) AMQP
9 3 (0.14%) SSH
10 2 (0.093%) DSL
10 2 (0.093%) Intrusion Detection
10 2 (0.093%) MQTT
10 2 (0.093%) VPN
11 1 (0.046%) IPsec
11 1 (0.046%) SAN
11 1 (0.046%) WAN
Database & Business Intelligence
1 1,582 (73.31%) Hadoop
2 1,468 (68.03%) Big Data
3 694 (32.16%) Apache Hive
4 599 (27.76%) NoSQL
5 397 (18.40%) Apache Cassandra
6 346 (16.03%) HBase
7 311 (14.41%) MongoDB
8 238 (11.03%) Data Warehouse
9 234 (10.84%) Impala
10 232 (10.75%) Amazon Redshift
11 190 (8.80%) Relational Database
12 167 (7.74%) SQL Server
13 139 (6.44%) Power BI
14 136 (6.30%) Data Lake
15 128 (5.93%) Flink
16 112 (5.19%) PostgreSQL
17 94 (4.36%) MySQL
18 92 (4.26%) Sqoop
19 84 (3.89%) SQL Server Integration Services
20 75 (3.48%) RDBMS
Development Applications
1 224 (10.38%) Git (software)
2 194 (8.99%) Jenkins
3 92 (4.26%) JIRA
4 65 (3.01%) JUnit
5 54 (2.50%) Gradle
6 44 (2.04%) Visual Studio
7 43 (1.99%) Maven
8 41 (1.90%) Atlassian Bamboo
8 41 (1.90%) Subversion
9 33 (1.53%) TeamCity
10 26 (1.20%) Vagrant
11 24 (1.11%) Selenium
12 22 (1.02%) Bitbucket
12 22 (1.02%) Jasmine
13 21 (0.97%) Visual Studio Team System
14 20 (0.93%) CircleCI
14 20 (0.93%) GoCD
15 19 (0.88%) Cucumber
16 18 (0.83%) GitLab
17 17 (0.79%) NUnit
General
1 465 (21.55%) Finance
2 217 (10.06%) Banking
3 195 (9.04%) Retail
4 140 (6.49%) Investment Banking
5 131 (6.07%) Telecoms
6 118 (5.47%) Marketing
7 85 (3.94%) Games
8 74 (3.43%) Financial Institution
9 71 (3.29%) Front Office
10 54 (2.50%) Electronics
11 50 (2.32%) Manufacturing
12 47 (2.18%) Law
13 43 (1.99%) Advertising
14 35 (1.62%) Automotive
15 24 (1.11%) Military
16 21 (0.97%) Pharmaceutical
17 10 (0.46%) Retail Banking
18 9 (0.42%) Publishing
19 6 (0.28%) German Language
19 6 (0.28%) Music Industry
Job Titles
1 613 (28.41%) Developer
2 502 (23.26%) Data Engineer
3 322 (14.92%) Data Scientist
4 319 (14.78%) Big Data Engineer
5 249 (11.54%) Architect
6 238 (11.03%) Java Developer
7 157 (7.28%) Data Architect
8 142 (6.58%) Big Data Developer
9 141 (6.53%) Senior Data Scientist
10 136 (6.30%) Senior Developer
11 107 (4.96%) Scala Developer
12 99 (4.59%) Software Engineer
13 90 (4.17%) Hadoop Engineer
14 81 (3.75%) Java Engineer
15 80 (3.71%) Python Developer
16 78 (3.61%) Hadoop Developer
17 73 (3.38%) Senior Data Engineer
17 73 (3.38%) Software Developer
18 68 (3.15%) Big Data Architect
19 65 (3.01%) DevOps Engineer
Libraries, Frameworks & Software Standards
1 842 (39.02%) Kafka
2 208 (9.64%) Web Services
3 180 (8.34%) AngularJS
4 149 (6.90%) HTML
5 144 (6.67%) Natural Language Processing
6 143 (6.63%) RESTful
7 142 (6.58%) Spring
8 140 (6.49%) Node.js
9 132 (6.12%) REST
10 127 (5.89%) .NET
11 123 (5.70%) Spark Streaming
11 123 (5.70%) TensorFlow
12 115 (5.33%) HTML5
13 113 (5.24%) React
14 102 (4.73%) Pandas
15 96 (4.45%) JSON
16 93 (4.31%) scikit-learn
17 89 (4.12%) RabbitMQ
18 87 (4.03%) Akka
19 77 (3.57%) CSS
Miscellaneous
1 335 (15.52%) Computer Science
2 301 (13.95%) Analytical Skills
3 291 (13.48%) Algorithms
4 126 (5.84%) Data Structures
5 123 (5.70%) Fintech
6 111 (5.14%) Greenfield Project
7 85 (3.94%) Self-Motivation
8 69 (3.20%) Cyberthreat
9 53 (2.46%) Derivative
10 46 (2.13%) Virtual Team
11 44 (2.04%) User Experience
12 38 (1.76%) Management Information System
13 33 (1.53%) Social Network
14 32 (1.48%) Public Cloud
15 30 (1.39%) Mobile App
16 29 (1.34%) Blog
17 27 (1.25%) Life Science
17 27 (1.25%) Market Risk
18 24 (1.11%) Equities
18 24 (1.11%) Linux Command Line
Operating Systems
1 342 (15.85%) Linux
2 175 (8.11%) Unix
3 68 (3.15%) Windows
4 20 (0.93%) Android
4 20 (0.93%) Apple iOS
5 17 (0.79%) Ubuntu
6 8 (0.37%) CentOS
6 8 (0.37%) Red Hat Enterprise Linux
7 7 (0.32%) VMS
8 3 (0.14%) CoreOS
9 2 (0.093%) DC/OS
9 2 (0.093%) Windows Server
10 1 (0.046%) AIX
10 1 (0.046%) Mac OS
10 1 (0.046%) Mesosphere
10 1 (0.046%) Windows Server 2012
Processes & Methodologies
1 887 (41.10%) Agile Software Development
2 796 (36.89%) Analytics
3 669 (31.00%) Machine Learning
4 667 (30.91%) Data Science
5 398 (18.44%) ETL
6 375 (17.38%) Software Engineering
7 363 (16.82%) DevOps
8 323 (14.97%) Data Modelling
9 309 (14.32%) TDD
10 301 (13.95%) Visualisation
11 291 (13.48%) Artificial Intelligence
12 278 (12.88%) Open Source
13 270 (12.51%) Data Analytics
13 270 (12.51%) Problem-Solving
14 260 (12.05%) Data Engineering
15 254 (11.77%) Mathematics
16 242 (11.21%) Business Intelligence
17 241 (11.17%) Microservices
18 232 (10.75%) Scrum
19 212 (9.82%) Continuous Integration
Programming Languages
1 1,260 (58.39%) Python
2 1,105 (51.20%) Java
3 878 (40.69%) Scala
4 840 (38.92%) SQL
5 379 (17.56%) R
6 240 (11.12%) JavaScript
7 228 (10.57%) C#
8 199 (9.22%) C++
9 112 (5.19%) Java 8
10 101 (4.68%) C
11 83 (3.85%) Shell Script
12 81 (3.75%) Bash Shell
13 72 (3.34%) Perl
13 72 (3.34%) Ruby
14 54 (2.50%) MATLAB
15 53 (2.46%) T-SQL
16 46 (2.13%) Go
17 41 (1.90%) PHP
18 36 (1.67%) PowerShell
19 32 (1.48%) TypeScript
Qualifications
1 396 (18.35%) Degree
2 136 (6.30%) Computer Science Degree
3 112 (5.19%) PhD
4 75 (3.48%) Security Cleared
5 50 (2.32%) DV Cleared
5 50 (2.32%) Numerate Degree
6 46 (2.13%) Master's Degree
7 23 (1.07%) SC Cleared
8 17 (0.79%) CEH
8 17 (0.79%) CISSP
8 17 (0.79%) GCIH
8 17 (0.79%) OSCP
9 7 (0.32%) ISEB
10 6 (0.28%) ISTQB
11 2 (0.093%) ISTQB Foundation Certification
11 2 (0.093%) Postgraduate
12 1 (0.046%) AWS Certification
12 1 (0.046%) ITIL Foundation Certificate
12 1 (0.046%) MCSA
12 1 (0.046%) Microsoft Certification
Quality Assurance & Compliance
1 99 (4.59%) QA
2 58 (2.69%) Data Quality
3 21 (0.97%) GDPR
4 14 (0.65%) MiFID
5 9 (0.42%) ISO/IEC 27001
6 7 (0.32%) GPG13
7 4 (0.19%) SLA
8 3 (0.14%) PCI DSS
8 3 (0.14%) Solvency II
9 1 (0.046%) HL7
9 1 (0.046%) PMO
System Software
1 291 (13.48%) HDFS
2 283 (13.11%) Docker
3 143 (6.63%) Apache Flume
4 44 (2.04%) Apache ZooKeeper
5 33 (1.53%) Virtual Machines
6 29 (1.34%) VirtualBox
7 19 (0.88%) Active Directory
8 17 (0.79%) Snort
9 14 (0.65%) VMware Infrastructure
10 8 (0.37%) Hyper-V
11 3 (0.14%) Oracle RAC
11 3 (0.14%) Xen
12 2 (0.093%) KVM
13 1 (0.046%) Firmware
13 1 (0.046%) LXC
13 1 (0.046%) Platform Symphony
13 1 (0.046%) Virtual Servers
13 1 (0.046%) VMware ESXi
Systems Management
1 170 (7.88%) Kubernetes
2 160 (7.41%) Oozie
3 125 (5.79%) Hadoop YARN
4 121 (5.61%) Ansible
5 98 (4.54%) Terraform
6 86 (3.99%) Puppet
7 77 (3.57%) Kibana
8 64 (2.97%) Mesos
9 51 (2.36%) Opscode Chef
10 46 (2.13%) Prometheus
11 42 (1.95%) logstash
12 36 (1.67%) Grafana
13 26 (1.20%) Docker Swarm
14 17 (0.79%) Ambari
14 17 (0.79%) Suricata
15 15 (0.70%) ITRS Geneos
16 9 (0.42%) Consul
16 9 (0.42%) ZABBIX
17 7 (0.32%) QRadar
18 6 (0.28%) Xymon
Vendors
1 278 (12.88%) Cloudera
2 226 (10.47%) Tableau
3 214 (9.92%) Hortonworks
4 210 (9.73%) Google
5 160 (7.41%) Microsoft
6 154 (7.14%) Oracle
7 109 (5.05%) Talend
8 107 (4.96%) SAP
9 97 (4.49%) Informatica
10 87 (4.03%) MapR
11 79 (3.66%) Databricks
12 66 (3.06%) IBM
13 48 (2.22%) Splunk
14 33 (1.53%) Atlassian
14 33 (1.53%) Sybase
15 31 (1.44%) TIBCO
16 28 (1.30%) Alteryx
17 24 (1.11%) Vertica
18 22 (1.02%) New Relic
19 20 (0.93%) HP