Period
to 11 December 2018

The following table provides summary statistics for permanent job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 11 December 2018 with a comparison to the same period in the previous 2 years.

Apache Spark
UK > England
6 months to
11 Dec 2018
Same period 2017 Same period 2016
Rank 238 260 338
Rank change year-on-year +22 +78 +247
Permanent jobs citing Apache Spark 2,138 2,294 1,654
As % of all permanent IT jobs advertised in England 1.44% 1.36% 0.95%
As % of the Application Platforms category 8.89% 7.85% 5.18%
Number of salaries quoted 1,660 1,894 1,383
England median annual salary £75,000 £67,500 £67,500
Median salary % change year-on-year +11.11% - +3.85%
10th Percentile £46,475 £42,325 £40,000
90th Percentile £102,500 £102,500 £100,000
UK median annual salary £72,500 £67,500 £67,500
% change year-on-year +7.41% - +3.85%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Permanent vacancies with a requirement for application platform skills 24,046 29,217 31,924
As % of all permanent IT jobs advertised in England 16.20% 17.36% 18.29%
Number of salaries quoted 19,438 23,908 27,032
England median annual salary £50,000 £47,500 £46,500
Median salary % change year-on-year +5.26% +2.15% +3.33%
10th Percentile £28,250 £27,500 £27,000
90th Percentile £85,000 £77,500 £75,000
UK median annual salary £50,000 £47,500 £45,000
% change year-on-year +5.26% +5.56% -

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a percentage of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Salary Trend in England

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in England.

Salary trend for Apache Spark in England

Apache Spark
Salary Histogram in England

The salary distribution of IT jobs citing Apache Spark in England over the 6 months to 11 December 2018.

Salary histogram for Apache Spark in England

Apache Spark
Job Locations in England

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the England region over the 6 months to 11 December 2018. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
London -7 1,419 £77,500 +10.71% 208
South East +41 261 £65,000 +13.04% 33
North of England +58 151 £70,000 +33.33% 24
South West +38 123 £60,000 +20.00% 12
North West +17 105 £70,000 +33.33% 8
Midlands +95 100 £75,000 +76.47% 8
East of England 0 76 £66,750 +11.25% 9
East Midlands +71 74 £75,000 +76.47% 3
Yorkshire +47 38 £50,500 -3.81% 12
West Midlands +28 26 £40,000 -5.88% 5
North East - 12 £100,000 - 4
Apache Spark
UK

For the 6 months to 11 December 2018, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all permanent job ads across the England region with a requirement for Apache Spark.

1 1,577 (73.76%) Hadoop
2 1,491 (69.74%) Big Data
3 1,317 (61.60%) Python
4 1,108 (51.82%) Java
5 935 (43.73%) Agile Software Development
6 896 (41.91%) Scala
7 880 (41.16%) Kafka
8 801 (37.46%) SQL
9 799 (37.37%) Amazon AWS
10 769 (35.97%) Analytics
11 737 (34.47%) Apache Hive
12 718 (33.58%) Machine Learning
13 712 (33.30%) Data Science
14 668 (31.24%) NoSQL
15 455 (21.28%) Microsoft Azure
16 453 (21.19%) Finance
17 449 (21.00%) Apache Cassandra
18 438 (20.49%) Apache
19 421 (19.69%) DevOps
20 416 (19.46%) Software Engineering
21 397 (18.57%) Degree
21 397 (18.57%) Data Modelling
22 391 (18.29%) HBase
23 379 (17.73%) ETL
24 370 (17.31%) R
25 353 (16.51%) Artificial Intelligence
26 343 (16.04%) Linux
27 338 (15.81%) Computer Science
28 316 (14.78%) Visualisation
29 315 (14.73%) MongoDB

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 438 (20.49%) Apache
2 276 (12.91%) Elasticsearch
3 198 (9.26%) Apache Pig
4 129 (6.03%) SAS
5 95 (4.44%) Apache Solr
6 33 (1.54%) Tomcat
7 26 (1.22%) Confluence
7 26 (1.22%) GlassFish
8 21 (0.98%) Jupyter
9 20 (0.94%) Apache Airflow
10 18 (0.84%) JBoss
11 16 (0.75%) WebSphere
12 15 (0.70%) nginx
13 11 (0.51%) Apache Geronimo
14 8 (0.37%) Cloud Foundry
15 6 (0.28%) GemFire
16 5 (0.23%) IIS
16 5 (0.23%) Luigi
16 5 (0.23%) OpenStack
17 4 (0.19%) WildFly
Applications
1 19 (0.89%) SPSS
2 18 (0.84%) Chatbot
3 11 (0.51%) GNU Octave
3 11 (0.51%) Weka
4 6 (0.28%) Microsoft Excel
4 6 (0.28%) Stata
5 4 (0.19%) Microsoft Office
6 3 (0.14%) Microsoft PowerPoint
6 3 (0.14%) MS Visio
7 1 (0.047%) CPLEX
7 1 (0.047%) Microsoft Project
Business Applications
1 2 (0.094%) NetSuite
2 1 (0.047%) Adobe Campaign
2 1 (0.047%) Eloqua
2 1 (0.047%) WebTrends
Cloud Services
1 799 (37.37%) Amazon AWS
2 455 (21.28%) Microsoft Azure
3 268 (12.54%) Google Cloud Platform
4 124 (5.80%) Amazon EMR
5 107 (5.00%) Cloud Computing
6 98 (4.58%) Serverless
7 79 (3.70%) GitHub
8 70 (3.27%) AWS Lambda
9 68 (3.18%) Amazon EC2
10 65 (3.04%) Amazon Kinesis
11 44 (2.06%) PaaS
12 42 (1.96%) Amazon S3
13 38 (1.78%) SaaS
14 33 (1.54%) Azure Data Factory
15 30 (1.40%) AWS CloudFormation
16 29 (1.36%) Cortana Intelligence Suite
17 24 (1.12%) Amazon SageMaker
17 24 (1.12%) IaaS
18 21 (0.98%) IBM Cloud
19 20 (0.94%) Slack
Communications & Networking
1 52 (2.43%) HTTP
2 39 (1.82%) Internet
3 14 (0.65%) Kerberos
4 13 (0.61%) DNS
5 9 (0.42%) Sourcefire
6 8 (0.37%) Firewall
7 6 (0.28%) AMQP
7 6 (0.28%) SSL
8 5 (0.23%) DAB
8 5 (0.23%) TCP/IP
9 4 (0.19%) Network Security
9 4 (0.19%) SMS
10 3 (0.14%) SSH
11 2 (0.094%) DSL
11 2 (0.094%) MQTT
11 2 (0.094%) VPN
12 1 (0.047%) 5G
12 1 (0.047%) NX-OS
12 1 (0.047%) SAN
12 1 (0.047%) WAN
Database & Business Intelligence
1 1,577 (73.76%) Hadoop
2 1,491 (69.74%) Big Data
3 737 (34.47%) Apache Hive
4 668 (31.24%) NoSQL
5 449 (21.00%) Apache Cassandra
6 391 (18.29%) HBase
7 315 (14.73%) MongoDB
8 259 (12.11%) Impala
9 256 (11.97%) Amazon Redshift
10 230 (10.76%) Data Warehouse
11 166 (7.76%) Flink
12 165 (7.72%) SQL Server
13 163 (7.62%) Relational Database
14 154 (7.20%) Power BI
15 141 (6.59%) Data Lake
16 134 (6.27%) PostgreSQL
17 106 (4.96%) MySQL
18 84 (3.93%) Sqoop
19 81 (3.79%) CouchDB
20 77 (3.60%) QlikView
Development Applications
1 219 (10.24%) Git (software)
2 165 (7.72%) Jenkins
3 102 (4.77%) JIRA
4 60 (2.81%) Visual Studio
5 49 (2.29%) Gradle
6 42 (1.96%) Maven
7 40 (1.87%) JUnit
8 34 (1.59%) Selenium
9 32 (1.50%) Atlassian Bamboo
10 30 (1.40%) Subversion
11 28 (1.31%) Visual Studio Team System
12 24 (1.12%) CircleCI
12 24 (1.12%) GoCD
13 23 (1.08%) JMeter
13 23 (1.08%) TeamCity
14 21 (0.98%) Ant
14 21 (0.98%) SoapUI
15 20 (0.94%) GitLab
15 20 (0.94%) Vagrant
16 16 (0.75%) Jasmine
General
1 453 (21.19%) Finance
2 198 (9.26%) Retail
3 171 (8.00%) Telecoms
4 163 (7.62%) Banking
5 109 (5.10%) Law
6 99 (4.63%) Marketing
7 78 (3.65%) Investment Banking
8 77 (3.60%) Games
9 66 (3.09%) Military
10 43 (2.01%) Manufacturing
11 40 (1.87%) Advertising
11 40 (1.87%) Automotive
12 38 (1.78%) Electronics
13 30 (1.40%) Financial Institution
14 25 (1.17%) Front Office
15 24 (1.12%) Pharmaceutical
16 13 (0.61%) Retail Banking
17 8 (0.37%) Billing
18 6 (0.28%) French Language
18 6 (0.28%) Music Industry
Job Titles
1 543 (25.40%) Data Engineer
2 541 (25.30%) Developer
3 354 (16.56%) Big Data Engineer
4 314 (14.69%) Data Scientist
5 259 (12.11%) Architect
6 192 (8.98%) Java Developer
7 181 (8.47%) Data Architect
8 129 (6.03%) Senior Data Scientist
9 123 (5.75%) Big Data Developer
10 110 (5.14%) Senior Developer
11 99 (4.63%) Senior Data Engineer
12 98 (4.58%) Hadoop Engineer
13 95 (4.44%) Software Engineer
14 93 (4.35%) Consultant
15 84 (3.93%) Scala Developer
16 81 (3.79%) Python Engineer
17 78 (3.65%) Big Data Architect
18 76 (3.55%) Python Developer
19 74 (3.46%) Java Engineer
20 68 (3.18%) Software Developer
Libraries, Frameworks & Software Standards
1 880 (41.16%) Kafka
2 227 (10.62%) Web Services
3 156 (7.30%) AngularJS
4 148 (6.92%) Spark Streaming
5 145 (6.78%) HTML
6 144 (6.74%) Natural Language Processing
7 140 (6.55%) Node.js
8 136 (6.36%) REST
9 123 (5.75%) RESTful
10 121 (5.66%) Spring
10 121 (5.66%) TensorFlow
11 108 (5.05%) HTML5
11 108 (5.05%) React
12 101 (4.72%) .NET
13 100 (4.68%) scikit-learn
14 98 (4.58%) Pandas
15 93 (4.35%) D3.js
15 93 (4.35%) JSON
16 85 (3.98%) Akka
17 80 (3.74%) CSS
Miscellaneous
1 279 (13.05%) Algorithms
2 253 (11.83%) Analytical Skills
3 108 (5.05%) Data Structures
4 103 (4.82%) Fintech
5 95 (4.44%) Greenfield Project
6 80 (3.74%) Self-Motivation
7 49 (2.29%) Virtual Team
8 48 (2.25%) Social Network
9 41 (1.92%) Derivative
10 38 (1.78%) Public Cloud
11 35 (1.64%) Life Science
12 31 (1.45%) User Experience
13 28 (1.31%) Blog
14 25 (1.17%) Management Information System
14 25 (1.17%) Market Risk
15 23 (1.08%) Mobile App
16 22 (1.03%) Linux Command Line
16 22 (1.03%) Social Media
17 21 (0.98%) Cyberthreat
17 21 (0.98%) Internet of Things
Operating Systems
1 343 (16.04%) Linux
2 188 (8.79%) Unix
3 68 (3.18%) Windows
4 20 (0.94%) Android
4 20 (0.94%) Apple iOS
5 11 (0.51%) Ubuntu
6 10 (0.47%) CentOS
7 9 (0.42%) Red Hat Enterprise Linux
8 5 (0.23%) VMS
9 2 (0.094%) CoreOS
9 2 (0.094%) DC/OS
9 2 (0.094%) Windows Server
10 1 (0.047%) AIX
10 1 (0.047%) Mac OS
10 1 (0.047%) Windows Server 2012
Processes & Methodologies
1 935 (43.73%) Agile Software Development
2 769 (35.97%) Analytics
3 718 (33.58%) Machine Learning
4 712 (33.30%) Data Science
5 421 (19.69%) DevOps
6 416 (19.46%) Software Engineering
7 397 (18.57%) Data Modelling
8 379 (17.73%) ETL
9 353 (16.51%) Artificial Intelligence
10 338 (15.81%) Computer Science
11 316 (14.78%) Visualisation
12 307 (14.36%) Data Engineering
13 306 (14.31%) Open Source
14 296 (13.84%) Business Intelligence
15 291 (13.61%) Data Analytics
15 291 (13.61%) TDD
16 270 (12.63%) Mathematics
17 235 (10.99%) Microservices
18 231 (10.80%) Scrum
19 210 (9.82%) Statistics
Programming Languages
1 1,317 (61.60%) Python
2 1,108 (51.82%) Java
3 896 (41.91%) Scala
4 801 (37.46%) SQL
5 370 (17.31%) R
6 209 (9.78%) JavaScript
7 208 (9.73%) C#
8 181 (8.47%) C++
9 87 (4.07%) C
10 79 (3.70%) Java 8
11 72 (3.37%) Bash Shell
12 71 (3.32%) Shell Script
13 64 (2.99%) Ruby
14 54 (2.53%) Perl
15 53 (2.48%) T-SQL
16 48 (2.25%) Go
17 47 (2.20%) PHP
18 40 (1.87%) MATLAB
18 40 (1.87%) PowerShell
19 28 (1.31%) PL/SQL
Qualifications
1 397 (18.57%) Degree
2 142 (6.64%) Computer Science Degree
3 95 (4.44%) PhD
4 65 (3.04%) Numerate Degree
5 60 (2.81%) Security Cleared
6 33 (1.54%) Master's Degree
7 32 (1.50%) DV Cleared
8 26 (1.22%) SC Cleared
9 10 (0.47%) CEH
10 9 (0.42%) CISSP
10 9 (0.42%) GCIH
10 9 (0.42%) OSCP
11 6 (0.28%) ISEB
11 6 (0.28%) ISTQB
12 4 (0.19%) MCAD
12 4 (0.19%) MCDBA
12 4 (0.19%) MCSD
12 4 (0.19%) Microsoft Certification
13 2 (0.094%) ISTQB Foundation Certification
13 2 (0.094%) TOGAF Certification
Quality Assurance & Compliance
1 97 (4.54%) Data Quality
2 93 (4.35%) QA
3 20 (0.94%) GDPR
4 17 (0.80%) MiFID
5 5 (0.23%) ISO/IEC 27001
5 5 (0.23%) PCI DSS
6 3 (0.14%) GPG13
6 3 (0.14%) Solvency II
7 1 (0.047%) EMIR
7 1 (0.047%) HL7
7 1 (0.047%) ISO 9001
7 1 (0.047%) PMO
7 1 (0.047%) SLA
System Software
1 262 (12.25%) Docker
2 242 (11.32%) HDFS
3 148 (6.92%) Apache Flume
4 57 (2.67%) Apache ZooKeeper
5 21 (0.98%) Virtual Machines
6 19 (0.89%) VirtualBox
7 17 (0.80%) Active Directory
8 9 (0.42%) Snort
9 6 (0.28%) VMware Infrastructure
10 3 (0.14%) Hyper-V
11 1 (0.047%) Firmware
11 1 (0.047%) Platform Symphony
11 1 (0.047%) VMware ESXi
Systems Management
1 205 (9.59%) Oozie
2 175 (8.19%) Hadoop YARN
3 161 (7.53%) Kubernetes
4 120 (5.61%) Ansible
5 117 (5.47%) Terraform
6 83 (3.88%) Kibana
7 78 (3.65%) Puppet
8 65 (3.04%) Mesos
9 59 (2.76%) Opscode Chef
10 50 (2.34%) logstash
11 46 (2.15%) Prometheus
12 37 (1.73%) Grafana
13 27 (1.26%) Ambari
14 17 (0.80%) Docker Swarm
15 11 (0.51%) ITRS Geneos
16 10 (0.47%) Nagios
17 9 (0.42%) Suricata
18 6 (0.28%) Oracle ZFS
18 6 (0.28%) ZABBIX
19 5 (0.23%) Salt
Vendors
1 280 (13.10%) Cloudera
2 264 (12.35%) Tableau
3 251 (11.74%) Hortonworks
4 210 (9.82%) Google
5 188 (8.79%) Microsoft
6 167 (7.81%) Oracle
7 115 (5.38%) Databricks
8 110 (5.14%) MapR
8 110 (5.14%) SAP
9 78 (3.65%) Informatica
10 72 (3.37%) Talend
11 61 (2.85%) IBM
12 49 (2.29%) Splunk
13 44 (2.06%) CA Agile Central
14 33 (1.54%) Alteryx
15 30 (1.40%) TIBCO
16 25 (1.17%) Teradata
17 22 (1.03%) Ab Initio
17 22 (1.03%) Netezza
18 19 (0.89%) Greenplum