Period
to 16 December 2017

The following table provides summary statistics for permanent job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 16 December 2017 with a comparison to the same period in the previous 2 years.

Apache Spark
UK > England
6 months to
16 Dec 2017
Same period 2016 Same period 2015
Rank 252 330 566
Rank change year-on-year +78 +236 +579
Permanent jobs citing Apache Spark 2277 1692 948
As % of all permanent IT jobs advertised in England 1.36% 0.97% 0.47%
As % of the Application Platforms category 7.87% 5.31% 2.50%
Number of salaries quoted 1885 1417 770
England median annual salary £67,500 £67,500 £65,000
Median salary % change year-on-year - +3.85% -7.14%
10th Percentile £42,500 £40,000 £41,250
90th Percentile £102,500 £100,000 £93,750
UK median annual salary £67,500 £67,500 £65,000
% change year-on-year - +3.85% -7.14%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Permanent vacancies with a requirement for application platform skills 28915 31849 37962
As % of all permanent IT jobs advertised in England 17.32% 18.21% 18.77%
Number of salaries quoted 23675 26947 31918
England median annual salary £47,500 £46,500 £45,000
Median salary % change year-on-year +2.15% +3.33% +5.88%
10th Percentile £27,500 £27,000 £26,250
90th Percentile £77,500 £75,000 £73,750
UK median annual salary £47,500 £45,000 £45,000
% change year-on-year +5.56% - +5.88%

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a percentage of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Salary Trend in England

This chart provides the 3-month moving average for salaries quoted in permanent IT jobs citing Apache Spark in England.

Salary trend for Apache Spark in England

Apache Spark
Salary Histogram in England

The salary distribution of IT jobs citing Apache Spark in England over the 6 months to 16 December 2017.

Salary histogram for Apache Spark in England

Apache Spark
Job Locations in England

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the England region over the 6 months to 16 December 2017. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Job
Vacancies
London +62 1681 £70,000 -6.67% 160
South East +122 223 £57,500 -1.71% 9
North of England +12 123 £52,500 -0.47% 21
East of England +104 105 £60,000 +11.63% 11
South West -49 100 £50,000 - 5
North West +10 96 £52,500 -4.55% 15
Midlands -4 37 £42,500 -29.17% 6
Yorkshire +39 27 £52,500 -10.64% 5
East Midlands +15 22 £42,500 -5.56% 5
West Midlands -19 15 £42,500 -34.62% 1
Apache Spark
UK

For the 6 months to 16 December 2017, IT jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the number co-occurrences and its proportion to all job ads across the England region with a requirement for Apache Spark.

1 1581 (69.43%) Hadoop
2 1513 (66.45%) Big Data
3 1323 (58.10%) Python
4 1212 (53.23%) Java
5 995 (43.70%) Agile Software Development
6 928 (40.76%) Scala
7 878 (38.56%) Analytics
8 795 (34.91%) Machine Learning
9 756 (33.20%) SQL
10 685 (30.08%) Amazon AWS
11 627 (27.54%) Kafka
12 576 (25.30%) Finance
13 534 (23.45%) Apache Hive
14 482 (21.17%) NoSQL
15 415 (18.23%) Software Engineering
16 414 (18.18%) Docker
17 406 (17.83%) Apache Cassandra
18 390 (17.13%) Degree
19 383 (16.82%) Linux
20 376 (16.51%) DevOps
21 365 (16.03%) MongoDB
22 354 (15.55%) R
23 348 (15.28%) Apache
24 327 (14.36%) Business Intelligence
25 321 (14.10%) Algorithms
26 315 (13.83%) Computer Science
27 312 (13.70%) JavaScript
28 306 (13.44%) Scrum
28 306 (13.44%) Analytical Skills
29 300 (13.18%) Continuous Integration

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 348 (15.28%) Apache
2 279 (12.25%) Elasticsearch
3 158 (6.94%) Apache Pig
4 127 (5.58%) SAS
5 92 (4.04%) Apache Solr
6 53 (2.33%) Apache Airflow
7 52 (2.28%) Tomcat
8 42 (1.84%) OpenStack
9 41 (1.80%) Confluence
10 25 (1.10%) WebSphere
11 21 (0.92%) JBoss
12 19 (0.83%) Jetty
13 18 (0.79%) WebLogic
14 15 (0.66%) CMS
15 11 (0.48%) nginx
16 9 (0.40%) Jupyter
17 8 (0.35%) GemFire
17 8 (0.35%) IPython
17 8 (0.35%) Luigi
18 7 (0.31%) GeoServer
Applications
1 35 (1.54%) Microsoft Excel
2 31 (1.36%) SPSS
3 5 (0.22%) Microsoft Office
4 4 (0.18%) GNU Octave
4 4 (0.18%) Mathematica
4 4 (0.18%) Microsoft PowerPoint
4 4 (0.18%) Weka
5 3 (0.13%) Chatbot
5 3 (0.13%) S-PLUS
5 3 (0.13%) Spreadsheet
5 3 (0.13%) Stata
6 2 (0.088%) HipChat
7 1 (0.044%) LaTeX
Business Applications
1 3 (0.13%) Dynamics CRM
2 1 (0.044%) Capstone
2 1 (0.044%) Summit
Cloud Services
1 685 (30.08%) Amazon AWS
2 294 (12.91%) Microsoft Azure
3 155 (6.81%) Google Cloud Platform
4 89 (3.91%) GitHub
5 81 (3.56%) Amazon Kinesis
6 75 (3.29%) SaaS
7 71 (3.12%) Amazon EMR
8 55 (2.42%) Amazon S3
9 44 (1.93%) Cloud Computing
10 36 (1.58%) PaaS
11 35 (1.54%) AWS Lambda
12 30 (1.32%) Amazon EC2
13 29 (1.27%) Serverless
14 25 (1.10%) Azure Machine Learning
15 19 (0.83%) IaaS
15 19 (0.83%) Virtual Private Cloud
16 16 (0.70%) AWS CloudFormation
17 11 (0.48%) Amazon CloudWatch
17 11 (0.48%) Google Analytics
18 10 (0.44%) Datadog
Communications & Networking
1 31 (1.36%) Kerberos
2 28 (1.23%) Internet
3 14 (0.61%) DNS
3 14 (0.61%) HTTP
4 13 (0.57%) Firewall
5 12 (0.53%) TCP/IP
6 10 (0.44%) VPN
7 9 (0.40%) ICMP
8 8 (0.35%) LDAP
8 8 (0.35%) Network Security
9 6 (0.26%) DAB
10 3 (0.13%) SSL
11 2 (0.088%) GSM
11 2 (0.088%) LTE
11 2 (0.088%) UMTS
12 1 (0.044%) MQTT
12 1 (0.044%) Skype
12 1 (0.044%) Software-Defined Networking
12 1 (0.044%) Wi-Fi
12 1 (0.044%) Wireless
Database & Business Intelligence
1 1581 (69.43%) Hadoop
2 1513 (66.45%) Big Data
3 534 (23.45%) Apache Hive
4 482 (21.17%) NoSQL
5 406 (17.83%) Apache Cassandra
6 365 (16.03%) MongoDB
7 294 (12.91%) HBase
8 277 (12.17%) Relational Database
9 238 (10.45%) Data Warehouse
10 221 (9.71%) SQL Server
11 185 (8.12%) PostgreSQL
12 169 (7.42%) MySQL
13 163 (7.16%) Amazon Redshift
14 155 (6.81%) Impala
15 146 (6.41%) RDBMS
16 115 (5.05%) Data Mining
17 105 (4.61%) Redis
18 98 (4.30%) Sqoop
19 83 (3.65%) Neo4j
20 76 (3.34%) BigQuery
Development Applications
1 228 (10.01%) Git (software)
2 189 (8.30%) Jenkins
3 156 (6.85%) JIRA
4 90 (3.95%) Maven
5 55 (2.42%) Selenium
6 53 (2.33%) Atlassian Bamboo
7 52 (2.28%) Bitbucket
8 43 (1.89%) JUnit
9 41 (1.80%) Cucumber
10 34 (1.49%) Eclipse
11 31 (1.36%) IntelliJ
11 31 (1.36%) Talend Open Studio
11 31 (1.36%) Team Foundation Server
12 26 (1.14%) Vagrant
13 25 (1.10%) TeamCity
14 24 (1.05%) Gradle
15 19 (0.83%) Gerrit
16 18 (0.79%) Sonatype Nexus
16 18 (0.79%) Subversion
17 15 (0.66%) webpack
General
1 576 (25.30%) Finance
2 195 (8.56%) Marketing
3 172 (7.55%) Banking
4 153 (6.72%) Telecoms
5 140 (6.15%) Retail
6 100 (4.39%) Electronics
7 94 (4.13%) Manufacturing
8 74 (3.25%) Legal
9 71 (3.12%) Investment Banking
10 61 (2.68%) Front Office
11 43 (1.89%) Games
12 42 (1.84%) Financial Institution
13 38 (1.67%) Advertising
14 31 (1.36%) Pharmaceutical
15 25 (1.10%) Retail Banking
16 16 (0.70%) Law
17 15 (0.66%) Automotive
18 8 (0.35%) Publishing
19 7 (0.31%) Aerospace
20 5 (0.22%) Corporate Banking
Job Titles
1 644 (28.28%) Developer
2 397 (17.44%) Data Engineer
3 326 (14.32%) Data Scientist
4 279 (12.25%) Java Developer
5 239 (10.50%) Big Data Engineer
6 206 (9.05%) Architect
7 149 (6.54%) Software Engineer
8 132 (5.80%) Scala Developer
9 131 (5.75%) Big Data Developer
10 122 (5.36%) Senior Developer
11 115 (5.05%) Data Architect
12 102 (4.48%) Hadoop Engineer
13 101 (4.44%) Python Developer
14 92 (4.04%) Java Engineer
15 85 (3.73%) Senior Java Developer
16 80 (3.51%) DevOps Engineer
17 77 (3.38%) Big Data Architect
18 73 (3.21%) Hadoop Developer
19 69 (3.03%) Java Scala Developer
20 64 (2.81%) Senior Data Scientist
Libraries, Frameworks & Software Standards
1 627 (27.54%) Kafka
2 215 (9.44%) Spring
3 189 (8.30%) Natural Language Processing
4 188 (8.26%) Web Services
5 179 (7.86%) Node.js
6 172 (7.55%) AngularJS
7 158 (6.94%) HTML
8 145 (6.37%) CSS
9 132 (5.80%) REST
10 116 (5.09%) RabbitMQ
11 107 (4.70%) React
12 101 (4.44%) Akka
13 87 (3.82%) .NET
14 79 (3.47%) Hibernate
15 78 (3.43%) HTML5
16 73 (3.21%) scikit-learn
17 72 (3.16%) RESTful
18 71 (3.12%) Elastic Stack
18 71 (3.12%) Pandas
19 69 (3.03%) Spark Streaming
Miscellaneous
1 321 (14.10%) Algorithms
2 315 (13.83%) Computer Science
3 306 (13.44%) Analytical Skills
4 139 (6.10%) Greenfield Project
5 114 (5.01%) Fintech
6 77 (3.38%) Data Structures
7 57 (2.50%) Cyberthreat
8 45 (1.98%) User Experience
9 42 (1.84%) Clustering
10 39 (1.71%) Derivative
11 38 (1.67%) Fixed Income
12 37 (1.62%) Hedge funds
13 31 (1.36%) Enterprise Software
13 31 (1.36%) Mobile App
14 30 (1.32%) Public Cloud
15 29 (1.27%) Equities
16 28 (1.23%) Market Risk
17 25 (1.10%) Data Centre
18 22 (0.97%) Foreign Exchange (FX)
19 21 (0.92%) PKI
Operating Systems
1 383 (16.82%) Linux
2 104 (4.57%) Unix
3 66 (2.90%) Windows
4 19 (0.83%) Android
5 14 (0.61%) Red Hat Enterprise Linux
6 13 (0.57%) Apple iOS
6 13 (0.57%) Ubuntu
7 5 (0.22%) CentOS
8 4 (0.18%) openSUSE
8 4 (0.18%) SUSE
9 3 (0.13%) CoreOS
9 3 (0.13%) Mac OS X
10 2 (0.088%) Mesosphere
10 2 (0.088%) Solaris
11 1 (0.044%) AIX
11 1 (0.044%) Windows Server
Processes & Methodologies
1 995 (43.70%) Agile Software Development
2 878 (38.56%) Analytics
3 795 (34.91%) Machine Learning
4 415 (18.23%) Software Engineering
5 376 (16.51%) DevOps
6 327 (14.36%) Business Intelligence
7 306 (13.44%) Scrum
8 300 (13.18%) Continuous Integration
9 298 (13.09%) Data Analytics
10 297 (13.04%) TDD
11 283 (12.43%) Open Source
12 278 (12.21%) ETL
13 271 (11.90%) Visualisation
14 228 (10.01%) Microservices
15 217 (9.53%) MapReduce
16 208 (9.13%) Mathematics
17 192 (8.43%) Statistics
18 185 (8.12%) Artificial Intelligence
19 179 (7.86%) Test Automation
20 165 (7.25%) Data Visualisation
Programming Languages
1 1323 (58.10%) Python
2 1212 (53.23%) Java
3 928 (40.76%) Scala
4 756 (33.20%) SQL
5 354 (15.55%) R
6 312 (13.70%) JavaScript
7 215 (9.44%) C#
8 193 (8.48%) C++
9 162 (7.11%) Ruby
10 117 (5.14%) Go
11 107 (4.70%) Bash Shell
12 103 (4.52%) C
13 76 (3.34%) Shell Script
14 70 (3.07%) MATLAB
15 48 (2.11%) Perl
16 32 (1.41%) PHP
17 23 (1.01%) ES6
18 22 (0.97%) Haskell
19 21 (0.92%) VBA
20 20 (0.88%) PL/SQL
Qualifications
1 390 (17.13%) Degree
2 141 (6.19%) PhD
3 139 (6.10%) Computer Science Degree
4 101 (4.44%) Security Cleared
5 64 (2.81%) DV Cleared
5 64 (2.81%) Master's Degree
6 38 (1.67%) Numerate Degree
7 31 (1.36%) SC Cleared
8 15 (0.66%) ISEB
8 15 (0.66%) ISTQB
9 13 (0.57%) CISSP
10 12 (0.53%) CEH
10 12 (0.53%) GCIH
10 12 (0.53%) OSCP
11 9 (0.40%) MBA
12 6 (0.26%) AWS Certification
13 5 (0.22%) Postgraduate
14 4 (0.18%) ISTQB Foundation Certification
15 2 (0.088%) Microsoft Certification
15 2 (0.088%) PMI Certification
Quality Assurance & Compliance
1 80 (3.51%) Data Quality
2 77 (3.38%) QA
3 14 (0.61%) ISO/IEC 27001
4 12 (0.53%) GCP
4 12 (0.53%) GPG13
5 7 (0.31%) GDPR
6 6 (0.26%) SLA
7 5 (0.22%) Solvency II
8 4 (0.18%) Actionable Recommendations
8 4 (0.18%) Sarbanes-Oxley
9 1 (0.044%) MiFID
System Software
1 414 (18.18%) Docker
2 190 (8.34%) HDFS
3 130 (5.71%) Apache Flume
4 38 (1.67%) Apache ZooKeeper
5 17 (0.75%) VMware Infrastructure
6 16 (0.70%) Virtual Machines
7 14 (0.61%) Hyper-V
8 12 (0.53%) Snort
9 3 (0.13%) VirtualBox
10 2 (0.088%) MapR-FS
11 1 (0.044%) Active Directory
11 1 (0.044%) Cronacle
11 1 (0.044%) Cygwin
11 1 (0.044%) GridGain
Systems Management
1 174 (7.64%) Kubernetes
2 156 (6.85%) Ansible
3 114 (5.01%) Puppet
4 101 (4.44%) Hadoop YARN
5 98 (4.30%) Mesos
6 93 (4.08%) Opscode Chef
7 92 (4.04%) Terraform
8 85 (3.73%) Oozie
9 31 (1.36%) Kibana
10 21 (0.92%) Salt
11 18 (0.79%) logstash
12 14 (0.61%) Ambari
12 14 (0.61%) Marathon
13 12 (0.53%) Suricata
14 11 (0.48%) Grafana
15 9 (0.40%) Graphite
15 9 (0.40%) ITRS Geneos
16 7 (0.31%) Nagios
17 6 (0.26%) QRadar
18 5 (0.22%) Rancher
Vendors
1 207 (9.09%) Google
2 190 (8.34%) Oracle
3 175 (7.69%) Cloudera
4 168 (7.38%) Tableau
5 138 (6.06%) Microsoft
6 106 (4.66%) SAP
7 97 (4.26%) Hortonworks
8 54 (2.37%) Teradata
9 50 (2.20%) Splunk
10 40 (1.76%) MapR
11 36 (1.58%) VMware
12 33 (1.45%) Informatica
13 31 (1.36%) Talend
13 31 (1.36%) Twitter
14 27 (1.19%) RedGate
15 26 (1.14%) Atlassian
16 25 (1.10%) IBM
17 24 (1.05%) Alteryx
18 19 (0.83%) TIBCO
19 17 (0.75%) Netezza