Period
to 23 October 2019

The following table provides summary statistics for contract job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 23 October 2019 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
UK > England
6 months to
23 Oct 2019
Same period 2018 Same period 2017
Rank 168 191 176
Rank change year-on-year +23 -15 +91
Contract jobs citing Apache Spark 1,221 1,388 1,387
As % of all contract IT jobs advertised in England 1.58% 1.47% 1.50%
As % of the Application Platforms category 10.56% 8.74% 8.97%
Number of daily rates quoted 924 996 1,016
England median daily rate £563 £550 £550
Median daily rate % change year-on-year +2.27% - +1.85%
10th Percentile £427 £425 £400
90th Percentile £725 £713 £688
UK median daily rate £550 £550 £550
Number of hourly rates quoted 3 9 8
England median hourly rate £24.65 £45.00 £19.34
Median hourly rate % change year-on-year -45.22% +132.68% -5.66%
10th Percentile £21.93 £16.65 £16.64
90th Percentile £32.93 £94.35 £47.50
UK median hourly rate £24.65 £45.00 £19.34
% change year-on-year -45.22% +132.68% -5.66%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Contract vacancies with a requirement for application platform skills 11,560 15,878 15,463
As % of all contract IT jobs advertised in England 14.96% 16.79% 16.78%
Number of daily rates quoted 7,571 10,467 10,196
England median daily rate £463 £465 £450
Median daily rate % change year-on-year -0.54% +3.33% +5.88%
10th Percentile £263 £275 £263
90th Percentile £638 £631 £613
UK median daily rate £450 £450 £445
% change year-on-year - +1.12% +5.95%
Number of hourly rates quoted 263 383 387
England median hourly rate £20.83 £21.50 £20.44
Median hourly rate % change year-on-year -3.12% +5.21% +2.18%
10th Percentile £11.00 £12.00 £11.25
90th Percentile £53.75 £50.40 £48.75
UK median hourly rate £21.50 £21.25 £20.00
% change year-on-year +1.18% +6.25% -

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a percentage of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Contractor Daily Rate Trend in England

This chart provides the 3-month moving average for daily rates quoted in contract jobs citing Apache Spark in England.

Contractor daily rate trend for Apache Spark in England

Apache Spark
Contractor Daily Rate Histogram in England

The daily rate distribution of IT jobs citing Apache Spark in England over the 6 months to 23 October 2019.

Contractor daily rate histogram for Apache Spark in England

Apache Spark
Contractor Hourly Rate Trend in England

This chart provides the 3-month moving average for contractor hourly rates quoted in IT jobs citing Apache Spark in England.

Contractor hourly rate trend for Apache Spark in England

Apache Spark
Contractor Hourly Rate Histogram in England

The hourly rate distribution of IT jobs citing Apache Spark in England over the 6 months to 23 October 2019.

Contractor hourly rate histogram for Apache Spark in England

Apache Spark
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the England region over the 6 months to 23 October 2019. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London +31 899 £575 - 161
South East -30 117 £531 +3.51% 24
North of England +30 96 £531 +1.19% 16
North West +30 52 £525 -4.55% 9
East of England +21 47 £525 -4.55% 10
Yorkshire +18 37 £538 +13.16% 5
Midlands +62 30 £513 +22.45% 9
South West -39 28 £600 +14.29% 10
West Midlands +45 17 £470 +25.33% 1
East Midlands +27 13 £525 +25.37% 8
North East +20 7 £563 -4.26% 2
Apache Spark
UK

For the 6 months to 23 October 2019, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region with a requirement for Apache Spark.

1 781 (63.96%) Python
2 763 (62.49%) Big Data
3 722 (59.13%) Hadoop
4 516 (42.26%) SQL
5 458 (37.51%) AWS
6 454 (37.18%) Agile Software Development
6 454 (37.18%) Kafka
7 407 (33.33%) Java
8 387 (31.70%) Scala
9 368 (30.14%) Apache Hive
10 348 (28.50%) Data Science
11 301 (24.65%) Analytics
12 287 (23.51%) Azure
13 286 (23.42%) DevOps
14 282 (23.10%) Finance
15 259 (21.21%) ETL
16 257 (21.05%) Apache
17 249 (20.39%) Docker
18 248 (20.31%) Machine Learning
19 226 (18.51%) GCP
19 226 (18.51%) Linux
20 221 (18.10%) Data Engineering
21 208 (17.04%) NoSQL
22 203 (16.63%) Data Warehouse
23 202 (16.54%) Git (software)
24 193 (15.81%) Data Modelling
25 188 (15.40%) Kubernetes
26 184 (15.07%) Jenkins
27 167 (13.68%) Computer Science
28 165 (13.51%) Google

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 257 (21.05%) Apache
2 91 (7.45%) Elasticsearch
3 70 (5.73%) Apache Airflow
4 46 (3.77%) Jupyter
5 29 (2.38%) SAS
6 23 (1.88%) Apache Solr
7 20 (1.64%) Apache Pig
8 19 (1.56%) Confluence
9 14 (1.15%) Luigi
10 10 (0.82%) Pachyderm
11 9 (0.74%) Jetty
11 9 (0.74%) Tomcat
12 6 (0.49%) nginx
12 6 (0.49%) Oracle Coherence
13 5 (0.41%) Cloud Foundry
13 5 (0.41%) JBoss
13 5 (0.41%) Oracle Self-Service HR
13 5 (0.41%) WebLogic
14 4 (0.33%) ColdFusion
14 4 (0.33%) WebSphere
Applications
1 33 (2.70%) Microsoft Excel
2 5 (0.41%) Microsoft Office
3 1 (0.082%) Microsoft Project
3 1 (0.082%) Stata
Business Applications
1 7 (0.57%) FINCAD Analytics
2 6 (0.49%) SAP Oil and Gas
3 2 (0.16%) Adobe Campaign
3 2 (0.16%) Cognos Business Intelligence
3 2 (0.16%) Distributed Ledger
4 1 (0.082%) NetSuite
Cloud Services
1 458 (37.51%) AWS
2 287 (23.51%) Azure
3 226 (18.51%) GCP
4 123 (10.07%) Amazon S3
5 118 (9.66%) Amazon EMR
6 117 (9.58%) Serverless
7 100 (8.19%) AWS Lambda
8 93 (7.62%) Amazon Kinesis
9 50 (4.10%) Power Platform
10 49 (4.01%) Azure Data Factory
11 44 (3.60%) Amazon SQS
11 44 (3.60%) GitHub
12 37 (3.03%) Amazon EC2
13 31 (2.54%) OpenShift
14 28 (2.29%) CloudFront
15 21 (1.72%) AWS CloudFormation
16 20 (1.64%) Cloud Computing
17 19 (1.56%) PaaS
18 18 (1.47%) Amazon SageMaker
18 18 (1.47%) Azure Machine Learning
Communications & Networking
1 28 (2.29%) Kerberos
2 14 (1.15%) SSL
3 13 (1.06%) DNS
3 13 (1.06%) HTTP
4 7 (0.57%) SAN
5 6 (0.49%) Firewall
5 6 (0.49%) TCP/IP
6 5 (0.41%) Internet
6 5 (0.41%) SSH
7 4 (0.33%) DSL
7 4 (0.33%) F5 BIG-IP GTM
7 4 (0.33%) F5 BIG-IP LTM
7 4 (0.33%) FTP
8 3 (0.25%) Software-Defined Networking
9 2 (0.16%) Multicast
10 1 (0.082%) LAN
10 1 (0.082%) VPN
10 1 (0.082%) WAN
Database & Business Intelligence
1 763 (62.49%) Big Data
2 722 (59.13%) Hadoop
3 368 (30.14%) Apache Hive
4 208 (17.04%) NoSQL
5 203 (16.63%) Data Warehouse
6 154 (12.61%) Impala
7 121 (9.91%) Data Lake
8 118 (9.66%) PostgreSQL
9 113 (9.25%) HBase
10 110 (9.01%) MongoDB
11 109 (8.93%) Amazon Redshift
12 87 (7.13%) Apache Cassandra
13 86 (7.04%) Relational Database
14 80 (6.55%) SQL Server
15 64 (5.24%) BigQuery
16 61 (5.00%) MySQL
17 60 (4.91%) Spark SQL
18 52 (4.26%) Flink
19 50 (4.10%) Power BI
20 46 (3.77%) Sqoop
Development Applications
1 202 (16.54%) Git (software)
2 184 (15.07%) Jenkins
3 61 (5.00%) JIRA
4 39 (3.19%) Atlassian Bamboo
5 34 (2.78%) Sonatype Nexus
6 33 (2.70%) Maven
7 26 (2.13%) Artifactory
8 23 (1.88%) IntelliJ
9 21 (1.72%) Gradle
10 19 (1.56%) GitLab
11 16 (1.31%) Vagrant
12 15 (1.23%) Cucumber
13 14 (1.15%) Selenium
14 11 (0.90%) CircleCI
14 11 (0.90%) Eclipse
14 11 (0.90%) Mercurial
15 10 (0.82%) TeamCity
15 10 (0.82%) Travis CI
16 9 (0.74%) Subversion
16 9 (0.74%) Visual Studio Team System
General
1 282 (23.10%) Finance
2 111 (9.09%) Banking
3 63 (5.16%) Front Office
4 42 (3.44%) Investment Banking
5 36 (2.95%) Legal
6 31 (2.54%) Retail
7 21 (1.72%) Public Sector
8 17 (1.39%) Marketing
9 16 (1.31%) Telecoms
10 13 (1.06%) Retail Banking
11 10 (0.82%) Advertising
11 10 (0.82%) Financial Institution
11 10 (0.82%) Publishing
12 9 (0.74%) Pharmaceutical
13 7 (0.57%) Games
14 6 (0.49%) Manufacturing
15 3 (0.25%) Aerospace
16 2 (0.16%) Automotive
16 2 (0.16%) International Banking
17 1 (0.082%) Police
Job Titles
1 490 (40.13%) Data Engineer
2 289 (23.67%) Developer
3 173 (14.17%) Big Data Engineer
4 124 (10.16%) Python Engineer
5 110 (9.01%) Architect
6 93 (7.62%) Python Developer
7 85 (6.96%) AWS Engineer
7 85 (6.96%) Data Scientist
7 85 (6.96%) Java Developer
8 76 (6.22%) Big Data Developer
9 72 (5.90%) Data Architect
10 65 (5.32%) DevOps Engineer
11 58 (4.75%) Senior Data Engineer
12 56 (4.59%) Hadoop Engineer
13 55 (4.50%) Senior Developer
14 53 (4.34%) Cloud Engineer
15 52 (4.26%) Scala Developer
16 43 (3.52%) Hadoop Developer
17 38 (3.11%) Apache Spark Developer
18 36 (2.95%) Platform Engineer
Libraries, Frameworks & Software Standards
1 454 (37.18%) Kafka
2 162 (13.27%) PySpark
3 84 (6.88%) Pandas
4 76 (6.22%) Spring
5 69 (5.65%) Spark Streaming
6 54 (4.42%) Web Services
7 49 (4.01%) scikit-learn
8 48 (3.93%) Natural Language Processing
9 46 (3.77%) React
10 45 (3.69%) NumPy
11 41 (3.36%) TensorFlow
12 39 (3.19%) Spring Boot
13 38 (3.11%) RESTful
14 34 (2.78%) Apache NiFi
15 32 (2.62%) RabbitMQ
16 30 (2.46%) JSON
16 30 (2.46%) REST
17 27 (2.21%) Apache Beam
18 26 (2.13%) AngularJS
19 24 (1.97%) Apache Avro
Miscellaneous
1 114 (9.34%) Algorithms
2 103 (8.44%) Analytical Skills
3 91 (7.45%) Data Structures
4 67 (5.49%) Greenfield Project
5 35 (2.87%) User Experience
6 33 (2.70%) Self-Motivation
7 25 (2.05%) Public Cloud
8 22 (1.80%) Cloud Native
9 20 (1.64%) Derivative
10 16 (1.31%) Foreign Exchange (FX)
10 16 (1.31%) Grid Computing
10 16 (1.31%) Hedge funds
11 15 (1.23%) Fixed Income
12 13 (1.06%) Linux Command Line
13 11 (0.90%) Credit Risk
13 11 (0.90%) Market Risk
14 9 (0.74%) Replication
15 8 (0.66%) Lidar
15 8 (0.66%) Life Science
16 7 (0.57%) Smart City
Operating Systems
1 226 (18.51%) Linux
2 90 (7.37%) Unix
3 66 (5.41%) Windows
4 34 (2.78%) Red Hat Enterprise Linux
5 22 (1.80%) CentOS
6 12 (0.98%) Ubuntu
7 9 (0.74%) Windows Server
8 7 (0.57%) Android
8 7 (0.57%) Apple iOS
8 7 (0.57%) VMS
9 5 (0.41%) Debian
9 5 (0.41%) Mac OS X
10 2 (0.16%) SUSE Linux Enterprise Server
11 1 (0.082%) Oracle Linux
11 1 (0.082%) Windows Server 2012
Processes & Methodologies
1 454 (37.18%) Agile Software Development
2 348 (28.50%) Data Science
3 301 (24.65%) Analytics
4 286 (23.42%) DevOps
5 259 (21.21%) ETL
6 248 (20.31%) Machine Learning
7 221 (18.10%) Data Engineering
8 193 (15.81%) Data Modelling
9 167 (13.68%) Computer Science
10 142 (11.63%) Scrum
11 134 (10.97%) Continuous Integration
12 127 (10.40%) TDD
13 119 (9.75%) Migration
13 119 (9.75%) Software Engineering
14 118 (9.66%) Artificial Intelligence
15 108 (8.85%) Microservices
15 108 (8.85%) Problem-Solving
16 93 (7.62%) Business Intelligence
17 89 (7.29%) Visualisation
18 83 (6.80%) Data Analysis
Programming Languages
1 781 (63.96%) Python
2 516 (42.26%) SQL
3 407 (33.33%) Java
4 387 (31.70%) Scala
5 119 (9.75%) R
6 73 (5.98%) JavaScript
7 39 (3.19%) C#
8 38 (3.11%) Bash Shell
9 34 (2.78%) Go
10 33 (2.70%) C++
11 26 (2.13%) Ruby
12 23 (1.88%) Perl
12 23 (1.88%) Shell Script
13 20 (1.64%) Clojure
13 20 (1.64%) Java 8
14 19 (1.56%) Kotlin
15 18 (1.47%) C
15 18 (1.47%) T-SQL
16 17 (1.39%) Groovy
17 15 (1.23%) PL/SQL
Qualifications
1 123 (10.07%) Degree
2 48 (3.93%) Security Cleared
3 47 (3.85%) Computer Science Degree
3 47 (3.85%) PhD
4 34 (2.78%) SC Cleared
5 30 (2.46%) Master's Degree
6 28 (2.29%) AWS Certification
7 17 (1.39%) PMI Certification
7 17 (1.39%) PMP
8 14 (1.15%) Numerate Degree
9 8 (0.66%) BPSS Clearance
10 6 (0.49%) DBS Check
10 6 (0.49%) DV Cleared
11 4 (0.33%) CCIE
11 4 (0.33%) CCNP
11 4 (0.33%) Cisco Certification
12 3 (0.25%) ISTQB
12 3 (0.25%) TOGAF Certification
13 2 (0.16%) HNC
13 2 (0.16%) ITIL Certification
Quality Assurance & Compliance
1 32 (2.62%) Data Quality
2 19 (1.56%) QA
3 13 (1.06%) GDPR
4 7 (0.57%) MiFID
5 6 (0.49%) EMIR
6 4 (0.33%) Disclosure Scotland
7 3 (0.25%) PCI DSS
8 1 (0.082%) ISO/IEC 27001
8 1 (0.082%) WCAG
System Software
1 249 (20.39%) Docker
2 115 (9.42%) HDFS
3 36 (2.95%) Apache Flume
4 16 (1.31%) Apache ZooKeeper
5 12 (0.98%) Active Directory
6 9 (0.74%) VirtualBox
7 7 (0.57%) Virtual Desktop
7 7 (0.57%) Virtual Servers
8 3 (0.25%) VMware Infrastructure
9 1 (0.082%) Virtual Machines
Systems Management
1 188 (15.40%) Kubernetes
2 110 (9.01%) Ansible
3 98 (8.03%) Terraform
4 77 (6.31%) Hadoop YARN
5 59 (4.83%) Oozie
6 47 (3.85%) Puppet
7 28 (2.29%) Opscode Chef
8 13 (1.06%) Kibana
9 11 (0.90%) logstash
10 10 (0.82%) Kubeflow
11 9 (0.74%) Docker Swarm
12 6 (0.49%) Grafana
12 6 (0.49%) Salt
13 5 (0.41%) Ambari
13 5 (0.41%) Mesos
13 5 (0.41%) Prometheus
14 4 (0.33%) Cobbler
14 4 (0.33%) FortiGate
14 4 (0.33%) Graphite
14 4 (0.33%) Red Hat Satellite
Vendors
1 165 (13.51%) Google
2 162 (13.27%) Cloudera
3 131 (10.73%) Oracle
4 107 (8.76%) Databricks
5 81 (6.63%) Informatica
6 78 (6.39%) Tableau
7 77 (6.31%) Hortonworks
7 77 (6.31%) Microsoft
8 55 (4.50%) Talend
9 44 (3.60%) SAP
10 39 (3.19%) IBM
11 36 (2.95%) Red Hat
12 28 (2.29%) IBM Sterling
13 26 (2.13%) Splunk
14 25 (2.05%) Teradata
15 17 (1.39%) MapR
15 17 (1.39%) Palantir
15 17 (1.39%) StreamSets
16 14 (1.15%) MicroStrategy
17 13 (1.06%) Sybase