Period
to 5 December 2020

The following table provides summary statistics for contract job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 5 December 2020 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
UK > England
6 months to
5 Dec 2020
Same period 2019 Same period 2018
Rank 203 202 193
Rank change year-on-year -1 -9 -16
Contract jobs citing Apache Spark 418 995 1,389
As % of all contract jobs advertised in England 1.32% 1.33% 1.45%
As % of the Application Platforms category 9.06% 9.06% 8.64%
Number of daily rates quoted 305 744 987
Median daily rate £555 £575 £550
Median daily rate % change year-on-year -3.48% +4.55% -
10th Percentile £400 £425 £425
90th Percentile £713 £725 £713
UK median daily rate £550 £575 £550
% change year-on-year -4.35% +4.55% -
Number of hourly rates quoted 8 4 3
Median hourly rate £22.44 £29.83 £45.00
Median hourly rate % change year-on-year -24.76% -33.72% +92.14%
10th Percentile £9.75 £22.27 £42.50
90th Percentile £47.75 £48.13 £150.90
UK median hourly rate £22.44 £29.83 £45.00
% change year-on-year -24.76% -33.72% +92.14%

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Contract vacancies with a requirement for application platform skills 4,614 10,982 16,078
As % of all contract IT jobs advertised in England 14.58% 14.71% 16.80%
Number of daily rates quoted 2,880 7,158 10,486
Median daily rate £463 £450 £475
Median daily rate % change year-on-year +2.78% -5.26% +5.56%
10th Percentile £275 £254 £280
90th Percentile £650 £638 £638
UK median daily rate £463 £450 £456
% change year-on-year +2.78% -1.25% +1.27%
Number of hourly rates quoted 189 260 372
Median hourly rate £35.00 £20.51 £21.50
Median hourly rate % change year-on-year +70.65% -4.60% +7.50%
10th Percentile £13.23 £11.00 £12.03
90th Percentile £60.40 £53.75 £47.50
Median hourly rate £35.00 £21.50 £21.50
% change year-on-year +62.79% - +7.50%

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a proportion of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Contractor Daily Rate Trend in England

3-month moving average daily rate quoted in jobs citing Apache Spark in England.

Daily rate trend for Apache Spark in England

Apache Spark
Daily Rate Histogram in England

Daily rate distribution for jobs citing Apache Spark in England over the 6 months to 5 December 2020.

Daily rate histogram for Apache Spark in England

Apache Spark
Contractor Hourly Rate Trend in England

3-month moving average hourly rates quoted in jobs citing Apache Spark in England.

Hourly rate trend for Apache Spark in England

Apache Spark
Hourly Rate Histogram in England

Hourly rate distribution of jobs citing Apache Spark in England over the 6 months to 5 December 2020.

Hourly rate histogram for Apache Spark in England

Apache Spark
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the England region over the 6 months to 5 December 2020. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London +6 332 £575 - 79
South East +74 34 £475 -9.52% 11
North of England +39 19 £401 -27.18% 24
East of England +66 15 £438 -16.67% 2
North West +56 14 £345 -37.27% 8
South West +85 5 £575 -4.17% 10
Yorkshire +24 5 £513 -6.82% 16
Midlands +40 3 £400 -23.81% 3
East Midlands +35 2 - - 1
West Midlands +44 1 £400 -23.81% 2
Apache Spark
UK

For the 6 months to 5 December 2020, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region with a requirement for Apache Spark.

1 257 (61.48%) Python
2 236 (56.46%) Big Data
3 223 (53.35%) Hadoop
4 171 (40.91%) Java
5 164 (39.23%) SQL
6 151 (36.12%) Scala
7 142 (33.97%) AWS
8 118 (28.23%) Apache
9 117 (27.99%) Apache Hive
10 114 (27.27%) Kafka
10 114 (27.27%) Finance
11 106 (25.36%) Agile Software Development
12 89 (21.29%) Data Science
13 87 (20.81%) Kubernetes
14 86 (20.57%) Azure
15 84 (20.10%) Docker
16 83 (19.86%) ETL
17 80 (19.14%) Analytics
18 74 (17.70%) Machine Learning
19 73 (17.46%) Front Office
20 69 (16.51%) Security Cleared
21 67 (16.03%) Banking
22 64 (15.31%) DevOps
23 62 (14.83%) Data Modelling
24 60 (14.35%) Software Engineering
24 60 (14.35%) Degree
24 60 (14.35%) HDFS
25 59 (14.11%) Data Engineering
26 58 (13.88%) NoSQL
27 55 (13.16%) Terraform

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 118 (28.23%) Apache
2 31 (7.42%) Elasticsearch
3 22 (5.26%) Apache Pig
4 20 (4.78%) Apache Airflow
5 14 (3.35%) Confluence
6 8 (1.91%) SAS
7 7 (1.67%) Jupyter
8 4 (0.96%) ArcGIS
8 4 (0.96%) Tornado
9 3 (0.72%) Luigi
10 2 (0.48%) Apache Synapse
10 2 (0.48%) Oracle Coherence
11 1 (0.24%) Apache Solr
11 1 (0.24%) Camunda
11 1 (0.24%) nginx
11 1 (0.24%) Oracle Workflow
11 1 (0.24%) R Markdown
Applications
1 3 (0.72%) Microsoft Excel
2 2 (0.48%) Chatbot
3 1 (0.24%) Microsoft PowerPoint
Business Applications
1 4 (0.96%) SAP Oil and Gas
2 1 (0.24%) Oracle BPM
2 1 (0.24%) Sparx Enterprise Architect
Cloud Services
1 142 (33.97%) AWS
2 86 (20.57%) Azure
3 47 (11.24%) GCP
4 27 (6.46%) Amazon S3
5 26 (6.22%) Amazon EMR
6 23 (5.50%) Azure Data Factory
7 21 (5.02%) Serverless
8 15 (3.59%) Power Platform
9 14 (3.35%) Amazon EC2
10 12 (2.87%) Virtual Private Cloud
11 8 (1.91%) Amazon EKS
11 8 (1.91%) Azure Functions
12 7 (1.67%) Amazon SageMaker
12 7 (1.67%) Cloud Computing
12 7 (1.67%) GitHub
13 6 (1.44%) AWS Lambda
14 5 (1.20%) Azure Active Directory
14 5 (1.20%) SaaS
15 4 (0.96%) AWS CloudFormation
15 4 (0.96%) PaaS
Communications & Networking
1 5 (1.20%) SSL
2 4 (0.96%) Kerberos
2 4 (0.96%) VPN
3 3 (0.72%) Firewall
3 3 (0.72%) HTTP
4 2 (0.48%) TCP/IP
5 1 (0.24%) DNS
5 1 (0.24%) Internet
5 1 (0.24%) Skype
5 1 (0.24%) Software-Defined Networking
5 1 (0.24%) Wi-Fi
5 1 (0.24%) Wireless
Database & Business Intelligence
1 236 (56.46%) Big Data
2 223 (53.35%) Hadoop
3 117 (27.99%) Apache Hive
4 58 (13.88%) NoSQL
5 44 (10.53%) Impala
6 41 (9.81%) Data Warehouse
7 36 (8.61%) Apache Ignite
8 32 (7.66%) Data Lake
9 30 (7.18%) PostgreSQL
10 27 (6.46%) Tableau
11 26 (6.22%) HBase
12 21 (5.02%) Relational Database
13 18 (4.31%) Spark SQL
14 17 (4.07%) Apache Cassandra
15 16 (3.83%) Amazon RDS
15 16 (3.83%) Data Mining
16 15 (3.59%) Power BI
16 15 (3.59%) SQL Server
17 14 (3.35%) Azure SQL Database
17 14 (3.35%) BigQuery
Development Applications
1 52 (12.44%) Git (software)
2 44 (10.53%) Jenkins
3 20 (4.78%) JIRA
4 14 (3.35%) TeamCity
5 10 (2.39%) Selenium
6 8 (1.91%) Cucumber
7 7 (1.67%) GitLab
8 6 (1.44%) ProC
9 5 (1.20%) Atlassian Bamboo
10 4 (0.96%) Hudson
10 4 (0.96%) Maven
10 4 (0.96%) MLflow
11 3 (0.72%) Gradle
11 3 (0.72%) kappa
11 3 (0.72%) SonarQube
12 2 (0.48%) ERwin
13 1 (0.24%) Bitbucket
13 1 (0.24%) CircleCI
13 1 (0.24%) JUnit
13 1 (0.24%) Sonatype Nexus
General
1 114 (27.27%) Finance
2 73 (17.46%) Front Office
3 67 (16.03%) Banking
4 25 (5.98%) Investment Banking
5 17 (4.07%) Public Sector
6 14 (3.35%) Marketing
7 13 (3.11%) Retail
8 8 (1.91%) Manufacturing
8 8 (1.91%) Pharmaceutical
9 7 (1.67%) Electronics
10 5 (1.20%) Aerospace
10 5 (1.20%) Telecoms
11 4 (0.96%) Legal
12 3 (0.72%) Financial Institution
12 3 (0.72%) Retail Banking
13 2 (0.48%) Games
13 2 (0.48%) Law
13 2 (0.48%) Military
14 1 (0.24%) Automotive
14 1 (0.24%) Billing
Job Titles
1 140 (33.49%) Developer
2 138 (33.01%) Data Engineer
3 52 (12.44%) Python Engineer
4 43 (10.29%) Architect
5 42 (10.05%) Big Data Engineer
6 40 (9.57%) Software Engineer
7 36 (8.61%) Developer/Software Engineer
8 35 (8.37%) Python Developer
9 33 (7.89%) Java Developer
10 32 (7.66%) Data Scientist
11 25 (5.98%) Data Architect
11 25 (5.98%) Scala Developer
12 24 (5.74%) AWS Engineer
13 23 (5.50%) Senior Data Engineer
13 23 (5.50%) Senior Developer
13 23 (5.50%) Solutions Architect
14 18 (4.31%) Python Software Engineer
14 18 (4.31%) Software Developer
15 15 (3.59%) Big Data Developer
16 14 (3.35%) Azure Engineer
Libraries, Frameworks & Software Standards
1 114 (27.27%) Kafka
2 46 (11.00%) Web Services
3 44 (10.53%) PySpark
4 40 (9.57%) React
5 39 (9.33%) Node.js
6 36 (8.61%) gRPC
6 36 (8.61%) Protocol Buffers
7 19 (4.55%) JSON
7 19 (4.55%) TensorFlow
8 17 (4.07%) Spark Streaming
9 15 (3.59%) Pandas
9 15 (3.59%) PyTorch
10 14 (3.35%) Akka
10 14 (3.35%) Apache NiFi
10 14 (3.35%) Elastic Stack
11 13 (3.11%) scikit-learn
12 11 (2.63%) NumPy
12 11 (2.63%) Spring
12 11 (2.63%) XML
13 10 (2.39%) .NET
Miscellaneous
1 46 (11.00%) Analytical Skills
2 39 (9.33%) Algorithms
2 39 (9.33%) Fixed Income
3 38 (9.09%) Equities
4 36 (8.61%) Derivative
5 15 (3.59%) Data Structures
6 13 (3.11%) Foreign Exchange (FX)
6 13 (3.11%) Greenfield Project
7 12 (2.87%) Analytical Mindset
8 10 (2.39%) Cloud Native
8 10 (2.39%) Credit Risk
8 10 (2.39%) FMCG
9 7 (1.67%) Public Cloud
10 6 (1.44%) Life Science
10 6 (1.44%) Market Risk
10 6 (1.44%) User Experience
10 6 (1.44%) Virtual Team
11 5 (1.20%) Management Information System
12 3 (0.72%) Linked Data
12 3 (0.72%) Reference Data
Operating Systems
1 31 (7.42%) Linux
2 8 (1.91%) Unix
3 6 (1.44%) Ubuntu
4 5 (1.20%) Windows
5 4 (0.96%) Debian
5 4 (0.96%) Red Hat Enterprise Linux
Processes & Methodologies
1 106 (25.36%) Agile Software Development
2 89 (21.29%) Data Science
3 83 (19.86%) ETL
4 80 (19.14%) Analytics
5 74 (17.70%) Machine Learning
6 64 (15.31%) DevOps
7 62 (14.83%) Data Modelling
8 60 (14.35%) Software Engineering
9 59 (14.11%) Data Engineering
10 52 (12.44%) Visualisation
11 50 (11.96%) Computer Science
12 42 (10.05%) Mathematics
13 41 (9.81%) Asset Management
14 39 (9.33%) Investment Management
14 39 (9.33%) Scrum
15 37 (8.85%) ETL Development
15 37 (8.85%) TDD
16 36 (8.61%) Performance Attribution
17 35 (8.37%) Business Intelligence
17 35 (8.37%) Microservices
Programming Languages
1 257 (61.48%) Python
2 171 (40.91%) Java
3 164 (39.23%) SQL
4 151 (36.12%) Scala
5 53 (12.68%) JavaScript
6 46 (11.00%) Kotlin
7 40 (9.57%) C#
8 35 (8.37%) R
9 23 (5.50%) C++
10 15 (3.59%) Shell Script
11 14 (3.35%) Haskell
12 9 (2.15%) Perl
13 7 (1.67%) Bash Shell
13 7 (1.67%) Java 8
14 6 (1.44%) PHP
14 6 (1.44%) T-SQL
15 4 (0.96%) Groovy
15 4 (0.96%) PowerShell
15 4 (0.96%) Ruby
15 4 (0.96%) U-SQL
Qualifications
1 69 (16.51%) Security Cleared
2 60 (14.35%) Degree
3 54 (12.92%) SC Cleared
4 30 (7.18%) Numerate Degree
5 19 (4.55%) Computer Science Degree
6 12 (2.87%) TOGAF Certification
7 11 (2.63%) Master's Degree
8 10 (2.39%) AWS Certification
9 8 (1.91%) DV Cleared
9 8 (1.91%) PhD
10 1 (0.24%) CSPO
Quality Assurance & Compliance
1 11 (2.63%) QA
2 8 (1.91%) Data Quality
3 7 (1.67%) MiFID
4 2 (0.48%) Sarbanes-Oxley
4 2 (0.48%) SLA
5 1 (0.24%) CDISC
5 1 (0.24%) GDPR
5 1 (0.24%) GxP
5 1 (0.24%) ISO 20022
5 1 (0.24%) PCI DSS
5 1 (0.24%) SEPA
5 1 (0.24%) TARGET2
System Software
1 84 (20.10%) Docker
2 60 (14.35%) HDFS
3 15 (3.59%) Apache ZooKeeper
4 5 (1.20%) Active Directory
4 5 (1.20%) Apache Flume
5 4 (0.96%) Active Directory Lightweight Directory Services
6 1 (0.24%) Virtual Machines
Systems Management
1 87 (20.81%) Kubernetes
2 55 (13.16%) Terraform
3 29 (6.94%) Ansible
4 22 (5.26%) Hadoop YARN
5 14 (3.35%) Oozie
6 7 (1.67%) Puppet
7 5 (1.20%) Opscode Chef
8 2 (0.48%) Knative
8 2 (0.48%) Kubeflow
8 2 (0.48%) Mesos
9 1 (0.24%) Ambari
9 1 (0.24%) Control-M
9 1 (0.24%) Fluentd
9 1 (0.24%) Grafana
9 1 (0.24%) Istio
9 1 (0.24%) Kibana
9 1 (0.24%) logstash
9 1 (0.24%) vRealize Orchestrator
Vendors
1 43 (10.29%) Databricks
2 41 (9.81%) Cloudera
3 36 (8.61%) Bloomberg
4 32 (7.66%) Oracle
5 31 (7.42%) Google
6 18 (4.31%) Microsoft
7 16 (3.83%) Hortonworks
8 15 (3.59%) Teradata
9 13 (3.11%) Informatica
10 12 (2.87%) Ab Initio
11 9 (2.15%) SAP
12 7 (1.67%) MuleSoft
12 7 (1.67%) Red Hat
12 7 (1.67%) TIBCO
13 6 (1.44%) Alteryx
13 6 (1.44%) Atlassian
13 6 (1.44%) Splunk
14 5 (1.20%) Sage
14 5 (1.20%) Salesforce.com
14 5 (1.20%) Talend