Period
to 28 February 2021

The following table provides summary statistics for contract job vacancies advertised in England with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 28 February 2021 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
UK > England
6 months to
28 Feb 2021
Same period 2020 Same period 2019
Rank 180 205 180
Rank change year-on-year +25 -25 -6
Contract jobs citing Apache Spark 556 798 1,427
As % of all contract jobs advertised in England 1.46% 1.31% 1.58%
As % of the Application Platforms category 10.49% 9.17% 9.67%
Number of daily rates quoted 467 615 1,050
Median daily rate £563 £557 £550
Median daily rate % change year-on-year +0.99% +1.27% -
10th Percentile £413 £413 £438
90th Percentile £688 £725 £713
UK median daily rate £555 £550 £550
% change year-on-year +0.91% - -
Number of hourly rates quoted 4 0 3
Median hourly rate £23.77 - £45.00
10th Percentile £21.59 - £42.50
90th Percentile £61.77 - £55.50
UK median hourly rate £23.77 - £45.00

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in England with a requirement for application platform skills.

All Application Platform Skills
England
Contract vacancies with a requirement for application platform skills 5,298 8,701 14,761
As % of all contract IT jobs advertised in England 13.96% 14.25% 16.30%
Number of daily rates quoted 3,635 5,846 9,784
Median daily rate £475 £463 £475
Median daily rate % change year-on-year +2.70% -2.63% +5.56%
10th Percentile £275 £263 £285
90th Percentile £650 £650 £638
UK median daily rate £475 £450 £475
% change year-on-year +5.56% -5.26% +5.56%
Number of hourly rates quoted 149 175 290
Median hourly rate £25.50 £19.00 £18.50
Median hourly rate % change year-on-year +34.21% +2.70% -1.07%
10th Percentile £13.48 £11.20 £12.00
90th Percentile £62.03 £53.00 £47.50
Median hourly rate £32.00 £19.25 £18.50
% change year-on-year +66.21% +4.07% -1.07%

Apache Spark
Job Vacancy Trend in England

Job postings citing Apache Spark as a proportion of all IT jobs advertised in England.

Job vacancy trend for Apache Spark in England

Apache Spark
Contractor Daily Rate Trend in England

3-month moving average daily rate quoted in jobs citing Apache Spark in England.

Daily rate trend for Apache Spark in England

Apache Spark
Daily Rate Histogram in England

Daily rate distribution for jobs citing Apache Spark in England over the 6 months to 28 February 2021.

Daily rate histogram for Apache Spark in England

Apache Spark
Contractor Hourly Rate Trend in England

3-month moving average hourly rates quoted in jobs citing Apache Spark in England.

Hourly rate trend for Apache Spark in England

Apache Spark
Hourly Rate Histogram in England

Hourly rate distribution of jobs citing Apache Spark in England over the 6 months to 28 February 2021.

Hourly rate histogram for Apache Spark in England

Apache Spark
Contract Job Locations in England

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the England region over the 6 months to 28 February 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London +19 414 £575 -2.75% 116
South East +32 40 £500 - 15
North of England +48 32 £550 - 19
South West +59 20 £500 -9.09% 3
North West +53 18 £485 -11.82% 9
East of England +37 17 £350 -28.21% 5
Yorkshire +30 14 £550 +3.77% 9
Midlands +25 7 £375 -28.57% 4
West Midlands +22 7 £375 -28.57% 4
Apache Spark
UK

For the 6 months to 28 February 2021, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the England region with a requirement for Apache Spark.

1 327 (58.81%) Python
2 303 (54.50%) Hadoop
3 275 (49.46%) Big Data
4 232 (41.73%) Java
5 222 (39.93%) Scala
6 216 (38.85%) SQL
7 179 (32.19%) AWS
8 170 (30.58%) Agile Software Development
9 169 (30.40%) Kafka
10 165 (29.68%) Finance
11 163 (29.32%) Apache Hive
12 159 (28.60%) Apache
13 118 (21.22%) Analytics
14 117 (21.04%) Machine Learning
15 114 (20.50%) Data Science
16 105 (18.88%) Azure
17 102 (18.35%) DevOps
17 102 (18.35%) ETL
18 96 (17.27%) HDFS
19 93 (16.73%) Banking
20 88 (15.83%) Kubernetes
21 84 (15.11%) Docker
22 83 (14.93%) Front Office
22 83 (14.93%) Data Engineering
23 81 (14.57%) Software Engineering
23 81 (14.57%) NoSQL
24 79 (14.21%) Data Warehouse
25 78 (14.03%) TDD
26 76 (13.67%) Data Lake
27 74 (13.31%) Security Cleared

Apache Spark
Co-occurring IT Skills in England by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 159 (28.60%) Apache
2 52 (9.35%) Apache Airflow
3 33 (5.94%) Confluence
4 32 (5.76%) Elasticsearch
5 19 (3.42%) Apache Pig
6 8 (1.44%) Jupyter
7 5 (0.90%) Apache Solr
7 5 (0.90%) Apache Synapse
7 5 (0.90%) Oracle Coherence
8 4 (0.72%) ArcGIS
8 4 (0.72%) Luigi
9 2 (0.36%) Oracle Workflow
10 1 (0.18%) Camunda
10 1 (0.18%) nginx
10 1 (0.18%) SAS
10 1 (0.18%) Tornado
Applications
1 3 (0.54%) Microsoft Excel
2 2 (0.36%) Chatbot
3 1 (0.18%) Microsoft Office
3 1 (0.18%) Microsoft PowerPoint
3 1 (0.18%) MS Visio
3 1 (0.18%) Stata
Business Applications
1 1 (0.18%) Oracle BPM
1 1 (0.18%) SAP CRM
1 1 (0.18%) SAP ERP
1 1 (0.18%) SAP IS-U
Cloud Services
1 179 (32.19%) AWS
2 105 (18.88%) Azure
3 52 (9.35%) Amazon S3
4 50 (8.99%) GCP
5 42 (7.55%) Amazon EMR
6 30 (5.40%) Serverless
7 26 (4.68%) Amazon EC2
8 25 (4.50%) Azure Data Factory
9 22 (3.96%) Power Platform
10 21 (3.78%) GitHub
11 20 (3.60%) Amazon Kinesis
12 17 (3.06%) AWS Lambda
13 13 (2.34%) AWS CloudFormation
14 12 (2.16%) Amazon SageMaker
15 11 (1.98%) Cloud Computing
16 8 (1.44%) Azure Functions
16 8 (1.44%) Google Analytics
17 7 (1.26%) Amazon ECS
17 7 (1.26%) AWS CodePipeline
17 7 (1.26%) Virtual Private Cloud
Communications & Networking
1 16 (2.88%) HTTP
2 5 (0.90%) Kerberos
3 4 (0.72%) DNS
3 4 (0.72%) Network Security
4 3 (0.54%) Firewall
4 3 (0.54%) VPN
5 2 (0.36%) TCP/IP
6 1 (0.18%) Internet
6 1 (0.18%) Software-Defined Networking
Database & Business Intelligence
1 303 (54.50%) Hadoop
2 275 (49.46%) Big Data
3 163 (29.32%) Apache Hive
4 81 (14.57%) NoSQL
5 79 (14.21%) Data Warehouse
6 76 (13.67%) Data Lake
7 66 (11.87%) HBase
8 58 (10.43%) Impala
9 44 (7.91%) Relational Database
10 40 (7.19%) Spark SQL
11 36 (6.47%) Apache Ignite
12 35 (6.29%) PostgreSQL
13 32 (5.76%) SQL Server
14 31 (5.58%) Amazon Redshift
14 31 (5.58%) Tableau
15 30 (5.40%) Apache Cassandra
16 27 (4.86%) Amazon Athena
17 24 (4.32%) Amazon RDS
18 23 (4.14%) MongoDB
19 22 (3.96%) Parquet
Development Applications
1 68 (12.23%) Jenkins
2 67 (12.05%) Git (software)
3 41 (7.37%) JIRA
4 23 (4.14%) Sonatype Nexus
5 21 (3.78%) Maven
6 18 (3.24%) TeamCity
7 14 (2.52%) GitLab
7 14 (2.52%) Gradle
8 12 (2.16%) Cucumber
9 11 (1.98%) IBM UrbanCode
9 11 (1.98%) RStudio
9 11 (1.98%) Selenium
10 9 (1.62%) Atlassian Bamboo
11 8 (1.44%) MLflow
12 6 (1.08%) SonarQube
13 5 (0.90%) Bitbucket
13 5 (0.90%) JUnit
13 5 (0.90%) ProC
14 4 (0.72%) IntelliJ
15 3 (0.54%) Artifactory
General
1 165 (29.68%) Finance
2 93 (16.73%) Banking
3 83 (14.93%) Front Office
4 46 (8.27%) Investment Banking
5 22 (3.96%) Public Sector
6 20 (3.60%) Marketing
7 15 (2.70%) Retail
8 8 (1.44%) Pharmaceutical
9 5 (0.90%) Financial Institution
9 5 (0.90%) Legal
10 4 (0.72%) Games
10 4 (0.72%) Manufacturing
10 4 (0.72%) Retail Banking
10 4 (0.72%) Telecoms
11 3 (0.54%) Automotive
12 2 (0.36%) Electronics
12 2 (0.36%) French Language
12 2 (0.36%) Military
13 1 (0.18%) Hebrew Language
13 1 (0.18%) Local Government
Job Titles
1 203 (36.51%) Developer
2 195 (35.07%) Data Engineer
3 57 (10.25%) Python Engineer
4 54 (9.71%) Big Data Engineer
5 51 (9.17%) Java Developer
6 50 (8.99%) Scala Developer
6 50 (8.99%) Software Engineer
7 48 (8.63%) Architect
7 48 (8.63%) AWS Engineer
8 42 (7.55%) Data Scientist
9 37 (6.65%) Developer/Software Engineer
9 37 (6.65%) Python Developer
10 32 (5.76%) Apache Spark Developer
11 31 (5.58%) Hadoop Developer
12 30 (5.40%) Senior Developer
13 26 (4.68%) Data Architect
13 26 (4.68%) Scala Engineer
14 24 (4.32%) Big Data Developer
15 22 (3.96%) Software Developer
16 20 (3.60%) Senior Data Engineer
Libraries, Frameworks & Software Standards
1 169 (30.40%) Kafka
2 66 (11.87%) Web Services
3 59 (10.61%) PySpark
4 40 (7.19%) Node.js
4 40 (7.19%) React
5 39 (7.01%) Protocol Buffers
6 36 (6.47%) gRPC
7 30 (5.40%) JSON
8 24 (4.32%) XML
9 21 (3.78%) Spark Streaming
10 20 (3.60%) NumPy
10 20 (3.60%) Spring
11 19 (3.42%) Pandas
11 19 (3.42%) PyTorch
12 18 (3.24%) TensorFlow
13 16 (2.88%) Apache NiFi
13 16 (2.88%) scikit-learn
14 15 (2.70%) RESTful
15 12 (2.16%) HTML
16 11 (1.98%) Natural Language Processing
Miscellaneous
1 74 (13.31%) Analytical Skills
2 59 (10.61%) Algorithms
3 41 (7.37%) Equities
4 37 (6.65%) Data Structures
4 37 (6.65%) Fixed Income
5 36 (6.47%) Derivative
6 15 (2.70%) Self-Motivation
7 13 (2.34%) Linux Command Line
8 11 (1.98%) Foreign Exchange (FX)
8 11 (1.98%) Greenfield Project
9 10 (1.80%) Grid Computing
10 9 (1.62%) Cloud Native
10 9 (1.62%) User Experience
10 9 (1.62%) Virtual Team
11 8 (1.44%) Analytical Mindset
11 8 (1.44%) Market Risk
11 8 (1.44%) Public Cloud
12 7 (1.26%) Credit Risk
13 6 (1.08%) Life Science
14 4 (0.72%) FMCG
Operating Systems
1 54 (9.71%) Linux
2 16 (2.88%) Unix
3 3 (0.54%) Windows
4 2 (0.36%) Ubuntu
5 1 (0.18%) Android
5 1 (0.18%) Windows Server
Processes & Methodologies
1 170 (30.58%) Agile Software Development
2 118 (21.22%) Analytics
3 117 (21.04%) Machine Learning
4 114 (20.50%) Data Science
5 102 (18.35%) DevOps
5 102 (18.35%) ETL
6 83 (14.93%) Data Engineering
7 81 (14.57%) Software Engineering
8 78 (14.03%) TDD
9 72 (12.95%) Data Modelling
10 65 (11.69%) Computer Science
11 58 (10.43%) Mathematics
12 57 (10.25%) Scrum
13 56 (10.07%) BDD
14 54 (9.71%) Visualisation
15 49 (8.81%) Business Intelligence
16 44 (7.91%) Data Ingestion
17 43 (7.73%) Artificial Intelligence
18 39 (7.01%) Data Analytics
18 39 (7.01%) Problem-Solving
Programming Languages
1 327 (58.81%) Python
2 232 (41.73%) Java
3 222 (39.93%) Scala
4 216 (38.85%) SQL
5 56 (10.07%) JavaScript
6 54 (9.71%) R
7 44 (7.91%) Kotlin
8 35 (6.29%) C#
9 27 (4.86%) C++
10 24 (4.32%) Shell Script
11 19 (3.42%) Haskell
12 11 (1.98%) Bash Shell
13 8 (1.44%) Elm
13 8 (1.44%) PL/SQL
14 7 (1.26%) Java 8
15 5 (0.90%) Groovy
15 5 (0.90%) T-SQL
15 5 (0.90%) U-SQL
16 4 (0.72%) Go
17 3 (0.54%) SPARQL
Qualifications
1 74 (13.31%) Security Cleared
2 68 (12.23%) Degree
3 62 (11.15%) SC Cleared
4 31 (5.58%) Numerate Degree
5 17 (3.06%) Computer Science Degree
6 14 (2.52%) AWS Certification
7 13 (2.34%) TOGAF Certification
8 12 (2.16%) PhD
9 8 (1.44%) Master's Degree
10 5 (0.90%) DV Cleared
11 1 (0.18%) BPSS Clearance
Quality Assurance & Compliance
1 30 (5.40%) QA
2 19 (3.42%) Data Quality
3 3 (0.54%) MiFID
4 2 (0.36%) GDPR
4 2 (0.36%) ISO 20022
4 2 (0.36%) SEPA
4 2 (0.36%) TARGET2
5 1 (0.18%) PCI DSS
5 1 (0.18%) SLA
System Software
1 96 (17.27%) HDFS
2 84 (15.11%) Docker
3 18 (3.24%) Apache ZooKeeper
4 10 (1.80%) Apache Flume
5 3 (0.54%) Active Directory Lightweight Directory Services
6 1 (0.18%) Virtual Machines
Systems Management
1 88 (15.83%) Kubernetes
2 62 (11.15%) Terraform
3 38 (6.83%) Ansible
4 34 (6.12%) Hadoop YARN
5 28 (5.04%) Oozie
6 12 (2.16%) Puppet
7 8 (1.44%) Opscode Chef
8 6 (1.08%) Kibana
9 5 (0.90%) Ambari
9 5 (0.90%) Grafana
10 3 (0.54%) Prometheus
11 2 (0.36%) Kubeflow
11 2 (0.36%) Mesos
12 1 (0.18%) Control-M
12 1 (0.18%) Knative
12 1 (0.18%) vRealize Orchestrator
Vendors
1 52 (9.35%) Oracle
2 49 (8.81%) Databricks
3 46 (8.27%) Cloudera
4 39 (7.01%) Google
5 36 (6.47%) Bloomberg
6 27 (4.86%) Microsoft
7 18 (3.24%) Hortonworks
7 18 (3.24%) Informatica
8 13 (2.34%) Talend
8 13 (2.34%) Teradata
9 10 (1.80%) Ab Initio
9 10 (1.80%) TIBCO
10 9 (1.62%) StreamSets
11 8 (1.44%) SAP
12 7 (1.26%) MapR
13 6 (1.08%) Alteryx
13 6 (1.08%) Splunk
14 5 (0.90%) Atlassian
14 5 (0.90%) Sage
15 4 (0.72%) Salesforce.com