Apache Spark Contracts

Apache Spark
UK

The following table provides summary statistics for contract job vacancies with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 27 April 2024 with a comparison to the same period in the previous 2 years.

6 months to
27 Apr 2024
Same period 2023 Same period 2022
Rank 391 357 233
Rank change year-on-year -34 -124 -53
Contract jobs citing Apache Spark 213 411 1,141
As % of all contract jobs advertised in the UK 0.49% 0.69% 1.28%
As % of the Application Platforms category 5.14% 6.25% 10.28%
Number of daily rates quoted 152 266 742
10th Percentile £376 £475 £450
25th Percentile £436 £513 £519
Median daily rate (50th Percentile) £510 £600 £600
Median % change year-on-year -15.00% - +6.67%
75th Percentile £646 £729 £702
90th Percentile £775 £788 £813
UK excluding London median daily rate £463 £550 £550
% change year-on-year -15.91% - +8.37%
Number of hourly rates quoted 3 5 3
10th Percentile £22.93 - £62.35
25th Percentile £24.74 £70.50 £68.50
Median hourly rate £30.50 £78.00 £78.75
Median % change year-on-year -60.90% -0.95% +86.39%
75th Percentile £51.00 - £84.83
90th Percentile £61.65 - £88.47
UK excluding London median hourly rate £30.50 £78.00 -
% change year-on-year -60.90% - -

All Application Platform Skills
UK

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies with a requirement for application platform skills.

Contract vacancies with a requirement for application platform skills 4,145 6,581 11,101
As % of all contract IT jobs advertised in the UK 9.57% 11.00% 12.50%
Number of daily rates quoted 2,849 4,620 7,600
10th Percentile £250 £263 £275
25th Percentile £363 £400 £406
Median daily rate (50th Percentile) £500 £510 £520
Median % change year-on-year -1.96% -1.92% +9.47%
75th Percentile £600 £638 £625
90th Percentile £750 £750 £713
UK excluding London median daily rate £450 £475 £450
% change year-on-year -5.26% +5.56% +5.88%
Number of hourly rates quoted 161 150 212
10th Percentile £13.00 £15.50 £14.25
25th Percentile £15.42 £18.50 £16.25
Median hourly rate £27.52 £42.50 £22.00
Median % change year-on-year -35.26% +93.18% -2.22%
75th Percentile £46.18 £66.00 £48.75
90th Percentile £68.00 £79.19 £65.54
UK excluding London median hourly rate £23.75 £26.24 £18.75
% change year-on-year -9.49% +39.95% -6.25%

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a proportion of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Contractor Daily Rate Trend

3-month moving average daily rate quoted in jobs citing Apache Spark.

Daily rate trend for Apache Spark in the UK

Apache Spark
Daily Rate Histogram

Daily rate distribution for jobs citing Apache Spark over the 6 months to 27 April 2024.

Daily rate histogram for Apache Spark in the UK

Apache Spark
Contractor Hourly Rate Trend

3-month moving average hourly rates quoted in jobs citing Apache Spark.

Hourly rate trend for Apache Spark in the UK

Apache Spark
Hourly Rate Histogram

Hourly rate distribution of jobs citing Apache Spark over the 6 months to 27 April 2024.

Hourly rate histogram for Apache Spark in the UK

Apache Spark
Top 16 Contract Locations

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the UK over the 6 months to 27 April 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Jobs
England -34 174 £525 -12.50% 124
London -44 124 £550 -8.33% 77
Work from Home +30 68 £506 -20.59% 102
UK excluding London +62 63 £463 -15.91% 56
North of England +42 20 £510 +7.37% 13
North West +23 14 £510 +7.37% 7
Scotland +44 12 £400 - 12
Midlands +30 12 £425 -32.00% 6
East Midlands - 8 £438 - 2
East of England +16 7 £355 -38.26% 9
South East +13 6 £375 -42.31% 6
Yorkshire +21 5 £425 -19.05% 3
Northern Ireland +8 5 £494 - 2
West Midlands +29 4 £420 -32.80% 4
South West +4 1 £533 +2.90% 9
North East - 1 - - 3

Apache Spark
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 60 (28.17%) Apache
2 30 (14.08%) Apache Airflow
3 2 (0.94%) SAS
4 1 (0.47%) Apache Pulsar
4 1 (0.47%) Luigi
Applications
1 5 (2.35%) Microsoft Excel
2 2 (0.94%) Microsoft Office
2 2 (0.94%) Microsoft PowerPoint
Business Applications
1 2 (0.94%) Dynamics CRM
2 1 (0.47%) Aladdin
2 1 (0.47%) SAP GRC
Cloud Services
1 93 (43.66%) Azure
2 87 (40.85%) AWS
3 36 (16.90%) GCP
4 33 (15.49%) Amazon EMR
5 29 (13.62%) Serverless
6 28 (13.15%) Azure Data Factory
7 26 (12.21%) Amazon S3
8 25 (11.74%) AWS Lambda
9 21 (9.86%) Power Platform
10 20 (9.39%) Azure Synapse Analytics
10 20 (9.39%) Snowflake
11 17 (7.98%) Amazon EC2
12 12 (5.63%) Amazon Kinesis
13 10 (4.69%) Amazon SQS
14 9 (4.23%) Entra ID
15 8 (3.76%) GitHub
16 7 (3.29%) Azure Monitor
17 6 (2.82%) Azure Cosmos DB
17 6 (2.82%) Azure Logic Apps
18 5 (2.35%) Azure DevOps
Communications & Networking
1 5 (2.35%) FTP
Database & Business Intelligence
1 118 (55.40%) Big Data
2 63 (29.58%) Hadoop
3 45 (21.13%) Data Warehouse
4 44 (20.66%) Data Lake
5 40 (18.78%) Apache Hive
6 35 (16.43%) NoSQL
7 24 (11.27%) Amazon Redshift
8 22 (10.33%) Relational Database
9 21 (9.86%) Impala
9 21 (9.86%) Power BI
10 18 (8.45%) SQL Server
10 18 (8.45%) Tableau
11 16 (7.51%) Spark SQL
12 15 (7.04%) Delta Lake
13 13 (6.10%) HBase
14 10 (4.69%) BigQuery
14 10 (4.69%) Flink
15 9 (4.23%) Apache Cassandra
16 8 (3.76%) Sqoop
17 7 (3.29%) Azure SQL Database
Development Applications
1 31 (14.55%) Git
2 23 (10.80%) JIRA
3 15 (7.04%) Jenkins
4 12 (5.63%) Maven
5 8 (3.76%) Subversion
6 6 (2.82%) GitLab
7 5 (2.35%) TeamCity
8 4 (1.88%) Gradle
8 4 (1.88%) JUnit
9 3 (1.41%) Cypress.io
9 3 (1.41%) Deequ
10 2 (0.94%) Android Studio
10 2 (0.94%) Ant
10 2 (0.94%) Jaeger
10 2 (0.94%) Liquibase
10 2 (0.94%) Mockito
10 2 (0.94%) ScalaTest
10 2 (0.94%) Visual Studio Code
11 1 (0.47%) RStudio
11 1 (0.47%) Terragrunt
General
1 61 (28.64%) Social Skills
2 54 (25.35%) Analytical Skills
3 48 (22.54%) Finance
4 21 (9.86%) Banking
5 11 (5.16%) Investment Banking
6 10 (4.69%) Marketing
7 8 (3.76%) Public Sector
8 6 (2.82%) Pharmaceutical
8 6 (2.82%) Presentation Skills
9 5 (2.35%) Manufacturing
10 4 (1.88%) Inclusion and Diversity
11 3 (1.41%) Legal
12 2 (0.94%) Czech Language
12 2 (0.94%) Telecoms
13 1 (0.47%) Back Office
13 1 (0.47%) Corporate Banking
13 1 (0.47%) Games
13 1 (0.47%) Law
13 1 (0.47%) Publishing
13 1 (0.47%) Retail
Job Titles
1 78 (36.62%) Data Engineer
2 55 (25.82%) Developer
3 28 (13.15%) Senior
4 27 (12.68%) Architect
5 18 (8.45%) Azure Engineer
5 18 (8.45%) Java Developer
6 15 (7.04%) Azure Data Engineer
7 14 (6.57%) Solutions Architect
8 13 (6.10%) Lead
8 13 (6.10%) Senior Data Engineer
9 12 (5.63%) Scala Developer
10 11 (5.16%) Cloud Engineer
10 11 (5.16%) Data Architect
11 9 (4.23%) Python Developer
12 8 (3.76%) AWS Architect
12 8 (3.76%) AWS Solution Architect
12 8 (3.76%) Lead Data Engineer
13 7 (3.29%) Data Analyst
13 7 (3.29%) Data Scientist
13 7 (3.29%) Software Engineer
Libraries, Frameworks & Software Standards
1 77 (36.15%) Kafka
2 42 (19.72%) PySpark
3 12 (5.63%) JSON
4 11 (5.16%) Pandas
5 10 (4.69%) Elastic Stack
6 9 (4.23%) Spring Boot
7 8 (3.76%) SAML
7 8 (3.76%) Spring
7 8 (3.76%) XML
8 7 (3.29%) Apache NiFi
8 7 (3.29%) RESTful
9 6 (2.82%) Apache Avro
9 6 (2.82%) Spark Streaming
10 5 (2.35%) OpenTracing
11 4 (1.88%) JSON Schema
11 4 (1.88%) REST
11 4 (1.88%) TensorFlow
11 4 (1.88%) Web Services
12 3 (1.41%) .NET
12 3 (1.41%) scikit-learn
Miscellaneous
1 16 (7.51%) Algorithms
2 13 (6.10%) Data Structures
3 9 (4.23%) Distributed Systems
4 7 (3.29%) Cloud Native
4 7 (3.29%) Multi-touch
5 5 (2.35%) Distributed Computing
5 5 (2.35%) Hedge funds
6 4 (1.88%) Self-Motivation
7 3 (1.41%) Data Protection Act
7 3 (1.41%) Onboarding
8 2 (0.94%) Cyber Threat
8 2 (0.94%) Data Fabric
8 2 (0.94%) Derivative
8 2 (0.94%) Life Science
8 2 (0.94%) Market Risk
9 1 (0.47%) Analytical Mindset
9 1 (0.47%) Enterprise Software
9 1 (0.47%) Greenfield Project
9 1 (0.47%) Mobile App
9 1 (0.47%) Public Cloud
Operating Systems
1 19 (8.92%) Linux
2 16 (7.51%) Unix
3 2 (0.94%) Android
3 2 (0.94%) Red Hat Enterprise Linux
Processes & Methodologies
1 86 (40.38%) ETL
2 81 (38.03%) Data Pipeline
3 67 (31.46%) Agile
4 64 (30.05%) CI/CD
5 53 (24.88%) Analytics
6 49 (23.00%) Data Engineering
7 45 (21.13%) Data Modelling
8 42 (19.72%) Problem-Solving
9 38 (17.84%) Data Science
10 35 (16.43%) DevOps
11 34 (15.96%) Machine Learning
12 32 (15.02%) Business Intelligence
13 30 (14.08%) Continuous Integration
13 30 (14.08%) Software Engineering
14 24 (11.27%) Data Analysis
14 24 (11.27%) Data Integration
14 24 (11.27%) SDLC
15 23 (10.80%) Test Automation
16 22 (10.33%) Computer Science
16 22 (10.33%) Scrum
Programming Languages
1 140 (65.73%) Python
2 119 (55.87%) SQL
3 81 (38.03%) Java
3 81 (38.03%) Scala
4 15 (7.04%) R
5 12 (5.63%) C#
6 9 (4.23%) Shell Script
7 8 (3.76%) Perl
8 6 (2.82%) Bash
8 6 (2.82%) C++
8 6 (2.82%) Kotlin
9 5 (2.35%) T-SQL
10 3 (1.41%) JavaScript
10 3 (1.41%) PL/SQL
11 2 (0.94%) C
11 2 (0.94%) Groovy
11 2 (0.94%) Haskell
11 2 (0.94%) U-SQL
12 1 (0.47%) ANSI SQL
12 1 (0.47%) Gremlin
Qualifications
1 22 (10.33%) Degree
2 20 (9.39%) Security Cleared
3 16 (7.51%) SC Cleared
4 11 (5.16%) Computer Science Degree
5 4 (1.88%) BPSS Clearance
5 4 (1.88%) Master's Degree
5 4 (1.88%) Numerate Degree
6 3 (1.41%) AWS Certification
6 3 (1.41%) Azure Certification
6 3 (1.41%) DV Cleared
6 3 (1.41%) PhD
7 2 (0.94%) AWS Certified Solutions Architect
8 1 (0.47%) Microsoft Certification
8 1 (0.47%) OCP
8 1 (0.47%) Oracle Certification
Quality Assurance & Compliance
1 29 (13.62%) Data Quality
2 4 (1.88%) GDPR
2 4 (1.88%) QA
3 2 (0.94%) Accessibility
3 2 (0.94%) California Consumer Privacy Act
4 1 (0.47%) GRC
System Software
1 22 (10.33%) Docker
2 15 (7.04%) HDFS
3 1 (0.47%) Apache Flume
3 1 (0.47%) Apache ZooKeeper
3 1 (0.47%) Ceph
3 1 (0.47%) NFS
Systems Management
1 29 (13.62%) Kubernetes
2 16 (7.51%) Terraform
3 9 (4.23%) Hadoop YARN
4 7 (3.29%) Prometheus
5 5 (2.35%) Zipkin
6 4 (1.88%) Ansible
7 3 (1.41%) Grafana
7 3 (1.41%) Oozie
8 2 (0.94%) Ambari
8 2 (0.94%) Puppet
9 1 (0.47%) Docker Swarm
9 1 (0.47%) Ganglia
9 1 (0.47%) Kibana
9 1 (0.47%) Mesos
9 1 (0.47%) Podman
9 1 (0.47%) Progress Chef
9 1 (0.47%) RSA Archer
Vendors
1 50 (23.47%) Databricks
2 25 (11.74%) Microsoft
3 21 (9.86%) Talend
4 17 (7.98%) Informatica
5 12 (5.63%) Google
6 11 (5.16%) Cloudera
6 11 (5.16%) Oracle
7 7 (3.29%) Teradata
8 5 (2.35%) Splunk
9 4 (1.88%) SAP
10 3 (1.41%) Ab Initio
11 2 (0.94%) AppDynamics
11 2 (0.94%) Bloomberg
11 2 (0.94%) CyberArk
12 1 (0.47%) Ariba
12 1 (0.47%) Confluent
12 1 (0.47%) Okta
12 1 (0.47%) OneTrust
12 1 (0.47%) Symantec
12 1 (0.47%) Varonis