Period
to

The following table provides summary statistics for contract job vacancies advertised in London with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 17 May 2022 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
England > London
6 months to
17 May 2022
Same period 2021 Same period 2020
Rank 176 127 139
Rank change year-on-year -49 +12 -37
Contract jobs citing Apache Spark 612 507 411
As % of all contract jobs advertised in London 1.59% 2.13% 2.00%
As % of the Application Platforms category 12.49% 15.12% 13.30%
Number of daily rates quoted 407 325 311
10th Percentile £454 £453 £413
25th Percentile £525 £513 £475
Median daily rate (50th Percentile) £615 £575 £550
Median % change year-on-year +6.96% +4.55% -1.79%
75th Percentile £744 £650 £650
90th Percentile £845 £725 £700
England median daily rate £600 £563 £550
% change year-on-year +6.67% +2.27% -
Number of hourly rates quoted 3 1 0
10th Percentile £62.35 - -
25th Percentile £68.50 £36.25 -
Median hourly rate £78.75 £37.50 -
Median % change year-on-year +110.00% - -
75th Percentile £84.83 £38.75 -
90th Percentile £88.47 - -
England median hourly rate £78.75 £37.50 -
% change year-on-year +110.00% - -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in London with a requirement for application platform skills.

All Application Platform Skills
London
Contract vacancies with a requirement for application platform skills 4,901 3,354 3,091
As % of all contract IT jobs advertised in London 12.72% 14.11% 15.05%
Number of daily rates quoted 3,452 2,397 2,133
10th Percentile £369 £325 £313
25th Percentile £463 £418 £413
Median daily rate (50th Percentile) £570 £513 £500
Median % change year-on-year +11.12% +2.50% -4.76%
75th Percentile £675 £613 £600
90th Percentile £775 £688 £700
England median daily rate £525 £483 £475
% change year-on-year +8.81% +1.58% -
Number of hourly rates quoted 43 26 27
10th Percentile £21.31 £17.00 £15.00
25th Percentile £26.34 £21.25 £19.56
Median hourly rate £37.50 £40.25 £26.48
Median % change year-on-year -6.83% +52.00% -2.83%
75th Percentile £61.88 £57.50 £44.69
90th Percentile £74.00 £68.95 £61.20
England median hourly rate £35.00 £24.50 £20.00
% change year-on-year +42.86% +22.50% +17.23%

Apache Spark
Job Vacancy Trend in London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in London.

Job vacancy trend for Apache Spark in London

Apache Spark
Contractor Daily Rate Trend in London

3-month moving average daily rate quoted in jobs citing Apache Spark in London.

Daily rate trend for Apache Spark in London

Apache Spark
Daily Rate Histogram in London

Daily rate distribution for jobs citing Apache Spark in London over the 6 months to 17 May 2022.

Daily rate histogram for Apache Spark in London

Apache Spark
Contractor Hourly Rate Trend in London

3-month moving average hourly rates quoted in jobs citing Apache Spark in London.

Hourly rate trend for Apache Spark in London

Apache Spark
Hourly Rate Histogram in London

Hourly rate distribution of jobs citing Apache Spark in London over the 6 months to 17 May 2022.

Hourly rate histogram for Apache Spark in London

Apache Spark
Contract Job Locations in London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the London region over the 6 months to 17 May 2022. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Jobs
Central London -31 65 £663 +10.42% 35
South London -10 3 £600 -4.00%
West London -14 1 £400 -35.67% 9
Apache Spark
England

For the 6 months to 17 May 2022, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the London region with a requirement for Apache Spark.

1 341 (55.72%) Python
2 330 (53.92%) Big Data
3 289 (47.22%) Kafka
4 266 (43.46%) Azure
5 264 (43.14%) AWS
6 258 (42.16%) Java
7 240 (39.22%) Agile
8 238 (38.89%) Analytics
9 225 (36.76%) Scala
10 219 (35.78%) CI/CD
11 213 (34.80%) SQL
12 192 (31.37%) Hadoop
13 190 (31.05%) Data Engineering
14 171 (27.94%) ETL
15 160 (26.14%) Finance
16 158 (25.82%) DevOps
17 156 (25.49%) Machine Learning
17 156 (25.49%) Databricks
18 138 (22.55%) Data Modelling
19 137 (22.39%) Data Pipeline
20 136 (22.22%) Apache
21 121 (19.77%) Data Science
22 120 (19.61%) Data Warehouse
23 116 (18.95%) Git (software)
24 115 (18.79%) Business Intelligence
25 113 (18.46%) NoSQL
25 113 (18.46%) GCP
26 104 (16.99%) Data Lake
27 103 (16.83%) Docker
28 99 (16.18%) Kubernetes

Apache Spark
Co-occurring IT Skills in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 136 (22.22%) Apache
2 69 (11.27%) Apache Airflow
3 24 (3.92%) Elasticsearch
4 13 (2.12%) Apache Pig
4 13 (2.12%) Luigi
5 12 (1.96%) Jupyter
6 11 (1.80%) Confluence
7 9 (1.47%) nginx
8 8 (1.31%) Apache Pulsar
9 7 (1.14%) Oracle Coherence
9 7 (1.14%) SAS
10 5 (0.82%) Apache Solr
10 5 (0.82%) Tomcat
11 4 (0.65%) uWSGI
12 3 (0.49%) WebLogic
13 2 (0.33%) JBoss
13 2 (0.33%) OpenStack
13 2 (0.33%) SharePoint
13 2 (0.33%) WebSphere
14 1 (0.16%) Tornado
Applications
1 6 (0.98%) SPSS
2 3 (0.49%) Microsoft Excel
2 3 (0.49%) Microsoft PowerPoint
3 1 (0.16%) Microsoft Office
Business Applications
1 5 (0.82%) SAP Oil and Gas
2 1 (0.16%) SAP S/4HANA
Cloud Services
1 266 (43.46%) Azure
2 264 (43.14%) AWS
3 113 (18.46%) GCP
4 73 (11.93%) Amazon S3
5 61 (9.97%) Serverless
6 56 (9.15%) Azure Data Factory
6 56 (9.15%) Power Platform
7 53 (8.66%) Amazon EMR
8 50 (8.17%) GitHub
9 43 (7.03%) Azure Synapse Analytics
10 32 (5.23%) Azure Functions
11 29 (4.74%) Amazon EC2
12 27 (4.41%) Azure Cosmos DB
13 22 (3.59%) Amazon Kinesis
13 22 (3.59%) AWS CloudFormation
13 22 (3.59%) Snowflake
14 21 (3.43%) AWS Lambda
15 20 (3.27%) Amazon SageMaker
16 19 (3.10%) dbt
17 17 (2.78%) Azure Monitor
Communications & Networking
1 6 (0.98%) FTP
2 5 (0.82%) HTTP
3 4 (0.65%) Network Security
3 4 (0.65%) SSH
3 4 (0.65%) WAN
4 3 (0.49%) Firewall
4 3 (0.49%) IPFS
4 3 (0.49%) Kerberos
Database & Business Intelligence
1 330 (53.92%) Big Data
2 192 (31.37%) Hadoop
3 120 (19.61%) Data Warehouse
4 113 (18.46%) NoSQL
5 104 (16.99%) Data Lake
6 84 (13.73%) Apache Hive
7 71 (11.60%) Amazon Redshift
8 61 (9.97%) SQL Server
9 59 (9.64%) MongoDB
10 56 (9.15%) Power BI
11 50 (8.17%) Apache Cassandra
12 49 (8.01%) Flink
13 48 (7.84%) HBase
13 48 (7.84%) Metadata
14 41 (6.70%) Delta Lake
15 40 (6.54%) Amazon RDS
16 39 (6.37%) Amazon Athena
16 39 (6.37%) Tableau
17 37 (6.05%) Pentaho
17 37 (6.05%) Relational Database
Development Applications
1 116 (18.95%) Git (software)
2 73 (11.93%) Jenkins
3 48 (7.84%) Maven
4 35 (5.72%) GitLab
5 33 (5.39%) Gradle
5 33 (5.39%) JIRA
5 33 (5.39%) TeamCity
6 16 (2.61%) MLflow
7 9 (1.47%) CircleCI
7 9 (1.47%) ER Studio
7 9 (1.47%) ERwin
8 8 (1.31%) Subversion
9 7 (1.14%) Sonatype Nexus
10 5 (0.82%) Eclipse
10 5 (0.82%) IntelliJ
10 5 (0.82%) kappa
11 4 (0.65%) Atlassian Bamboo
11 4 (0.65%) GoCD
11 4 (0.65%) JUnit
11 4 (0.65%) SonarQube
General
1 160 (26.14%) Finance
2 93 (15.20%) Banking
3 88 (14.38%) Social Skills
4 75 (12.25%) Analytical Skills
5 33 (5.39%) Investment Banking
5 33 (5.39%) Marketing
6 25 (4.08%) Public Sector
7 22 (3.59%) Financial Institution
7 22 (3.59%) Presentation Skills
8 18 (2.94%) Retail
9 16 (2.61%) Legal
10 14 (2.29%) Publishing
11 10 (1.63%) Law
12 7 (1.14%) Front Office
13 6 (0.98%) Games
13 6 (0.98%) Telecoms
14 3 (0.49%) Manufacturing
15 2 (0.33%) Documentation Skills
15 2 (0.33%) Electronics
15 2 (0.33%) Pharmaceutical
Job Titles
1 305 (49.84%) Data Engineer
2 100 (16.34%) Developer
3 61 (9.97%) Big Data Engineer
4 53 (8.66%) Architect
5 47 (7.68%) Python Engineer
6 45 (7.35%) Azure Engineer
6 45 (7.35%) Machine Learning Engineer
7 38 (6.21%) Data Architect
8 37 (6.05%) Java Engineer
9 34 (5.56%) Senior Data Engineer
10 28 (4.58%) Scala Developer
11 26 (4.25%) Cloud Engineer
12 23 (3.76%) AWS Engineer
13 22 (3.59%) Hadoop Engineer
14 21 (3.43%) Big Data Developer
15 19 (3.10%) Java Developer
15 19 (3.10%) Senior Developer
16 18 (2.94%) Apache Spark Developer
16 18 (2.94%) Solutions Architect
17 17 (2.78%) DevOps Engineer
Libraries, Frameworks & Software Standards
1 289 (47.22%) Kafka
2 62 (10.13%) PySpark
3 48 (7.84%) Spark Streaming
4 34 (5.56%) Pandas
5 30 (4.90%) TensorFlow
6 26 (4.25%) Apache Beam
7 24 (3.92%) Apache NiFi
8 22 (3.59%) PyTorch
8 22 (3.59%) scikit-learn
9 18 (2.94%) NumPy
10 15 (2.45%) REST
11 13 (2.12%) pytest
11 13 (2.12%) React
12 12 (1.96%) Apache Avro
12 12 (1.96%) CSS
12 12 (1.96%) Elastic Stack
12 12 (1.96%) Flask
13 11 (1.80%) HTML
14 10 (1.63%) JSON
14 10 (1.63%) XML
Miscellaneous
1 60 (9.80%) Algorithms
2 34 (5.56%) Data Structures
3 27 (4.41%) Public Cloud
4 24 (3.92%) Self-Motivation
5 17 (2.78%) Linux Command Line
6 16 (2.61%) Greenfield Project
7 15 (2.45%) Management Information System
8 14 (2.29%) Hedge funds
9 12 (1.96%) iPad
10 11 (1.80%) Equities
10 11 (1.80%) Market Risk
11 10 (1.63%) Cloud Native
11 10 (1.63%) Virtual Team
12 9 (1.47%) Enterprise Software
13 8 (1.31%) Data Centre
14 7 (1.14%) Legacy Systems
15 6 (0.98%) Social Media
16 5 (0.82%) Derivative
16 5 (0.82%) Foreign Exchange (FX)
16 5 (0.82%) Renewable Energy
Operating Systems
1 81 (13.24%) Linux
2 17 (2.78%) Unix
3 5 (0.82%) Windows
4 4 (0.65%) Ubuntu
5 2 (0.33%) Android
5 2 (0.33%) Apple iOS
6 1 (0.16%) Debian
6 1 (0.16%) Red Hat Enterprise Linux
Processes & Methodologies
1 240 (39.22%) Agile
2 238 (38.89%) Analytics
3 219 (35.78%) CI/CD
4 190 (31.05%) Data Engineering
5 171 (27.94%) ETL
6 158 (25.82%) DevOps
7 156 (25.49%) Machine Learning
8 138 (22.55%) Data Modelling
9 137 (22.39%) Data Pipeline
10 121 (19.77%) Data Science
11 115 (18.79%) Business Intelligence
12 94 (15.36%) TDD
13 80 (13.07%) Software Engineering
14 73 (11.93%) Migration
15 71 (11.60%) Scrum
16 68 (11.11%) BDD
17 67 (10.95%) Microservices
18 64 (10.46%) Artificial Intelligence
19 63 (10.29%) Data Analysis
20 58 (9.48%) Data Integration
Programming Languages
1 341 (55.72%) Python
2 258 (42.16%) Java
3 225 (36.76%) Scala
4 213 (34.80%) SQL
5 42 (6.86%) R
6 28 (4.58%) C#
7 24 (3.92%) Bash
8 21 (3.43%) C++
8 21 (3.43%) Shell Script
9 18 (2.94%) JavaScript
9 18 (2.94%) PowerShell
10 13 (2.12%) C
11 7 (1.14%) TypeScript
12 6 (0.98%) PHP
12 6 (0.98%) Ruby
13 5 (0.82%) Go
13 5 (0.82%) PL/SQL
14 4 (0.65%) Data Analysis Expressions (DAX)
14 4 (0.65%) T-SQL
15 3 (0.49%) MATLAB
Qualifications
1 54 (8.82%) Security Cleared
2 50 (8.17%) SC Cleared
3 26 (4.25%) Degree
4 11 (1.80%) Computer Science Degree
5 10 (1.63%) PhD
6 9 (1.47%) FFIEC
7 7 (1.14%) PMI Certification
7 7 (1.14%) PMP
8 6 (0.98%) BPSS Clearance
9 4 (0.65%) AWS Certification
9 4 (0.65%) CCA
9 4 (0.65%) DV Cleared
9 4 (0.65%) Master's Degree
10 1 (0.16%) Sun Certification
Quality Assurance & Compliance
1 47 (7.68%) Data Quality
2 13 (2.12%) GDPR
3 11 (1.80%) QA
4 9 (1.47%) GLBA
4 9 (1.47%) NIST
4 9 (1.47%) PCI DSS
4 9 (1.47%) SLA
5 3 (0.49%) IFRS
5 3 (0.49%) IFRS 17
6 1 (0.16%) PMO
System Software
1 103 (16.83%) Docker
2 39 (6.37%) HDFS
3 6 (0.98%) Apache Flume
4 4 (0.65%) Active Directory
5 3 (0.49%) Virtual Machines
6 1 (0.16%) Apache ZooKeeper
Systems Management
1 99 (16.18%) Kubernetes
2 74 (12.09%) Terraform
3 14 (2.29%) Oozie
4 11 (1.80%) Ansible
4 11 (1.80%) Grafana
5 10 (1.63%) Kibana
5 10 (1.63%) Kubeflow
5 10 (1.63%) logstash
6 5 (0.82%) Hadoop YARN
6 5 (0.82%) Prometheus
7 3 (0.49%) Argo
7 3 (0.49%) Istio
8 2 (0.33%) Ambari
Vendors
1 156 (25.49%) Databricks
2 72 (11.76%) Talend
3 61 (9.97%) Microsoft
4 47 (7.68%) Cloudera
5 43 (7.03%) Google
6 42 (6.86%) Oracle
7 26 (4.25%) Hortonworks
8 24 (3.92%) Salesforce.com
9 21 (3.43%) SAP
10 20 (3.27%) Informatica
11 19 (3.10%) Alteryx
12 16 (2.61%) Splunk
13 15 (2.45%) Salesforce Marketing Cloud
13 15 (2.45%) Teradata
14 7 (1.14%) IBM
14 7 (1.14%) Netezza
15 6 (0.98%) StreamSets
16 5 (0.82%) AppDynamics
16 5 (0.82%) MapR
17 4 (0.65%) Foundry