Period
to

The following table provides summary statistics for contract job vacancies advertised in the City of London with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 22 May 2022 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
Central London > City of London
6 months to
22 May 2022
Same period 2021 Same period 2020
Rank 134 91 98
Rank change year-on-year -43 +7 -50
Contract jobs citing Apache Spark 59 66 61
As % of all contract jobs advertised in the City of London 1.40% 2.31% 1.92%
As % of the Application Platforms category 10.91% 17.01% 13.93%
Number of daily rates quoted 37 51 50
10th Percentile £433 £519 £413
25th Percentile £563 £563 £453
Median daily rate (50th Percentile) £625 £600 £550
Median % change year-on-year +4.17% +9.09% -
75th Percentile £775 £638 £613
90th Percentile £815 £675 £676
Central London median daily rate £663 £600 £550
% change year-on-year +10.42% +9.09% -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in the City of London with a requirement for application platform skills.

All Application Platform Skills
City of London
Contract vacancies with a requirement for application platform skills 541 388 438
As % of all contract IT jobs advertised in the City of London 12.86% 13.61% 13.82%
Number of daily rates quoted 416 261 310
10th Percentile £356 £305 £313
25th Percentile £461 £425 £413
Median daily rate (50th Percentile) £540 £525 £500
Median % change year-on-year +2.86% +5.00% -4.76%
75th Percentile £653 £600 £600
90th Percentile £738 £675 £675
Central London median daily rate £540 £525 £500
% change year-on-year +2.86% +5.00% -4.76%
Number of hourly rates quoted 4 0 4
10th Percentile £21.42 - £17.85
25th Percentile £23.88 - £22.13
Median hourly rate £26.42 - £25.00
Median % change year-on-year - - +26.58%
75th Percentile £30.50 - £33.50
90th Percentile £35.45 - £47.90
Central London median hourly rate £26.50 £20.00 £25.00
% change year-on-year +32.50% -20.00% +28.21%

Apache Spark
Job Vacancy Trend in the City of London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in the City of London.

Job vacancy trend for Apache Spark in the City of London

Apache Spark
Contractor Daily Rate Trend in the City of London

3-month moving average daily rate quoted in jobs citing Apache Spark in the City of London.

Daily rate trend for Apache Spark in the City of London

Apache Spark
Daily Rate Histogram in the City of London

Daily rate distribution for jobs citing Apache Spark in the City of London over the 6 months to 22 May 2022.

Daily rate histogram for Apache Spark in the City of London

For the 6 months to 22 May 2022, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the City of London region with a requirement for Apache Spark.

1 41 (69.49%) Big Data
2 36 (61.02%) Kafka
3 35 (59.32%) Python
4 33 (55.93%) AWS
5 30 (50.85%) SQL
6 25 (42.37%) Finance
7 23 (38.98%) Agile
8 22 (37.29%) Java
9 20 (33.90%) Scala
9 20 (33.90%) Hadoop
9 20 (33.90%) CI/CD
10 17 (28.81%) Data Engineering
11 16 (27.12%) Analytics
11 16 (27.12%) Azure
11 16 (27.12%) ETL
11 16 (27.12%) Data Pipeline
12 15 (25.42%) Data Modelling
12 15 (25.42%) Amazon S3
12 15 (25.42%) NoSQL
13 14 (23.73%) Amazon EMR
13 14 (23.73%) Relational Database
13 14 (23.73%) Apache Airflow
14 13 (22.03%) DevOps
14 13 (22.03%) Analytical Skills
14 13 (22.03%) GCP
14 13 (22.03%) Git (software)
15 12 (20.34%) Data Protection
15 12 (20.34%) Hedge funds
15 12 (20.34%) Apache Hive
15 12 (20.34%) AOP

Apache Spark
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 14 (23.73%) Apache Airflow
2 8 (13.56%) Apache
3 5 (8.47%) Luigi
4 4 (6.78%) Apache Pig
4 4 (6.78%) Tomcat
5 3 (5.08%) WebLogic
6 1 (1.69%) JBoss
6 1 (1.69%) Jupyter
6 1 (1.69%) WebSphere
Business Applications
1 1 (1.69%) SAP Oil and Gas
Cloud Services
1 33 (55.93%) AWS
2 16 (27.12%) Azure
3 15 (25.42%) Amazon S3
4 14 (23.73%) Amazon EMR
5 13 (22.03%) GCP
6 6 (10.17%) Serverless
7 5 (8.47%) Amazon EC2
7 5 (8.47%) Amazon SQS
7 5 (8.47%) AWS Lambda
8 4 (6.78%) Amazon SageMaker
8 4 (6.78%) AWS CodeBuild
8 4 (6.78%) AWS Fargate
8 4 (6.78%) AWS Step Functions
8 4 (6.78%) Azure Cosmos DB
8 4 (6.78%) dbt
8 4 (6.78%) GitHub
9 3 (5.08%) AWS CloudFormation
10 2 (3.39%) Power Platform
11 1 (1.69%) PaaS
11 1 (1.69%) Snowflake
Communications & Networking
1 3 (5.08%) HTTP
Database & Business Intelligence
1 41 (69.49%) Big Data
2 20 (33.90%) Hadoop
3 15 (25.42%) NoSQL
4 14 (23.73%) Relational Database
5 12 (20.34%) Apache Hive
5 12 (20.34%) SQL Server
6 11 (18.64%) Amazon Redshift
6 11 (18.64%) MongoDB
7 10 (16.95%) Amazon Athena
7 10 (16.95%) Amazon RDS
7 10 (16.95%) Data Warehouse
7 10 (16.95%) RDBMS
8 9 (15.25%) Apache Cassandra
9 8 (13.56%) Data Lake
9 8 (13.56%) Metadata
10 7 (11.86%) HBase
10 7 (11.86%) MySQL
10 7 (11.86%) Parquet
11 6 (10.17%) Data Vault
11 6 (10.17%) Presto
Development Applications
1 13 (22.03%) Git (software)
2 11 (18.64%) JIRA
3 7 (11.86%) Jenkins
4 4 (6.78%) Eclipse
4 4 (6.78%) IntelliJ
5 3 (5.08%) CircleCI
5 3 (5.08%) GitLab
5 3 (5.08%) GoCD
6 1 (1.69%) Gerrit
6 1 (1.69%) git-flow
6 1 (1.69%) HP UFT
6 1 (1.69%) Maven
6 1 (1.69%) NetBeans
6 1 (1.69%) Rational Robot
6 1 (1.69%) RIATest
6 1 (1.69%) SilkTest
6 1 (1.69%) Subversion
6 1 (1.69%) TeamCity
6 1 (1.69%) VSS/SourceSafe
6 1 (1.69%) WinRunner
General
1 25 (42.37%) Finance
2 13 (22.03%) Analytical Skills
3 12 (20.34%) Financial Institution
3 12 (20.34%) Legal
4 7 (11.86%) Social Skills
5 3 (5.08%) Banking
6 2 (3.39%) Public Sector
7 1 (1.69%) Investment Banking
7 1 (1.69%) Law
7 1 (1.69%) Publishing
Job Titles
1 27 (45.76%) Data Engineer
2 20 (33.90%) Developer
3 12 (20.34%) Cloud Engineer
4 7 (11.86%) Big Data Developer
4 7 (11.86%) Big Data Engineer
5 6 (10.17%) Talend Developer
6 5 (8.47%) Hadoop Engineer
6 5 (8.47%) Senior Data Engineer
7 4 (6.78%) Java Developer
7 4 (6.78%) Scala Developer
8 3 (5.08%) Architect
8 3 (5.08%) Azure Engineer
8 3 (5.08%) Senior Cloud Engineer
8 3 (5.08%) Solutions Architect
9 2 (3.39%) Apache Spark Developer
9 2 (3.39%) Data Scientist
9 2 (3.39%) Hadoop Developer
9 2 (3.39%) Java Hadoop Developer
9 2 (3.39%) Senior Developer
9 2 (3.39%) Senior Java Developer
Libraries, Frameworks & Software Standards
1 36 (61.02%) Kafka
2 9 (15.25%) Apache Avro
3 8 (13.56%) Spark Streaming
4 7 (11.86%) PySpark
5 6 (10.17%) JSON
5 6 (10.17%) XML
6 5 (8.47%) Pandas
6 5 (8.47%) scikit-learn
7 4 (6.78%) Flask
7 4 (6.78%) J2EE
7 4 (6.78%) pytest
7 4 (6.78%) RabbitMQ
8 3 (5.08%) Akka
8 3 (5.08%) Play Framework
8 3 (5.08%) Scalaz
9 2 (3.39%) Apache Beam
9 2 (3.39%) RESTful
9 2 (3.39%) TensorFlow
9 2 (3.39%) Web Services
10 1 (1.69%) SOAP
Miscellaneous
1 12 (20.34%) Hedge funds
2 9 (15.25%) Data Structures
3 8 (13.56%) Management Information System
4 7 (11.86%) Greenfield Project
5 6 (10.17%) Algorithms
6 5 (8.47%) Derivative
6 5 (8.47%) Equities
6 5 (8.47%) Foreign Exchange (FX)
6 5 (8.47%) Linux Command Line
7 3 (5.08%) Credit Risk
8 2 (3.39%) Legacy Systems
8 2 (3.39%) Self-Motivation
9 1 (1.69%) Enterprise Software
9 1 (1.69%) Public Cloud
9 1 (1.69%) Virtual Team
Operating Systems
1 6 (10.17%) Linux
1 6 (10.17%) Unix
Processes & Methodologies
1 23 (38.98%) Agile
2 20 (33.90%) CI/CD
3 17 (28.81%) Data Engineering
4 16 (27.12%) Analytics
4 16 (27.12%) Data Pipeline
4 16 (27.12%) ETL
5 15 (25.42%) Data Modelling
6 13 (22.03%) DevOps
7 12 (20.34%) AOP
7 12 (20.34%) Business Intelligence
7 12 (20.34%) Data Protection
8 11 (18.64%) Data Munging
8 11 (18.64%) Data Science
9 10 (16.95%) Data Analysis
9 10 (16.95%) OO
10 9 (15.25%) Data Integration
10 9 (15.25%) Visualisation
11 8 (13.56%) Computer Science
11 8 (13.56%) SDLC
11 8 (13.56%) Workflow
Programming Languages
1 35 (59.32%) Python
2 30 (50.85%) SQL
3 22 (37.29%) Java
4 20 (33.90%) Scala
5 5 (8.47%) R
6 2 (3.39%) Shell Script
7 1 (1.69%) JavaScript
Qualifications
1 6 (10.17%) SC Cleared
1 6 (10.17%) Security Cleared
2 2 (3.39%) BPSS Clearance
2 2 (3.39%) Computer Science Degree
2 2 (3.39%) Degree
2 2 (3.39%) Master's Degree
3 1 (1.69%) Sun Certification
Quality Assurance & Compliance
1 12 (20.34%) GDPR
2 11 (18.64%) Data Quality
3 2 (3.39%) SLA
4 1 (1.69%) QA
System Software
1 12 (20.34%) Docker
2 1 (1.69%) Apache Flume
Systems Management
1 8 (13.56%) Terraform
2 7 (11.86%) Kubernetes
3 3 (5.08%) Oozie
4 1 (1.69%) Ambari
4 1 (1.69%) Hadoop YARN
Vendors
1 11 (18.64%) Cloudera
1 11 (18.64%) Oracle
1 11 (18.64%) Talend
2 4 (6.78%) Databricks
3 3 (5.08%) CA Agile Central
3 3 (5.08%) Google
3 3 (5.08%) Microsoft
4 2 (3.39%) Alteryx
4 2 (3.39%) Netezza
4 2 (3.39%) SAP
5 1 (1.69%) Cognos
5 1 (1.69%) Green Hat
5 1 (1.69%) Hortonworks
5 1 (1.69%) HP
5 1 (1.69%) IBM
5 1 (1.69%) Sun
5 1 (1.69%) Teradata