Period
to 23 April 2021

The following table provides summary statistics for contract job vacancies advertised in the City of London with a requirement for Apache Spark skills. Included is a benchmarking guide to the contractor rates offered in vacancies that have cited Apache Spark over the 6 months to 23 April 2021 with a comparison to the same period in the previous 2 years.

Note that daily and hourly rates are treated separately in these statistics. When calculating contractor rate percentiles, daily rates are never derived from quoted hourly rates or vice versa.

Apache Spark
Central London > City of London
6 months to
23 Apr 2021
Same period 2020 Same period 2019
Rank 93 100 56
Rank change year-on-year +7 -44 +53
Contract jobs citing Apache Spark 53 70 277
As % of all contract jobs advertised in the City of London 2.09% 1.84% 4.13%
As % of the Application Platforms category 16.16% 12.84% 23.36%
Number of daily rates quoted 43 56 238
10th Percentile £444 £413 "
25th Percentile £539 £459 £513
Median daily rate (50th Percentile) £600 £550 £550
Median % change year-on-year +9.09% - -4.35%
75th Percentile £638 £614 £625
90th Percentile £685 £681 £688
Central London median daily rate £600 £559 £550
% change year-on-year +7.33% +1.64% -

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all contract job vacancies advertised in the City of London with a requirement for application platform skills.

All Application Platform Skills
City of London
Contract vacancies with a requirement for application platform skills 328 545 1,186
As % of all contract IT jobs advertised in the City of London 12.95% 14.30% 17.66%
Number of daily rates quoted 219 403 894
10th Percentile £305 £313 £300
25th Percentile £425 £413 £413
Median daily rate (50th Percentile) £525 £500 £525
Median % change year-on-year +5.00% -4.76% +5.00%
75th Percentile £602 £600 £589
90th Percentile £688 £673 £668
Central London median daily rate £525 £500 £525
% change year-on-year +5.00% -4.76% +5.00%
Number of hourly rates quoted 2 4 10
10th Percentile £14.48 " £15.68
25th Percentile £15.90 £24.50 £18.53
Median hourly rate £18.54 £26.00 £19.63
Median % change year-on-year -28.70% +32.48% -1.88%
75th Percentile £20.90 £34.63 £31.41
90th Percentile £22.16 £48.35 £39.75
Central London median hourly rate £20.00 £27.00 £19.25
% change year-on-year -25.93% +40.26% -3.75%

Apache Spark
Job Vacancy Trend in the City of London

Job postings citing Apache Spark as a proportion of all IT jobs advertised in the City of London.

Job vacancy trend for Apache Spark in the City of London

Apache Spark
Contractor Daily Rate Trend in the City of London

3-month moving average daily rate quoted in jobs citing Apache Spark in the City of London.

Daily rate trend for Apache Spark in the City of London

Apache Spark
Daily Rate Histogram in the City of London

Daily rate distribution for jobs citing Apache Spark in the City of London over the 6 months to 23 April 2021.

Daily rate histogram for Apache Spark in the City of London

Apache Spark
Contract Job Locations in the City of London

The table below looks at the demand and provides a guide to the median contractor rates quoted in IT jobs citing Apache Spark within the City of London region over the 6 months to 23 April 2021. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Contract
IT Job Ads
Median
Daily Rate
Past 6 Months
Median Daily Rate
% Change
on Same Period
Last Year
Live
Job
Vacancies
London Bridge +4 6 £600 +20.00%
Apache Spark
Central London

For the 6 months to 23 April 2021, IT contractor jobs citing Apache Spark also mentioned the following skills in order of popularity. The figures indicate the absolute number co-occurrences and as a proportion of all contract job ads across the City of London region with a requirement for Apache Spark.

1 35 (66.04%) Big Data
2 29 (54.72%) Hadoop
2 29 (54.72%) Python
3 26 (49.06%) SQL
4 24 (45.28%) AWS
5 22 (41.51%) Java
5 22 (41.51%) Data Lake
5 22 (41.51%) Analytical Skills
6 20 (37.74%) Kafka
7 19 (35.85%) Finance
8 17 (32.08%) Data Warehouse
8 17 (32.08%) Scala
9 16 (30.19%) Apache Hive
9 16 (30.19%) Apache Airflow
10 15 (28.30%) Machine Learning
10 15 (28.30%) Apache
11 14 (26.42%) NoSQL
11 14 (26.42%) Algorithms
12 13 (24.53%) Analytics
13 12 (22.64%) Agile Software Development
13 12 (22.64%) Data Science
14 11 (20.75%) Data Structures
14 11 (20.75%) Banking
14 11 (20.75%) MapReduce
15 10 (18.87%) Amazon Redshift
15 10 (18.87%) HTTP
15 10 (18.87%) Web Services
15 10 (18.87%) XML
16 9 (16.98%) PySpark
16 9 (16.98%) Data Analysis

Apache Spark
Co-occurring IT Skills in the City of London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same job type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 16 (30.19%) Apache Airflow
2 15 (28.30%) Apache
Cloud Services
1 24 (45.28%) AWS
2 8 (15.09%) Amazon Kinesis
3 7 (13.21%) Amazon EC2
4 4 (7.55%) Azure
5 3 (5.66%) Amazon EMR
6 2 (3.77%) Amazon S3
6 2 (3.77%) Cloud Computing
6 2 (3.77%) GCP
7 1 (1.89%) Azure Data Factory
7 1 (1.89%) Azure Synapse Analytics
Communications & Networking
1 10 (18.87%) HTTP
2 1 (1.89%) Firewall
Database & Business Intelligence
1 35 (66.04%) Big Data
2 29 (54.72%) Hadoop
3 22 (41.51%) Data Lake
4 17 (32.08%) Data Warehouse
5 16 (30.19%) Apache Hive
6 14 (26.42%) NoSQL
7 10 (18.87%) Amazon Redshift
8 8 (15.09%) Parquet
8 8 (15.09%) Snowflake Schema
9 7 (13.21%) Amazon RDS
9 7 (13.21%) Relational Database
10 6 (11.32%) Spark SQL
11 4 (7.55%) Impala
11 4 (7.55%) MongoDB
12 3 (5.66%) Flink
13 2 (3.77%) Couchbase
13 2 (3.77%) Data Mining
13 2 (3.77%) PostgreSQL
13 2 (3.77%) Presto
14 1 (1.89%) Data Mart
Development Applications
1 5 (9.43%) JIRA
2 1 (1.89%) Git (software)
2 1 (1.89%) Jenkins
2 1 (1.89%) ProC
2 1 (1.89%) Sonatype Nexus
General
1 19 (35.85%) Finance
2 11 (20.75%) Banking
3 2 (3.77%) Front Office
3 2 (3.77%) Games
3 2 (3.77%) Investment Banking
3 2 (3.77%) Local Government
3 2 (3.77%) Public Sector
4 1 (1.89%) Retail Banking
Job Titles
1 35 (66.04%) Data Engineer
2 15 (28.30%) Developer
3 10 (18.87%) AWS Engineer
4 8 (15.09%) Big Data Engineer
4 8 (15.09%) Java Developer
5 5 (9.43%) Hadoop Engineer
5 5 (9.43%) Python Engineer
6 4 (7.55%) Data Scientist
7 3 (5.66%) Architect
7 3 (5.66%) Senior Data Engineer
8 2 (3.77%) Analyst
8 2 (3.77%) Data Analyst
8 2 (3.77%) DevOps Developer
8 2 (3.77%) ETL Developer
8 2 (3.77%) Platform Engineer
8 2 (3.77%) Python Developer
8 2 (3.77%) Scala Engineer
8 2 (3.77%) Senior Data Scientist
9 1 (1.89%) Platform Architect
9 1 (1.89%) Scala Developer
Libraries, Frameworks & Software Standards
1 20 (37.74%) Kafka
2 10 (18.87%) Web Services
2 10 (18.87%) XML
3 9 (16.98%) PySpark
4 8 (15.09%) HTML
4 8 (15.09%) PostGIS
5 6 (11.32%) Spring
6 3 (5.66%) GraphX
6 3 (5.66%) MLlib
6 3 (5.66%) RESTful
6 3 (5.66%) Samza
7 2 (3.77%) JSON
8 1 (1.89%) Akka
8 1 (1.89%) RabbitMQ
8 1 (1.89%) Spring Boot
8 1 (1.89%) TensorFlow
8 1 (1.89%) Velocity
Miscellaneous
1 22 (41.51%) Analytical Skills
2 14 (26.42%) Algorithms
3 11 (20.75%) Data Structures
4 8 (15.09%) Self-Motivation
5 3 (5.66%) Grid Computing
6 2 (3.77%) Cloud Native
6 2 (3.77%) Foreign Exchange (FX)
6 2 (3.77%) Greenfield Project
6 2 (3.77%) Organisational Skills
7 1 (1.89%) Derivative
7 1 (1.89%) Equities
7 1 (1.89%) Fixed Income
7 1 (1.89%) Linux Command Line
7 1 (1.89%) Market Risk
7 1 (1.89%) Video\Live Streaming
Operating Systems
1 5 (9.43%) Linux
2 1 (1.89%) Unix
Processes & Methodologies
1 15 (28.30%) Machine Learning
2 13 (24.53%) Analytics
3 12 (22.64%) Agile Software Development
3 12 (22.64%) Data Science
4 11 (20.75%) MapReduce
5 9 (16.98%) Data Analysis
5 9 (16.98%) Data Transformation
5 9 (16.98%) Load Testing
6 8 (15.09%) Change Management
6 8 (15.09%) Continuous Deployment
6 8 (15.09%) Data Modelling
6 8 (15.09%) DevOps
6 8 (15.09%) Forensic Analysis
6 8 (15.09%) SQL Optimisation
6 8 (15.09%) Use Case
6 8 (15.09%) XML Parsing
7 6 (11.32%) Data Engineering
7 6 (11.32%) ETL
7 6 (11.32%) TDD
8 5 (9.43%) Artificial Intelligence
Programming Languages
1 29 (54.72%) Python
2 26 (49.06%) SQL
3 22 (41.51%) Java
4 17 (32.08%) Scala
5 4 (7.55%) R
6 3 (5.66%) C++
7 2 (3.77%) Groovy
7 2 (3.77%) JavaScript
8 1 (1.89%) C
Qualifications
1 3 (5.66%) SC Cleared
1 3 (5.66%) Security Cleared
2 2 (3.77%) Degree
3 1 (1.89%) AWS Certification
3 1 (1.89%) Computer Science Degree
3 1 (1.89%) PhD
Quality Assurance & Compliance
1 4 (7.55%) QA
2 1 (1.89%) SLA
System Software
1 6 (11.32%) HDFS
2 2 (3.77%) Docker
Systems Management
1 5 (9.43%) Kubernetes
2 4 (7.55%) Hadoop YARN
3 3 (5.66%) Ambari
3 3 (5.66%) Grafana
4 1 (1.89%) Oozie
Vendors
1 9 (16.98%) Cloudera
2 7 (13.21%) Talend
3 5 (9.43%) Hortonworks
4 4 (7.55%) Databricks
5 2 (3.77%) Oracle