PySpark Jobs in London

PySpark - Spark Python API
England > London

The table below provides summary statistics for permanent job vacancies advertised in London requiring PySpark skills. It includes a benchmarking guide to the annual salaries offered in vacancies that cited PySpark over the 6 months leading up to 31 October 2024, comparing them to the same period in the previous two years.

6 months to
31 Oct 2024
Same period 2023 Same period 2022
Rank 230 379 514
Rank change year-on-year +149 +135 +50
Permanent jobs citing PySpark 253 101 289
As % of all permanent jobs advertised in London 1.35% 0.51% 0.52%
As % of the Libraries, Frameworks & Software Standards category 5.20% 1.69% 1.37%
Number of salaries quoted 220 92 149
10th Percentile £69,500 £52,500 £55,000
25th Percentile £85,000 £59,375 £62,500
Median annual salary (50th Percentile) £107,500 £80,000 £80,000
Median % change year-on-year +34.38% - +14.29%
75th Percentile £142,500 £105,000 £90,000
90th Percentile £147,500 £144,250 £108,750
England median annual salary £105,000 £75,000 £80,000
% change year-on-year +40.00% -6.25% +14.29%

All Software Libraries and Frameworks
London

PySpark falls under the Software Libraries and Frameworks category. For comparison with the information above, the following table provides summary statistics for all permanent job vacancies requiring technical specification, industry standards, software libraries and framework skills in London.

Permanent vacancies with a requirement for technical specification, industry standards, software libraries and framework skills 4,865 5,982 21,052
As % of all permanent jobs advertised in London 25.89% 29.91% 37.84%
Number of salaries quoted 2,996 5,123 11,088
10th Percentile £51,250 £51,250 £47,500
25th Percentile £63,750 £62,500 £60,000
Median annual salary (50th Percentile) £85,000 £80,000 £78,142
Median % change year-on-year +6.25% +2.38% +4.19%
75th Percentile £110,000 £100,000 £97,500
90th Percentile £140,000 £120,000 £120,000
England median annual salary £65,000 £67,500 £65,000
% change year-on-year -3.70% +3.85% +8.33%

PySpark
Job Vacancy Trend in London

Job postings citing PySpark as a proportion of all IT jobs advertised in London.

Job vacancy trend for PySpark in London

PySpark
Salary Trend in London

3-month moving average salary quoted in jobs citing PySpark in London.

Salary trend for PySpark in London

PySpark
Salary Histogram in London

Salary distribution for jobs citing PySpark in London over the 6 months to 31 October 2024.

Salary histogram for PySpark in London

PySpark
Job Locations in London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing PySpark within the London region over the 6 months to 31 October 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
Central London +14 37 £90,000 +12.50% 4
West London - 1 £100,000 - 1
PySpark
England

PySpark
Co-occurring Skills and Capabilities in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 148 (58.50%) Apache Airflow
2 99 (39.13%) Apache
3 8 (3.16%) Apache Spark
4 1 (0.40%) SAS
Business Applications
1 1 (0.40%) SAP ERP
Cloud Services
1 168 (66.40%) AWS
2 161 (63.64%) GCP
3 148 (58.50%) dbt
4 127 (50.20%) Azure
5 95 (37.55%) Snowflake
6 93 (36.76%) Amazon EC2
6 93 (36.76%) Amazon EKS
7 92 (36.36%) Amazon ECS
8 54 (21.34%) Dagster
9 29 (11.46%) Azure Synapse Analytics
10 28 (11.07%) Azure Data Factory
11 21 (8.30%) Power Platform
12 13 (5.14%) Azure DevOps
13 10 (3.95%) Serverless
14 9 (3.56%) AWS Lambda
15 8 (3.16%) Amazon S3
16 6 (2.37%) GitHub
16 6 (2.37%) WhatsApp
17 5 (1.98%) Cloud Computing
18 4 (1.58%) PowerApps
Database & Business Intelligence
1 177 (69.96%) Data Lake
2 152 (60.08%) Amazon Athena
3 146 (57.71%) Dremio
4 92 (36.36%) Apache Arrow
4 92 (36.36%) Apache Ignite
5 23 (9.09%) Tableau
6 21 (8.30%) Power BI
7 18 (7.11%) Data Warehouse
8 8 (3.16%) Big Data
8 8 (3.16%) SQL Server
9 7 (2.77%) Amazon Redshift
9 7 (2.77%) Spark SQL
10 5 (1.98%) BigQuery
11 3 (1.19%) Delta Lake
12 2 (0.79%) Amazon RDS
12 2 (0.79%) Azure SQL Database
12 2 (0.79%) Snowflake Schema
13 1 (0.40%) Apache Hive
13 1 (0.40%) MongoDB
13 1 (0.40%) NoSQL
Development Applications
1 22 (8.70%) Git
2 2 (0.79%) git-flow
2 2 (0.79%) MLflow
3 1 (0.40%) GitLab
3 1 (0.40%) JIRA
3 1 (0.40%) JMeter
3 1 (0.40%) TestCafe
3 1 (0.40%) Zephyr
General
1 158 (62.45%) Finance
2 151 (59.68%) Banking
3 146 (57.71%) Front Office
4 92 (36.36%) Investment Banking
5 35 (13.83%) Social Skills
6 16 (6.32%) Inclusion and Diversity
7 10 (3.95%) Legal
8 9 (3.56%) Publishing
9 7 (2.77%) Presentation Skills
10 5 (1.98%) Analytical Skills
11 4 (1.58%) Marketing
12 1 (0.40%) Influencing Skills
12 1 (0.40%) Law
12 1 (0.40%) Pharmaceutical
12 1 (0.40%) Retail
Job Titles
1 124 (49.01%) Data Engineer
2 112 (44.27%) Python Engineer
3 80 (31.62%) Python Data Engineer
4 65 (25.69%) Lead
5 63 (24.90%) Software Engineer
6 35 (13.83%) AWS Engineer
7 34 (13.44%) Azure Engineer
7 34 (13.44%) Lead Data Engineer
8 32 (12.65%) Azure Data Engineer
8 32 (12.65%) Principal Engineer
8 32 (12.65%) Principal Software Engineer
8 32 (12.65%) Python Software Engineer
9 31 (12.25%) Architect
9 31 (12.25%) Developer
9 31 (12.25%) Java Engineer
9 31 (12.25%) Java Software Engineer
9 31 (12.25%) Senior
9 31 (12.25%) Solutions Architect
10 30 (11.86%) Lead Developer
10 30 (11.86%) Software Developer
Libraries, Frameworks & Software Standards
1 167 (66.01%) Pandas
2 149 (58.89%) NumPy
3 146 (57.71%) SciPy
4 92 (36.36%) gRPC
4 92 (36.36%) Protocol Buffers
5 19 (7.51%) scikit-learn
6 17 (6.72%) TensorFlow
7 3 (1.19%) SOAP
8 1 (0.40%) ADO
8 1 (0.40%) Kafka
8 1 (0.40%) Node.js
8 1 (0.40%) PyArrow
8 1 (0.40%) pytest
8 1 (0.40%) REST
Miscellaneous
1 93 (36.76%) Hedge funds
2 92 (36.36%) Derivative
2 92 (36.36%) Equities
2 92 (36.36%) Fixed Income
2 92 (36.36%) Foreign Exchange (FX)
3 18 (7.11%) Algorithms
4 3 (1.19%) Greenfield Project
5 2 (0.79%) Data Fabric
5 2 (0.79%) IoT
5 2 (0.79%) Management Information System
5 2 (0.79%) Product Ownership
6 1 (0.40%) Analytical Mindset
6 1 (0.40%) Blockchain
6 1 (0.40%) Distributed Computing
6 1 (0.40%) Public Cloud
6 1 (0.40%) Reinsurance
6 1 (0.40%) Self-Motivation
Processes & Methodologies
1 164 (64.82%) Agile
2 151 (59.68%) Software Engineering
3 146 (57.71%) Asset Management
4 100 (39.53%) Migration
5 97 (38.34%) Data Engineering
6 95 (37.55%) Analytics
7 60 (23.72%) Line Management
8 59 (23.32%) Fintech
9 55 (21.74%) Investment Management
10 45 (17.79%) Mentoring
11 36 (14.23%) Data Science
11 36 (14.23%) ETL
12 32 (12.65%) Machine Learning
13 29 (11.46%) Artificial Intelligence
14 26 (10.28%) Visualisation
15 23 (9.09%) Data Modelling
15 23 (9.09%) Mathematics
15 23 (9.09%) Natural Language Processing
16 22 (8.70%) Data Pipeline
16 22 (8.70%) Decision-Making
Programming Languages
1 240 (94.86%) Python
2 182 (71.94%) SQL
3 147 (58.10%) Java
4 93 (36.76%) JavaScript
4 93 (36.76%) TypeScript
5 13 (5.14%) T-SQL
6 7 (2.77%) R
6 7 (2.77%) Scala
7 2 (0.79%) Data Analysis Expressions
8 1 (0.40%) Groovy
Qualifications
1 26 (10.28%) Degree
2 17 (6.72%) DBS Check
2 17 (6.72%) Master's Degree
3 7 (2.77%) Security Cleared
4 6 (2.37%) Computer Science Degree
4 6 (2.37%) SC Cleared
5 1 (0.40%) Azure Certification
5 1 (0.40%) Postgraduate
Quality Assurance & Compliance
1 1 (0.40%) Data Quality
1 1 (0.40%) ISO 20022
1 1 (0.40%) QA
Systems Management
1 3 (1.19%) Terraform
2 2 (0.79%) Kubernetes
Vendors
1 81 (32.02%) Databricks
2 60 (23.72%) Bloomberg
2 60 (23.72%) Reuters
3 17 (6.72%) Microsoft
4 6 (2.37%) Google
5 2 (0.79%) Informatica
5 2 (0.79%) SnapLogic
6 1 (0.40%) Atlassian
6 1 (0.40%) Confluent
6 1 (0.40%) IBM
6 1 (0.40%) Marketo
6 1 (0.40%) Oracle
6 1 (0.40%) Salesforce
6 1 (0.40%) SAP
6 1 (0.40%) StreamSets