PySpark Jobs in London

PySpark - Spark Python API
England > London

The table below provides summary statistics for permanent job vacancies advertised in London requiring PySpark skills. It includes a benchmarking guide to the annual salaries offered in vacancies that cited PySpark over the 6 months leading up to 26 July 2024, comparing them to the same period in the previous two years.

6 months to
26 Jul 2024
Same period 2023 Same period 2022
Rank 240 419 526
Rank change year-on-year +179 +107 +39
Permanent jobs citing PySpark 269 141 457
As % of all permanent jobs advertised in London 1.30% 0.52% 0.61%
As % of the Libraries, Frameworks & Software Standards category 5.24% 1.64% 1.37%
Number of salaries quoted 201 119 153
10th Percentile £62,500 £60,000 £50,500
25th Percentile £75,000 £75,000 £62,500
Median annual salary (50th Percentile) £105,000 £100,000 £75,000
Median % change year-on-year +5.00% +33.33% +12.78%
75th Percentile £142,500 £140,000 £82,500
90th Percentile £147,500 £160,000 £90,000
England median annual salary £90,000 £80,000 £70,000
% change year-on-year +12.50% +14.29% +6.46%

All Software Libraries and Frameworks
London

PySpark falls under the Software Libraries and Frameworks category. For comparison with the information above, the following table provides summary statistics for all permanent job vacancies requiring technical specification, industry standards, software libraries and framework skills in London.

Permanent vacancies with a requirement for technical specification, industry standards, software libraries and framework skills 5,129 8,605 33,255
As % of all permanent jobs advertised in London 24.73% 31.73% 44.33%
Number of salaries quoted 3,874 7,101 12,543
10th Percentile £48,750 £50,000 £47,500
25th Percentile £62,500 £62,500 £58,250
Median annual salary (50th Percentile) £85,000 £80,000 £75,000
Median % change year-on-year +6.25% +6.67% +3.45%
75th Percentile £112,500 £100,000 £95,000
90th Percentile £142,500 £120,000 £115,000
England median annual salary £60,000 £67,500 £65,000
% change year-on-year -11.11% +3.85% +8.33%

PySpark
Job Vacancy Trend in London

Job postings citing PySpark as a proportion of all IT jobs advertised in London.

Job vacancy trend for PySpark in London

PySpark
Salary Trend in London

3-month moving average salary quoted in jobs citing PySpark in London.

Salary trend for PySpark in London

PySpark
Salary Histogram in London

Salary distribution for jobs citing PySpark in London over the 6 months to 26 July 2024.

Salary histogram for PySpark in London

PySpark
Job Locations in London

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing PySpark within the London region over the 6 months to 26 July 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
Central London +80 56 £70,000 -17.65% 5
West London -11 1 £64,000 -14.67% 1
PySpark
England

PySpark
Co-occurring Skills and Capabilities in London by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 111 (41.26%) Apache Airflow
2 81 (30.11%) Apache
3 17 (6.32%) Apache Spark
4 1 (0.37%) Apache Pig
4 1 (0.37%) SAS
4 1 (0.37%) SharePoint
Applications
1 5 (1.86%) Microsoft Excel
Business Applications
1 2 (0.74%) Primavera
2 1 (0.37%) SAP ERP
Cloud Services
1 133 (49.44%) AWS
2 129 (47.96%) Azure
3 126 (46.84%) GCP
4 105 (39.03%) dbt
5 79 (29.37%) Snowflake
6 73 (27.14%) Amazon EC2
6 73 (27.14%) Amazon EKS
7 72 (26.77%) Amazon ECS
8 40 (14.87%) Azure Synapse Analytics
9 37 (13.75%) Azure Data Factory
10 30 (11.15%) Dagster
11 25 (9.29%) Power Platform
12 15 (5.58%) Azure DevOps
13 11 (4.09%) Serverless
14 9 (3.35%) Azure Cosmos DB
15 8 (2.97%) Amazon S3
15 8 (2.97%) AWS Lambda
15 8 (2.97%) Cloud Computing
15 8 (2.97%) GitHub
16 7 (2.60%) PowerApps
Database & Business Intelligence
1 146 (54.28%) Data Lake
2 109 (40.52%) Amazon Athena
3 102 (37.92%) Dremio
4 72 (26.77%) Apache Arrow
4 72 (26.77%) Apache Ignite
5 46 (17.10%) Tableau
6 34 (12.64%) Data Warehouse
7 25 (9.29%) Power BI
8 15 (5.58%) Big Data
9 13 (4.83%) Spark SQL
10 11 (4.09%) Azure SQL Database
10 11 (4.09%) SQL Server
11 8 (2.97%) Delta Lake
12 6 (2.23%) Data Mining
13 5 (1.86%) BigQuery
13 5 (1.86%) Hadoop
14 4 (1.49%) Amazon Redshift
14 4 (1.49%) NoSQL
14 4 (1.49%) SQL Server Integration Services
15 3 (1.12%) Apache Hive
Development Applications
1 54 (20.07%) Git
2 5 (1.86%) Jenkins
3 3 (1.12%) JMeter
4 2 (0.74%) CircleCI
4 2 (0.74%) JIRA
4 2 (0.74%) Postman
4 2 (0.74%) SpecFlow
5 1 (0.37%) GitLab
5 1 (0.37%) Octopus Deploy
5 1 (0.37%) Selenium
5 1 (0.37%) SoapUI
5 1 (0.37%) Team Foundation Server
5 1 (0.37%) TeamCity
5 1 (0.37%) TestCafe
5 1 (0.37%) Zephyr
General
1 127 (47.21%) Finance
2 107 (39.78%) Banking
3 102 (37.92%) Front Office
4 72 (26.77%) Investment Banking
4 72 (26.77%) Social Skills
5 21 (7.81%) Inclusion and Diversity
6 18 (6.69%) Legal
7 8 (2.97%) Analytical Skills
7 8 (2.97%) Law
7 8 (2.97%) Publishing
8 5 (1.86%) Influencing Skills
9 4 (1.49%) Presentation Skills
10 2 (0.74%) Marketing
10 2 (0.74%) Retail
10 2 (0.74%) Retail Banking
11 1 (0.37%) Organisational Skills
11 1 (0.37%) Pharmaceutical
Job Titles
1 120 (44.61%) Data Engineer
2 80 (29.74%) Senior
3 72 (26.77%) Python Engineer
4 61 (22.68%) Lead
5 51 (18.96%) Python Data Engineer
6 47 (17.47%) Data Scientist
7 46 (17.10%) Senior Data Scientist
8 42 (15.61%) Software Engineer
9 38 (14.13%) Azure Engineer
10 37 (13.75%) Architect
11 36 (13.38%) AWS Engineer
12 34 (12.64%) Azure Data Engineer
13 32 (11.90%) Developer
13 32 (11.90%) Solutions Architect
14 30 (11.15%) AWS Developer
14 30 (11.15%) AWS Software Engineer
14 30 (11.15%) Developer/Software Engineer
14 30 (11.15%) Lead Developer
14 30 (11.15%) Software Developer
14 30 (11.15%) Team Leader
Libraries, Frameworks & Software Standards
1 151 (56.13%) Pandas
2 105 (39.03%) NumPy
3 104 (38.66%) SciPy
4 72 (26.77%) gRPC
4 72 (26.77%) Protocol Buffers
5 48 (17.84%) scikit-learn
6 45 (16.73%) TensorFlow
7 4 (1.49%) Kafka
8 3 (1.12%) SOAP
9 2 (0.74%) Apache Avro
9 2 (0.74%) Apache Beam
9 2 (0.74%) Plotly
9 2 (0.74%) PyTorch
9 2 (0.74%) Spark Streaming
10 1 (0.37%) ARM Templates
10 1 (0.37%) ASP.NET Web API
10 1 (0.37%) AWS CDK
10 1 (0.37%) PyArrow
10 1 (0.37%) Swagger
10 1 (0.37%) Twitter Bootstrap
Miscellaneous
1 72 (26.77%) Derivative
1 72 (26.77%) Equities
1 72 (26.77%) Fixed Income
1 72 (26.77%) Foreign Exchange (FX)
1 72 (26.77%) Hedge funds
2 51 (18.96%) Algorithms
3 9 (3.35%) Greenfield Project
4 5 (1.86%) Data Structures
5 2 (0.74%) Cloud Native
5 2 (0.74%) Distributed Applications
5 2 (0.74%) Distributed Systems
5 2 (0.74%) Hybrid Cloud
5 2 (0.74%) PropTech
6 1 (0.37%) Blockchain
6 1 (0.37%) Data Fabric
6 1 (0.37%) Distributed Computing
6 1 (0.37%) IoT
6 1 (0.37%) Legacy Applications
6 1 (0.37%) Management Information System
6 1 (0.37%) Public Cloud
Operating Systems
1 1 (0.37%) Linux
Processes & Methodologies
1 126 (46.84%) Agile
2 109 (40.52%) Software Engineering
3 104 (38.66%) Asset Management
4 86 (31.97%) Migration
5 80 (29.74%) Data Engineering
6 68 (25.28%) Analytics
7 66 (24.54%) Data Science
8 62 (23.05%) Line Management
9 59 (21.93%) Machine Learning
10 56 (20.82%) Artificial Intelligence
11 55 (20.45%) Visualisation
12 54 (20.07%) ETL
13 50 (18.59%) Decision-Making
14 47 (17.47%) Actionable Insight
14 47 (17.47%) MLOps
15 46 (17.10%) Mathematics
15 46 (17.10%) Natural Language Processing
15 46 (17.10%) Predictive Modelling
15 46 (17.10%) Statistics
16 45 (16.73%) Root Cause Analysis
Programming Languages
1 241 (89.59%) Python
2 221 (82.16%) SQL
3 106 (39.41%) Java
4 73 (27.14%) JavaScript
4 73 (27.14%) TypeScript
5 14 (5.20%) T-SQL
6 10 (3.72%) Scala
7 4 (1.49%) Data Analysis Expressions
7 4 (1.49%) R
8 1 (0.37%) C#
8 1 (0.37%) Groovy
8 1 (0.37%) Shell Script
Qualifications
1 61 (22.68%) Degree
2 46 (17.10%) DBS Check
3 45 (16.73%) Master's Degree
4 7 (2.60%) Computer Science Degree
5 2 (0.74%) Azure Certification
5 2 (0.74%) CIMA
6 1 (0.37%) Postgraduate
6 1 (0.37%) SC Cleared
6 1 (0.37%) Security Cleared
Quality Assurance & Compliance
1 9 (3.35%) Data Quality
2 4 (1.49%) QA
3 1 (0.37%) ISO 20022
System Software
1 2 (0.74%) HDFS
2 1 (0.37%) Apache Flume
2 1 (0.37%) Docker
Systems Management
1 5 (1.86%) Terraform
Vendors
1 127 (47.21%) Databricks
2 60 (22.30%) Bloomberg
2 60 (22.30%) Reuters
3 15 (5.58%) Microsoft
4 4 (1.49%) Google
4 4 (1.49%) Informatica
5 3 (1.12%) Cloudera
5 3 (1.12%) Oracle
5 3 (1.12%) SAP
5 3 (1.12%) SnapLogic
6 1 (0.37%) Atlassian
6 1 (0.37%) Confluent
6 1 (0.37%) IBM
6 1 (0.37%) Marketo
6 1 (0.37%) Salesforce
6 1 (0.37%) ServiceNow
6 1 (0.37%) StreamSets
6 1 (0.37%) Teradata