Apache Spark SQL
UK

The following table provides summary statistics for permanent job vacancies with a requirement for Spark SQL skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Spark SQL over the 6 months to 9 May 2024 with a comparison to the same period in the previous 2 years.

6 months to
9 May 2024
Same period 2023 Same period 2022
Rank 879 945 1149
Rank change year-on-year +66 +204 -269
Permanent jobs citing Spark SQL 23 27 80
As % of all permanent jobs advertised in the UK 0.023% 0.027% 0.051%
As % of the Database & Business Intelligence category 0.13% 0.12% 0.18%
Number of salaries quoted 17 18 15
10th Percentile - £59,500 -
25th Percentile £52,500 £62,500 £43,750
Median annual salary (50th Percentile) £60,000 £80,000 £66,000
Median % change year-on-year -25.00% +21.21% -5.71%
75th Percentile £62,500 £87,500 £76,250
90th Percentile £76,500 £90,000 £84,800
UK excluding London median annual salary £60,000 £65,000 £47,500
% change year-on-year -7.69% +36.84% -14.41%

All Database and Business Intelligence Skills
UK

Spark SQL is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for database or business intelligence skills.

Permanent vacancies with a requirement for database or business intelligence skills 18,170 21,615 43,836
As % of all permanent jobs advertised in the UK 18.33% 21.33% 27.94%
Number of salaries quoted 12,550 13,499 21,044
10th Percentile £35,000 £37,500 £36,250
25th Percentile £42,725 £47,500 £45,000
Median annual salary (50th Percentile) £57,500 £65,000 £60,000
Median % change year-on-year -11.54% +8.33% +9.09%
75th Percentile £76,250 £83,750 £79,313
90th Percentile £95,000 £102,500 £95,000
UK excluding London median annual salary £52,500 £55,000 £52,500
% change year-on-year -4.55% +4.76% +8.25%

Spark SQL
Job Vacancy Trend

Job postings citing Spark SQL as a proportion of all IT jobs advertised.

Job vacancy trend for Spark SQL in the UK

Spark SQL
Salary Trend

3-month moving average salary quoted in jobs citing Spark SQL.

Salary trend for Spark SQL in the UK

Spark SQL
Salary Histogram

Salary distribution for jobs citing Spark SQL over the 6 months to 9 May 2024.

Salary histogram for Spark SQL in the UK

Spark SQL
Top 6 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Spark SQL within the UK over the 6 months to 9 May 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
England +102 23 £60,000 -25.00% 2
London +108 19 £57,500 -30.30% 1
Work from Home +83 13 £55,000 -38.89% 1
UK excluding London 0 4 £60,000 -7.69% 2
South East +3 3 £60,000 +17.65%
South West - 1 £55,000 -

Spark SQL
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 10 (43.48%) Apache Spark
2 4 (17.39%) Apache Airflow
3 1 (4.35%) Apache
3 1 (4.35%) Apache Pig
Applications
1 7 (30.43%) Microsoft Excel
Business Applications
1 4 (17.39%) Primavera
Cloud Services
1 22 (95.65%) Azure
2 13 (56.52%) Azure Data Factory
3 12 (52.17%) Azure Synapse Analytics
4 8 (34.78%) Power Platform
5 7 (30.43%) Power Automate
5 7 (30.43%) PowerApps
5 7 (30.43%) Snowflake
6 5 (21.74%) AWS
7 4 (17.39%) Azure DevOps
7 4 (17.39%) GCP
8 3 (13.04%) Amazon EMR
8 3 (13.04%) Amazon Kinesis
9 1 (4.35%) Amazon EventBridge
9 1 (4.35%) Amazon S3
9 1 (4.35%) Amazon SQS
9 1 (4.35%) AWS CloudFormation
9 1 (4.35%) AWS Lambda
9 1 (4.35%) Cloud Computing
9 1 (4.35%) GitHub
9 1 (4.35%) Serverless
Communications & Networking
1 1 (4.35%) Firewall
Database & Business Intelligence
1 18 (78.26%) Data Lake
2 9 (39.13%) Big Data
2 9 (39.13%) Data Warehouse
3 8 (34.78%) Power BI
4 7 (30.43%) Data Mining
5 4 (17.39%) SQL Server Integration Services
6 3 (13.04%) Azure SQL Data Warehouse
6 3 (13.04%) HDInsight
6 3 (13.04%) OLAP
7 2 (8.70%) MySQL
7 2 (8.70%) NoSQL
7 2 (8.70%) PostgreSQL
7 2 (8.70%) SQL Server
8 1 (4.35%) Amazon Athena
8 1 (4.35%) Delta Lake
8 1 (4.35%) DynamoDB
8 1 (4.35%) Hadoop
8 1 (4.35%) PolyBase
8 1 (4.35%) Presto
8 1 (4.35%) Sqoop
Development Applications
1 1 (4.35%) Git
General
1 9 (39.13%) Finance
2 6 (26.09%) Social Skills
3 4 (17.39%) Analytical Skills
Job Titles
1 12 (52.17%) Data Engineer
2 7 (30.43%) Lead
2 7 (30.43%) Technical Leader
3 4 (17.39%) Architect
3 4 (17.39%) Azure Architect
3 4 (17.39%) Azure Data Architect
3 4 (17.39%) Data Architect
4 3 (13.04%) Azure Data Engineer
4 3 (13.04%) Azure Engineer
5 2 (8.70%) Senior
5 2 (8.70%) Senior Data Engineer
6 1 (4.35%) Consultant
6 1 (4.35%) Data Consultant
Libraries, Frameworks & Software Standards
1 18 (78.26%) PySpark
2 3 (13.04%) REST
2 3 (13.04%) SOAP
3 1 (4.35%) .NET
3 1 (4.35%) LAMP
3 1 (4.35%) Pandas
Miscellaneous
1 3 (13.04%) Algorithms
1 3 (13.04%) Data Structures
1 3 (13.04%) Management Information System
2 1 (4.35%) Cloud Native
Operating Systems
1 1 (4.35%) Linux
1 1 (4.35%) Windows
1 1 (4.35%) Windows Server
Processes & Methodologies
1 15 (65.22%) ETL
2 13 (56.52%) Data Pipeline
3 11 (47.83%) Business Intelligence
4 9 (39.13%) Migration
5 8 (34.78%) Data Engineering
5 8 (34.78%) DevOps
6 7 (30.43%) Agile
7 6 (26.09%) Data Integration
7 6 (26.09%) Wealth Management
8 4 (17.39%) Collaborative Working
8 4 (17.39%) Data Modelling
8 4 (17.39%) Data Transformation
8 4 (17.39%) Problem-Solving
8 4 (17.39%) Stakeholder Management
8 4 (17.39%) Waterfall
9 3 (13.04%) Coaching
9 3 (13.04%) Data Management
9 3 (13.04%) Decision-Making
9 3 (13.04%) Technical Leadership
9 3 (13.04%) Translating Business Requirements
Programming Languages
1 20 (86.96%) SQL
2 11 (47.83%) Python
3 8 (34.78%) Scala
4 4 (17.39%) Data Analysis Expressions
5 2 (8.70%) PL/SQL
6 1 (4.35%) Shell Script
6 1 (4.35%) T-SQL
Qualifications
1 7 (30.43%) Degree
2 4 (17.39%) CIMA
3 3 (13.04%) Computer Science Degree
Quality Assurance & Compliance
1 7 (30.43%) Data Quality
2 1 (4.35%) Accessibility
System Software
1 1 (4.35%) Apache Flume
Vendors
1 21 (91.30%) Databricks
2 9 (39.13%) Microsoft
3 4 (17.39%) SAP
4 3 (13.04%) Google
5 2 (8.70%) Oracle
6 1 (4.35%) MuleSoft