14 of 14 Spark SQL Jobs in London

operate scalable data pipelines using Microsoft Fabric (OneLake/Delta Lake, Data Factory, Synapse Data Engineering). Develop batch processing solutions using PySpark, Spark SQL, Python, and SQL , with a focus on performance, resilience, and data quality. Support reporting and MI use cases, including … equivalent. Active SC clearance (BPSS minimum). Strong hands-on experience with Microsoft Fabric and Azure data services. Advanced skills in PySpark, Spark SQL, Python, and SQL . Experience delivering data engineering solutions in government or similarly regulated environments. CI/CD, DevOps ...

British Citizenship, or ILR (Indefinite Leave to Remain), or UK Settlement Visa. We do not provide visa sponsorship. Key Requirements: Must have skills: -Spark & Scala Nice to have skills: -Spark Streaming, Hadoop, -Hive, -SQL, -Sqoop, - Impala Detailed Job Description: At least 8+ years … fundamental Data Structures and their usage At least 8+ years of experience in designing and developing large scale, distributed data processing pipelines using Apache Spark and related technologies. Having expertise in Spark Core, Spark SQL and Spark Streaming. Experience with ...

British Citizenship, or ILR (Indefinite Leave to Remain), or UK Settlement Visa. We do not provide visa sponsorship. Key Requirements:Must have skills:-Spark & Scala Nice to have skills:-Spark Streaming, Hadoop,-Hive,-SQL,-Sqoop,- Impala Detailed Job Description: At least 8+ years … fundamental Data Structures and their usage At least 8+ years of experience in designing and developing large scale, distributed data processing pipelines using Apache Spark and related technologies. Having expertise in Spark Core, Spark SQL and Spark Streaming. Experience with ...

Developing and deploying the Lakehouse platform using Databricks Building automation for cluster management, integration runtimes, and infrastructure components Leading implementation with Databricks, PySpark, Spark SQL, and Azure Data Factory Creating APIs for integration and automation Driving CI/CD, automated testing, and DevOps best practice Your … Skills and Experience You will have strong capability in: Azure and Databricks engineering PySpark, SQL, Python, and Spark SQL DevOps best practices including CI/CD, automation, and testing Infrastructure as code such as Terraform Stakeholder communication and cross-team collaboration How to Apply ...

data services across the organisation. Key Responsibilities Design, build, and maintain data pipelines using Databricks Develop ETL/ELT processes with PySpark and Spark SQL Transform and model structured and semi-structured datasets Improve performance, reliability, and cost efficiency of data workloads Ensure compliance with data … routes Required Skills & Experience Active SC Clearance - mandatory and non-negotiable Strong hands-on experience with Databricks in production environments Expert knowledge of Apache Spark , including PySpark and Spark SQL Proficiency with Python for data engineering Experience delivering solutions on cloud data platforms ...

combat travel-based issues and advice on expected interview topics/questions. Main Duties and Responsibilities Write clean and testable code using SQL and Python scripting languages, to enable our customer data products and business applications Build and manage data pipelines and notebooks, deploying code in a structured … Promote an innovative thinking process and encourage it in others. Working within the agile framework at Chambers Skills and Experience Strong proficiency in SQL , including Spark SQL and MS SQL Server , for querying, data manipulation, and performance optimization. Hands-on experience with ...

Data Warehouse capabilities. Design, manage, and optimise data storage within OneLake , ensuring well-governed, well-structured, and shareable data assets. Develop scalable Spark (PySpark/Spark SQL) data transformations through Fabric notebooks to support high-volume processing. Create, orchestrate, and monitor robust data ingestion … expecting to see: Required Skills Proven expertise across core Microsoft Fabric workloads and the unified analytics environment. Strong proficiency in Python, SQL , and Spark-based processing frameworks. Solid understanding of lakehouse architecture , including Delta/Parquet formats and data modelling. Experience designing secure, compliant, and well ...

relevant in data governance, compliance, or automation programs. Strong proficiency in Python for data engineering and automation. Hands-on experience with PySpark and Spark SQL in production environments. Solid knowledge of Hive, Impala, HDFS, and Parquet . Advanced SQL skills; experience with Oracle databases ...

Experience 5+ years' experience working as a Data Engineer or in a similar role Proven experience with cloud-based data platforms (Azure, AWS, SQL, Snowflake, Springserv); Strong proficiency in Spark SQL and PySpark , including complex transformations Experience building ETL/ELT pipelines using tools ...

years of relevant experience in data analytics, data quality, reporting controls, or data transformation programs (preferably in financial services). Proficiency in SQL (advanced querying, performance tuning, reconciliation logic). Strong proficiency in Python for data analysis and automation (pandas, data validation frameworks, scripting). Experience supporting … supporting Tableau dashboards (data validation, extract refresh checks, reconciliation to source). Nice-to-Have Ideally, you have experience with tools including PySpark, Spark SQL, Hive, Impala, HDFS, Parquet, and Oracle databases. Experience in regulated or enterprise data environments (investment banking, risk, finance, compliance). Exposure ...

Deep understanding of Delta Lake, Medallion architecture, Lakehouse patterns, distributed compute, data modelling, and performance optimisation. Strong programming skills in Python, PySpark, and Spark SQL, with experience implementing CI/CD pipelines (Azure DevOps, branching/merging, automation). Solid knowledge of data governance, lineage, access ...

London or London Market insurance sector Deep understanding of Delta Lake, Medallion architecture, Lakehouse patterns and data modelling. Strong Python, PySpark and Spark SQL capability Solid understanding of data governance Proven track record of ownership, delivery and operating effectively in fast-paced environments. What ...

Your new company One of the most influential Central Government Organisations in the current economic climate Your new role Data Scientist - SC Cleared - SQL, Python & R What you'll need to succeed My client is looking for an Analytical Data Scientist, leading/working alongside a team … Existing SC or DV Clearance. Essential Criteria: Experience of delivering high-quality coding projects that make use of at least two of: SQL, Python & R Experience of engaging with stakeholders across government to scope and deliver impactful analysis Experience of leading teams through complex data science projects ...

SAS2PY) and manual refactoring. Pipeline Engineering: Design, build, and troubleshoot complex ETL/ELT workflows and data marts on AWS. Performance Tuning: Optimise Spark workloads for execution efficiency, partitioning, and cost-effectiveness. Quality Assurance: Implement clean coding principles, modular design, and robust unit/comparative testing to ensure …/CD integration, and comprehensive technical documentation. Technical Requirements PySpark (P3): 5+ years of hands-on experience writing scalable, production-grade PySpark/Spark SQL. AWS Data Stack (P3): Strong proficiency in EMR, Glue, S3, Athena, and Glue Workflows. SAS Knowledge (P1): Solid foundation in SAS to enable ...