12 of 12 Spark SQL Jobs in London

extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data ...

extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data ...

hands-on Senior Databricks Solutions Architect to support a complex, high-performance data platform within a global organisation. Youll design and optimise Spark workloads, build production streaming … pipelines and deliver scalable Databricks solutions using Delta Live Tables, Workflows, Unity Catalog, Python, PySpark, Terraform and Databricks Asset Bundles. Key requirements: Advanced Apache Spark and Spark SQL optimisation Production Spark Structured Streaming experience Deep Databricks platform knowledge Strong Python and PySpark ...

Experience Data Engineering & Cloud Extensive experience designing, building and operating production data platforms using Azure Databricks and Azure. Strong hands-on expertise with Apache Spark, PySpark, Spark SQL and Delta Lake. Experience implementing and managing Unity Catalog, including governance, lineage and access controls. Proven … delivery of lakehouse and Medallion architectures. Advanced Python and SQL skills for scalable, production-grade solutions. Experience building modern batch and real-time data pipelines. Strong understanding of API design and data product delivery mechanisms. Experience deploying Databricks solutions using CI/CD pipelines, Azure DevOps and/ ...

interfaces, Search) Evaluating the performance and applicability of multiple tools against customer requirements Technical Skills and expertise required Experience on ADLS, Azure SQL DB and Datawarehouse Strong working experience in Implementation of Azure cloud components using Azure Data Factory , Azure Data Analytics, Azure Data Lake, Azure Data Catalogue … data applications on cloud solutions on Azure. Hands on experience in performance tuning and optimizing code running in Databricks environment. Good understanding of SQL, T-SQL and/or PL/SQL Should have experience working in Agile projects with knowledge in Jira. Good ...

experience with Power BI or similar tools such as Tableau or Qlik for data storytelling, data visualisation and reporting Strong programming skills in SQL or Spark SQL Strong numeracy and statistical modelling expertise Familiarity with DevOps, JIRA and GitHub tools ...

clients. This is a hands-on, client-facing role for someone who enjoys getting stuck into the data. Key Responsibilities Write and run SQL to check, profile and reconcile data across source systems, staging, data warehouse and lakehouse layers Test ETL/ELT pipelines end to end, validating … than lagging behind Our QA tooling and processes improve because gaps are named and fixed, not just worked around Competencies & Behaviours Essential Strong SQL: confident writing joins, CTEs, aggregations and window functions to interrogate, profile and reconcile data Data pipeline testing: experience testing ETL/ELT processes ...

your duties, you will be responsible for: Design, build, and deploy scalable data pipelines using Azure Databricks , Azure Data Factory , and Azure SQL Database . Optimise Databricks environments for performance, cost, and security. Develop and maintain data integration , ETL/ELT processes, and data quality frameworks. Translate business … Deep expertise in Azure Databricks , Azure Data Factory , Azure Synapse , and Azure Data Lake . Hands-on experience with Spark (PySpark/SparkSQL) , ETL/ELT design, and data pipeline orchestration . Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts. Experience working ...

manage stakeholders to align vision. Experience using AI and LLMs to benefit Data Engineering workflows. Knowledge in, or demonstrate ability to quickly learn, PySpark, SparkSQL, Delta tables, notebooks, pipelines and Git. An ability to integrate data using REST or oData API endpoints. An ability to implement data tests and monitoring ...

data quality and governance. Create and maintain ETL/ELT processes using Data Factory, Data Pipelines, Notebooks, and Spark. Develop, maintain, and enhance SQL scripts, stored procedures, views, and data models. Monitor and troubleshoot performance issues within Fabric environments. Implement security, access controls, and governance standards. Support … architects, and stakeholders to translate requirements into technical solutions. Produce technical documentation and knowledge transfer materials. What you will have Technical Skills Strong SQL development skills. Experience designing and implementing ETL/ELT solutions. Knowledge of Lakehouse and Data Warehouse architectures. Experience with Data Pipelines, Dataflows Gen2 ...

data quality and governance. Create and maintain ETL/ELT processes using Data Factory, Data Pipelines, Notebooks, and Spark. Develop, maintain, and enhance SQL scripts, stored procedures, views, and data models. Monitor and troubleshoot performance issues within Fabric environments. Implement security, access controls, and governance standards. Support … architects, and stakeholders to translate requirements into technical solutions. Produce technical documentation and knowledge transfer materials. What you will have Technical Skills Strong SQL development skills. Experience designing and implementing ETL/ELT solutions. Knowledge of Lakehouse and Data Warehouse architectures. Experience with Data Pipelines, Dataflows Gen2 ...

support data quality and governance.Create and maintain ETL/ELT processes using Data Factory, Data Pipelines, Notebooks, and Spark.Develop, maintain, and enhance SQL scripts, stored procedures, views, and data models.Monitor and troubleshoot performance issues within Fabric environments.Implement security, access controls, and governance standards.Support CI/CD deployment processes … business analysts, architects, and stakeholders to translate requirements into technical solutions.Produce technical documentation and knowledge transfer materials. What you will have Technical SkillsStrong SQL development skills. Experience designing and implementing ETL/ELT solutions. Knowledge of Lakehouse and Data Warehouse architectures. Experience with Data Pipelines, Dataflows Gen2 ...