10 of 10 Permanent Spark SQL Jobs in London

ResponsibilitiesImplement end-to-end Microsoft Fabric solutions based on approved architecture, including OneLake, Lakehouse, Warehouse, Data Factory pipelines, Dataflows Gen2, Notebooks (PySpark/Spark SQL) and Real-Time IntelligenceBuild medallion architectures (Bronze/Silver/Gold), including incremental loads, change data capture, slowly changing dimensions … error handlingSet up ingestion from diverse sources (SQL databases, APIs, files, SaaS applications, SAP, streaming sources) using pipelines, shortcuts and mirroringWrite clean, reusable and tested code in PySpark, Spark SQL, T-SQL and KQL, optimizing Delta tablesAdd data quality checks, validation rules ...

bonus and an extensive benefits package. What you'll be doingDesign, build and maintain scalable data pipelines and data products using Databricks, PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. … mature a growing data function. What you need to bring"Senior" level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc)Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data ...

extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data ...

interfaces, Search) Evaluating the performance and applicability of multiple tools against customer requirements Technical Skills and expertise required Experience on ADLS, Azure SQL DB and Datawarehouse Strong working experience in Implementation of Azure cloud components using Azure Data Factory , Azure Data Analytics, Azure Data Lake, Azure Data Catalogue … data applications on cloud solutions on Azure. Hands on experience in performance tuning and optimizing code running in Databricks environment. Good understanding of SQL, T-SQL and/or PL/SQL Should have experience working in Agile projects with knowledge in Jira. Good ...

clients. This is a hands-on, client-facing role for someone who enjoys getting stuck into the data. Key Responsibilities Write and run SQL to check, profile and reconcile data across source systems, staging, data warehouse and lakehouse layers Test ETL/ELT pipelines end to end, validating … than lagging behind Our QA tooling and processes improve because gaps are named and fixed, not just worked around Competencies & Behaviours Essential Strong SQL: confident writing joins, CTEs, aggregations and window functions to interrogate, profile and reconcile data Data pipeline testing: experience testing ETL/ELT processes ...

capacity (F-SKU) sizing and licensingDesign end-to-end Microsoft Fabric solutions: OneLake, Lakehouse, Warehouse, Data Factory pipelines, Dataflows Gen2, Notebooks (PySpark/Spark SQL), Real-Time Intelligence (Eventstreams, Eventhouse/KQL) and Power BI + Power BI CoPilotDesign governance, security and operating frameworks: data governance … operating and support models and handover documentationPlan and run migrations from older platforms (Azure Synapse, Azure Data Factory, SSIS/SSAS, on-premises SQL Server, Snowflake, Databricks, Power BI Premium P-SKUs, Tableau/Qlik, etc.) to FabricPlan Fabric capacity (F-SKUs) and look after cost, performance ...

your duties, you will be responsible for: Design, build, and deploy scalable data pipelines using Azure Databricks , Azure Data Factory , and Azure SQL Database . Optimise Databricks environments for performance, cost, and security. Develop and maintain data integration , ETL/ELT processes, and data quality frameworks. Translate business … Deep expertise in Azure Databricks , Azure Data Factory , Azure Synapse , and Azure Data Lake . Hands-on experience with Spark (PySpark/SparkSQL) , ETL/ELT design, and data pipeline orchestration . Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts. Experience working ...

data quality and governance. Create and maintain ETL/ELT processes using Data Factory, Data Pipelines, Notebooks, and Spark. Develop, maintain, and enhance SQL scripts, stored procedures, views, and data models. Monitor and troubleshoot performance issues within Fabric environments. Implement security, access controls, and governance standards. Support … architects, and stakeholders to translate requirements into technical solutions. Produce technical documentation and knowledge transfer materials. What you will have Technical Skills Strong SQL development skills. Experience designing and implementing ETL/ELT solutions. Knowledge of Lakehouse and Data Warehouse architectures. Experience with Data Pipelines, Dataflows Gen2 ...

data quality and governance. Create and maintain ETL/ELT processes using Data Factory, Data Pipelines, Notebooks, and Spark. Develop, maintain, and enhance SQL scripts, stored procedures, views, and data models. Monitor and troubleshoot performance issues within Fabric environments. Implement security, access controls, and governance standards. Support … architects, and stakeholders to translate requirements into technical solutions. Produce technical documentation and knowledge transfer materials. What you will have Technical Skills Strong SQL development skills. Experience designing and implementing ETL/ELT solutions. Knowledge of Lakehouse and Data Warehouse architectures. Experience with Data Pipelines, Dataflows Gen2 ...

support data quality and governance.Create and maintain ETL/ELT processes using Data Factory, Data Pipelines, Notebooks, and Spark.Develop, maintain, and enhance SQL scripts, stored procedures, views, and data models.Monitor and troubleshoot performance issues within Fabric environments.Implement security, access controls, and governance standards.Support CI/CD deployment processes … business analysts, architects, and stakeholders to translate requirements into technical solutions.Produce technical documentation and knowledge transfer materials. What you will have Technical SkillsStrong SQL development skills. Experience designing and implementing ETL/ELT solutions. Knowledge of Lakehouse and Data Warehouse architectures. Experience with Data Pipelines, Dataflows Gen2 ...