2 of 2 Spark SQL Jobs in London

technical designs, data models, ETL/ELT pipelines, and engineering implementation. Provide architectural guidance to data engineers working on Azure Databricks, ADF, PySpark, SQL, and Delta Lake. Collaborate with enterprise architecture, solution architecture, security, infrastructure, DevOps, and business teams. Define CI/CD, DevOps, deployment, monitoring, and operational … modelling, dimensional modelling, canonical data models, and enterprise data models. Strong experience in designing ETL/ELT data pipelines. Good understanding of PySpark, Spark SQL, Python, and SQL. Strong knowledge of data warehouse, data lake, data mart, and semantic layer architecture. Experience in defining data quality ...

Technical Requirements Expert-level dbt Core/Cloud proficiency, including advanced Jinja, macro development, and incremental materialization strategies. Deep experience with Databricks (Unity Catalog, SparkSQL) and Apache Iceberg on AWS S3. Ability to reverse-engineer Legacy Teradata/BTEQ and write highly optimized SQL for Databricks. Hands ...