2 of 2 Permanent Databricks Jobs in South London

zones (cleansed, conformed business layers) while securely handling identifiable data. Scalable Data Processing: Develop, tune, and monitor large-scale batch processing pipelines using Azure Databricks (Spark-based processing) and PySpark. Orchestration & Workflow: Build automated data workflows using Microsoft Fabric Data Factory and Azure Data Factory to move data efficiently across … with practical exposure to the Microsoft Fabric ecosystem (OneLake, Data Factory, Fabric Lakehouse).Big Data Processing: Deep technical expertise with Apache Spark/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python (PySpark).Security & Compliance: Direct ...

delivered as part of a key strategic program. Technical leader for the development and delivery of the new Underwriting Datamart based on Databricks platform. Champion the adoption of Cloud computing for Underwriting, driving seamless integration and best practices. Develop and maintain automated ETL pipelines to increase efficiencies within the target …/or projects with strong stakeholder management skills. Programming experience essential with direct exposure to Python and SQL. An understanding and experience of the Databricks platform desirable. Experience of working with, and knowledge of, Data Science practices desirable. Experience of automation and CI/CD pipelines desirable. Self-starter ...