3 of 3 Remote Spark SQL Jobs in the North of England

modeling, ETL/ELT development, and collaborative engineering practices. Key Responsibilities * Design, develop, and maintain scalable data pipelines using Azure Databricks (Python, PySpark, SQL). * Build and optimize ETL/ELT workflows that ingest data from various on-prem and cloud-based sources. * Work with Azure services including … Azure Data Lake Storage, Azure Data Factory, Azure Synapse Analytics, Azure SQL, and Event Hub. * Implement data quality validation, monitoring, metadata management, and governance processes. * Collaborate closely with data architects, analysts, and business stakeholders to understand data requirements. * Optimize Databricks clusters, jobs, and runtimes for performance and cost ...

quality data solutions. Key Responsibilities: Develop, optimise, and maintain ETL/ELT pipelines within Databricks Build reliable data ingestion frameworks using PySpark and Spark SQL Design well-structured data models across medallion/lakehouse architecture Work with DevOps teams to automate deployments using CI/ ...

quality data solutions. Key Responsibilities: Develop, optimise, and maintain ETL/ELT pipelines within Databricks Build reliable data ingestion frameworks using PySpark and Spark SQL Design well-structured data models across medallion/lakehouse architecture Work with DevOps teams to automate deployments using CI/ ...