4 of 4 Delta Lake Jobs in the South West

maintain Databricks pipelines (batch and incremental) using PySpark and SQL. Orchestrate end-to-end workflows with Azure Data Factory . Develop and optimise Delta Lake tables (partitioning, schema evolution, vacuuming). Implement Medallion Architecture (Bronze, Silver, Gold) for transforming raw data into business-ready datasets. Apply robust monitoring … development skills for large-scale data engineering. Proven experience with Databricks pipelines and workflow management. Expertise in Azure Data Factory orchestration. Solid knowledge of Delta Lake and Lakehouse principles. Hands-on experience with SQL for data transformation. Familiarity with Azure services (ADLS/Blob, Key Vault, SQL). ...

umbrella inside IR35 Duration: Initial 3-month, subject to extension Hours: Full-time position Responsibilities include: Develop, optimise, and debug Databricks notebooks, Delta Lake workflows, and Genie-bot automations Design, maintain, and enhance ETL pipelines , ensuring reliability and performance Validate, transform, and orchestrate data using Python … engineer, document, and communicate existing data flows and pipeline structures for handover to internal teams Requirements: Strong hands-on experience with Databricks , including notebooks, Delta Lake, and Genie bots Proven background in designing and maintaining ETL pipelines , including debugging and optimisation Advanced Python skills for transformation, automation ...

Python or Scala. Hands-on use of cloud data stacks (e.g. SageMaker Notebooks, S3, Glue, Athena) and modern storage formats or frameworks (e.g. Parquet, Delta Lake, Fabric). Experience with DevOps/DataOps tooling and practices (e.g. Terraform) and testing data pipelines, including end-to-end, data quality ...

Responsibilities Analyze and document existing data pipelines, including diagrams and data-flow mapping. Develop and automate ETL pipelines using Python. Work with Databricks notebooks, Delta Lake, and Databricks Genie bots. Design, optimize, and debug ETL pipelines. Use SQL for querying, validation, and optimization in the Lakehouse environment. Person ...