5 of 5 Remote/Hybrid Delta Lake Jobs in Berkshire

data into structured, analytics-ready formats. Optimise pipeline performance and ensure high availability of data services. Infrastructure & Architecture Architect and deploy scalable data lake solutions using Azure Data Lake Storage. Implement governance and security measures across the platform. Leverage Terraform or similar IaC tools for controlled and reproducible … deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations and CI/CD pipelines for Databricks deployments. Monitoring & Operations Implement monitoring solutions using Azure ...

experience with some of the following and be able to learn new technologies or concepts quickly: Deep experience with Databricks on Microsoft Azure, including Delta Lake, Unity Catalog, Delta Live Tables, PySpark and Databricks SQL, alongside complementary services such as Azure Data Factory, Synapse Analytics, and Microsoft … Fabric. Designing and implementing Lakehouse architectures in Databricks to unify data warehousing and data lake capabilities, enabling scalable analytics and governance. Building and orchestrating Databricks notebooks, Delta Live Tables, Databrick Jobs and integration with Azure Data Factory for enterprise-scale data ingestion and transformation. Proficiency in Python ...

experience with some of the following and be able to learn new technologies or concepts quickly: Deep experience with Databricks on Microsoft Azure, including Delta Lake, Unity Catalog, Delta Live Tables, PySpark and Databricks SQL, alongside complementary services such as Azure Data Factory, Synapse Analytics, and Microsoft … Fabric. Designing and implementing Lakehouse architectures in Databricks to unify data warehousing and data lake capabilities, enabling scalable analytics and governance. Building and orchestrating Databricks notebooks, Delta Live Tables, Databrick Jobs and integration with Azure Data Factory for enterprise-scale data ingestion and transformation. Proficiency in Python ...

have, not required Do not rule yourself out if you do not have these. They are a bonus, not a checklist. Experience with Delta Lake, Iceberg, Spark or Databricks, Palantir Experience optimising LLM serving with tools such as vLLM, TGI or TensorRT LLM Search and ranking experience. ...

delivery teams with design guidance and architecture governance. Identify risks, dependencies, and architectural decisions requiring senior approval. Drive data modelling best practices including Lakehouse, Delta Lake & dimensional modelling. Define standards, patterns, and secure-by-design principles across solutions. Maintain architecture artefacts and ensure accessibility across the programme. ...