16 of 16 Remote Delta Lake Jobs

dense/hybrid retrieval, reranking, and semantic search. Data Platform Engineering: Build scalable, streaming, and batch data processing pipelines across modern cloud lakehouse architectures (Delta Lake, streaming architectures, and unified governance layers). Systematic Evaluation & Benchmarking: Establish continuous model evaluation, reproducibility checks, tracing, and benchmarking frameworks (MLflow … Agentic Frameworks: Proven delivery with LangGraph, AutoGen, CrewAI, smolagents, or Model Context Protocol (MCP) integrations. Lakehouse & Data Engineering: Strong background with enterprise Databricks environments (Delta Lake, Unity Catalog, Delta Live Tables/DLT, Spark Structured Streaming, or Autoloader). Cloud & Model Ecosystem: Broad exposure to Azure cloud ...

data solutions, from conceptual and logical models through to physical implementation Building and maintaining scalable pipelines and lakehouse solutions using Azure Databricks, Spark and Delta Lake Designing and optimising Snowflake environments for performance, scalability and cost Supporting batch, streaming and real-time data processing, including high-volume transactional … environments Hands-on, current engineering skills: you still write the code, not just the diagrams Deep expertise with Azure Databricks, Spark (PySpark/Scala), Delta Lake and Lakehouse architecture Strong Snowflake architecture and optimisation experience Experience with Azure Data Factory, ADLS Gen2, Synapse Analytics and/or Microsoft ...

supporting colleagues. What you'll do ·Build and optimise pipelines using Azure Databricks, PySpark, SQL, ADF and ADLS Gen2. ·Develop governed lakehouse solutions using Delta Lake, medallion architecture and Unity Catalog. ·Own Power BI semantic models, DAX measures and row-level security, ensuring reporting is accurate, secure …/CD. ·Mentor colleagues and share good practice across the team. What you'll bring ·Strong production experience with Databricks, PySpark, Delta Lake and Unity Catalog. ·Hands-on Power BI expertise, including data modelling, DAX, RLS and performance optimisation. ·Strong SQL skills and experience with Azure data services. ...

deployment Worked across monitoring, logging and operational alerting Completed data-quality and reconciliation controls API/file/SFTP integration patterns Nice to have: Delta Lake/Delta MERGE Databricks Asset Bundles Azure DevOps MLOps/model deployment work history SFMC or Comarch integration Customer decisioning/ ...

lead engineer experience on enterprise scale Databricks platforms on Azure Strong hands on data engineering background with Python or PySpark and advanced SQL Delta Lake, Unity Catalog, medallion architecture, Databricks Workflows and Auto Loader Declarative pipelines (Delta Live Tables or Lakeflow) and Structured Streaming CI/… Databricks using Databricks Asset Bundles, Azure DevOps or GitHub Actions, and Terraform or Bicep Azure Data Factory, Azure SQL, Data Lake Storage Gen2 Entra ID, Key Vault, private endpoints, RBAC Experience of data governance and lineage, ideally with Microsoft Purview UK GDPR and DPA 2018 in practice, including public ...

experience designing, building and operating production data platforms using Azure Databricks and Azure. Strong hands-on expertise with Apache Spark, PySpark, Spark SQL and Delta Lake. Experience implementing and managing Unity Catalog, including governance, lineage and access controls. Proven delivery of lakehouse and Medallion architectures. Advanced Python ...

level discussion and technical detail Hands on implementation experience, ideally from a data engineering or developer background Databricks at architecture level on Azure, including Delta Lake, Unity Catalog and Spark optimisation Lakehouse and medallion patterns, batch and streaming ingestion, event driven design Conceptual, logical and physical data modelling. … Dimensional and data vault both useful Advanced SQL, Python or PySpark Declarative pipelines (Delta Live Tables or equivalent), Databricks Workflows, Auto Loader and Structured Streaming Microsoft Purview for data governance, cataloguing and lineage Azure Data Factory, Azure SQL, Data Lake Storage Gen2 Azure DevOps or GitHub Actions, Databricks ...

Databricks Define and enforce platform standards, ensuring compliance with data governance and security policies Build and optimize Databricks streaming workloads, including Structured Streaming and Delta Live Tables Collaborate with cross-functional teams to translate requirements into robust technical solutions Drive platform performance tuning, cost optimization and monitoring strategies … 3+ years in a tech lead capacity Strong hands-on expertise in Databricks for enterprise-scale workloads Proficiency in PySpark, Spark Structured Streaming, and Delta Lake Solid programming skills in Python and demonstrated SQL performance optimization experience Deep understanding of Azure Data Platform services and cloud-native engineering ...

Design, build and enhance a secure, scalable and reliable Databricks platformDevelop and maintain batch and real-time data pipelines using Databricks, Apache Spark and Delta LakeContribute to platform architecture, engineering standards and best practices across the data estateWork with Product, Analytics and Data Science teams to deliver effective data … continuous improvement across the engineering teamJob Requirements: Significant experience building and supporting scalable, production-grade data platformsStrong hands-on expertise with Databricks, Apache Spark, Delta Lake, Python and SQLGood understanding of cloud-based data engineering, ideally within Microsoft Azure environmentsExperience with Infrastructure as Code tools such as Terraform ...

best practices. Skills & Experience Strong hands-on experience with Azure Databricks. Experience building data pipelines using Python and/or PySpark. Understanding of Delta Lake, lakehouse architectures and modern data platform design. Experience working with Unity Catalog for data governance and access management. Strong SQL development, optimisation ...

best practices. Skills & Experience Strong hands-on experience with Azure Databricks. Experience building data pipelines using Python and/or PySpark. Understanding of Delta Lake, lakehouse architectures and modern data platform design. Experience working with Unity Catalog for data governance and access management. Strong SQL development, optimisation ...

best practices. Skills & Experience Strong hands-on experience with Azure Databricks. Experience building data pipelines using Python and/or PySpark. Understanding of Delta Lake, lakehouse architectures and modern data platform design. Experience working with Unity Catalog for data governance and access management. Strong SQL development, optimisation ...

personNice to have, not requiredDo not rule yourself out if you do not have these. They are a bonus, not a checklist. Experience with Delta Lake, Iceberg, Spark or Databricks, PalantirExperience optimising LLM serving with tools such as vLLM, TGI or TensorRT LLMSearch and ranking experience. For example ...

controls and records needed to support auditing, data traceability and compliance requirements. Performance and operational management Investigate and improve the performance of Spark workloads, Delta tables and SQL compute. Review resource consumption and identify ways to reduce unnecessary platform and cloud expenditure. Set up monitoring and alerting to identify … mentoring across the team. What we are looking for Practical experience managing Databricks in a production environment. A strong understanding of Apache Spark, Delta Lake and Unity Catalog. Experience working with one or more major cloud providers, including storage, networking and identity management. Confidence using Terraform ...

Data Warehousing . Strong background in incident resolution, requests, changes and problem-solving within SLAs. Hands-on experience with Spark, Data Architecture, SQL and Delta Lake. Cloud development experience (AWS, GCP or Azure). Working knowledge of Medallion Architecture. Demonstrated capability in implementing and supporting pipelines in demanding environments. ...

designs, data flows and integration specifications. Provide technical leadership across architecture and engineering teams. Essential Skills: Strong AWS and Databricks expertise, including Lakehouse and Delta Lake. Extensive data modelling and enterprise data architecture experience. Supply chain planning, demand forecasting and inventory optimisation. Knowledge of SCOR, data mesh and event ...