51 to 75 of 76 Remote/Hybrid Delta Lake Jobs

development, and technology governance Design of scalable, secure, and cost-optimised cloud data platforms Databricks & Azure Expertise Deep understanding of Databricks Serverless, SQL Warehouses, Delta Lake, Unity Catalog, and platform governance Azure networking, identity, security, and private connectivity architecture Enterprise-scale performance and workload optimisation FinOps Leadership Design ...

discussions, including vendor conversations and the technical sections of proposals Desirable Experience with NHS or other routine healthcare data at patient level Unity Catalog, Delta Lake and medallion structure, bronze to silver to gold Exposure to dbt and dimensional modelling Experience contributing to vendor or partner relationships, such ...

scalable data solutions. Strong hands-on experience with Microsoft Fabric and Databricks. Strong programming capability across PySpark and T-SQL. Experience working with Delta Lake, Lakehouse architectures and modern cloud data platforms. Understanding of data quality frameworks, testing approaches and governance principles. What's on Offer Salary ...

into technical solutions Requirements: Enterprise data warehouse architecture and design Microsoft Fabric, Azure Data Platform technologies and modern Lakehouse architectures Databricks, Unity Catalog and Delta Lake PySpark, Databricks, SQL Medallion Architecture implementation Data modelling approaches including Star Schema and Data Vault Data governance, security and access management ...

help the team do better and be better, again and again Write maintainable, testable code and enjoy providing code reviews Be experienced with Databricks, Delta Lake and Lakehouse architecture for efficient data management; experience with ETL processes and optimizing data pipelines for performance Be strong in Python ...

PyTorch.Experience with data visualisation tools such as PowerBI and TableauExperience deploying models into production using Azure servicesExposure to data engineering fundamentals (e.g., data pipelines, delta lake)Knowledge of experience tracking tools (MLFlow, Azure ML tracking)Connect to your business – TaxInnovation and tax consultancy may not seem ...

Disclosure and Barring Service)Mandatory SkillsAzure Databricks Platform EngineeringHands-on implementation and troubleshooting of Azure DatabricksDatabricks Serverless configuration and workload optimisationDatabricks SQL and Delta Lake developmentCluster management, compute policies, autoscaling, and workload monitoringData EngineeringStrong Python, PySpark, and SQL development skillsDesign and build ingestion and transformation pipelinesFull and incremental ...

analytical modelling and reporting. Familiarity with cloud platforms (Azure or AWS), particularly in hybrid data environments. Experience working with data Lakehouse technologies (e.g. Delta Lake, Databricks, or similar platforms). Be able to travel to programme and customer meetings across the UK as required. Security cleared or ability ...

working through ambiguity. Desirable MLOps excellence: reproducible pipelines, model versioning, CI/CD, observability, and automated evaluation. Data engineering: proficiency with Databricks, Apache Spark, Delta Lake, MLflow, and SQL; experience integrating datasets and maintaining data quality. Model training and optimisation: experience with pre‐training, fine‐tuning, distributed training ...

Experience owning delivery roadmaps, OKRs, and budgets for an architecture or engineering function. Strong technical foundation across the data platform stack: Databricks lakehouse architecture (Delta Lake, medallion patterns), Azure Synapse (star schemas, dedicated pools), and data pipeline orchestration (ADF or Databricks Workflows). Experience with data quality frameworks ...

Azure Databricks Data Engineer to design, develop, deploy, and optimize scalable data solutions on Microsoft Azure. You will build production-grade data pipelines with Delta Lake, Unity Catalog, and Databricks tools. The role requires hands-on expertise in PySpark, Python, SQL, and Spark architecture, plus experience with GitLab … based CI/CD, Databricks Asset Bundles, and Azure Data Lake Storage. #J-18808-Ljbffr ...

matters? The future of cybersecurity starts with you.About the Team:The data engineering team is on a mission to create a hyper scale data lake, which helps find bad actors and stop breaches. The team builds and operates systems to centralize all of the data the falcon platform collects … from you This team would be responsible for driving an application-based, self-service Data Replay that would leverage data within our security data lake and ensure we provide complete Business Continuity and Resiliency for downstream services or peers that ingest and process data within the CrowdStike Cloud Platform.What ...

will build robust ETL/ELT pipelines with Azure Data Factory, Synapse and Databricks, design real-time and batch ingestion, and modern data platforms (Delta Lake, lakehouse, ADLS Gen2). Hybrid-working is offered. #J-18808-Ljbffr ...

enable analytics, reporting, and AI initiatives across Pricing, Underwriting and Claims. You will design and build scalable data pipelines on Azure Databricks and Delta Lake, with governance, security, and real-time processing at scale. The role sits in the Enterprise Data Team and requires strong SQL and PySpark ...

observability Knowledge of Auto Loader, Lakeflow Connect and batch, streaming or incremental ingestion patterns Production-level PySpark and SQL development skills Strong knowledge of Delta Lake, including MERGE, schema evolution, OPTIMIZE, ZORDER and VACUUM Experience with Databricks Workflows, orchestration and CI/CD Understanding of monitoring, alerting, replay ...

covering the full lifecycle, discovery, design and build, through to production release and support Deep hands‐on experience of the core platform: Unity Catalog, Delta Lake, workflows and jobs, declarative pipelines, SQL warehouses, cluster and compute configuration Migrated data at scale off on‐prem or legacy platforms ...

Python, Data Modelling, Data Warehousing.Strong background in incident resolution, requests, changes and problem‐solving within SLAs.Hands‐on experience with Spark, Data Architecture, SQL and Delta Lake.Cloud development experience (Azure/Fabric).Working knowledge of Medallion Architecture.Demonstrated capability in implementing and supporting pipelines in demanding environments.Strong communication skills and confidence ...

practices, and escalating, managing and reporting control issues with transparency.Skills & ExperienceEssentialProven in-depth commercial experience in the following key areas:Databricks – Lakehouse architecture, Spark, Delta Lake, workflows and performance tuning.Apache Spark & Scala – High-performance distributed data processing and optimisation.Cloud Data Engineering (AWS) – Cloud-native data platforms and distributed … Applications DevelopmentDemonstrated knowledge of client core business functionsDemonstrated leadership, project management, and development skillsRelationship and consensus building skillsPreferredKafka/Event-Driven ArchitectureApache Iceberg/Delta LakePythonAI/ML & LLM integrationFinancial Services/Capital MarketsEducation:Bachelor’s degree/University degree or equivalent experienceMaster’s degree preferredWhat we’ll provide ...

data/analytics platforms and/or MLOps capabilities in a complex enterprise environment. Hands-on experience operating Databricks (workspace administration, cluster policies, jobs, Delta Lake, Unity Catalog, access controls, and production support) Experience implementing MLOps practices across the model lifecycle (CI/CD, versioning, monitoring, reproducibility) Strong ...

engineering standards, ensuring high-quality testing, documentation, code quality, reproducibility, and operational reliability. Your skills and experience: Experience with ML platforms including Databricks, MLflow, Delta Lake, and cloud environments. Proficient in Python, PySpark, and SQL, following production coding best practices. Understanding of data, distributed ML pipelines, and model ...

managing data/analytics platforms and/or MLOps capabilities in a complex enterprise environment.Hands-on experience operating Databricks (workspace administration, cluster policies, jobs, Delta Lake, Unity Catalog, access controls, and production support)Experience implementing MLOps practices across the model lifecycle (CI/CD, versioning, monitoring, reproducibility) Strong ...

Metaflow or similar workflow tools.Kafka, Pulsar, Redis Streams, RabbitMQ or other event-driven technologies.Triton Inference Server, Ray Serve, TorchServe, BentoML or Seldon Core.DVC, LakeFS, Delta Lake, Feast or other data-versioning and feature-store technologies.Terraform, Pulumi, CloudFormation or Ansible.GPU cluster management or high-performance computing.Aerial, satellite ...

consolidation.• Ensure efficient scheduling and orchestration of jobs using Databricks Workflows• Collaborate with data scientists and platform teams to improve query performance on Delta Lake.• Develop automated monitoring and alerting mechanisms using Databricks REST APIs, Prometheus, or Azure Monitor.• Maintain best practices for data engineering performance, testing, and documentation.Technical … Skills Required• Strong proficiency with Databricks, Apache Spark (SQL, PySpark, Scala), and Delta Lake.• Deep understanding of Spark execution plans, shuffle optimization, and data partitioning strategies.• Experience tuning job configurations (e.g., memory management, parallelism, caching, adaptive query execution).• Familiarity with orchestration tools: Databricks Workflows, Airflow, or Azure Data ...

cloud governance, security and data protection controls. A strong track record of integrating AWS architectures with modern data platforms such as Databricks Lakehouse or Delta Lake, and resolving complex cross-team architecture issues. Experience chairing or presenting at a Technical Design Authority or equivalent architecture governance forum, securing ...

Driven Design (DDD) Exceptional command of the English language Nice-to-Have We’d also love experience with: Version control for data and artefacts: DeltaLake DVC MLFlow Designing and operating cloud-based systems AWS and/or Databricks Messaging platforms Infrastructure as Code tools such as: Terraform CloudFormation Pulumi Kubernetes ...