1 to 25 of 27 Remote/Hybrid Permanent Delta Lake Jobs

build of large-scale data platforms using Azure Databricks and modern cloud technologiesImplement and optimize ETL workflows and streaming pipelines with PySpark and Delta Live Tables following Lakehouse principlesEnhance performance, manage cloud costs and ensure platform reliability for structured streaming workloadsDefine data governance, security and quality standards to maintain … including advanced optimization, data skew mitigation and query tuningStrong programming skills in Python with knowledge of modern software design principlesPractical experience with structured streaming, Delta Lake and Delta Live TablesProven experience in Lakehouse migration and modernization using open table formats such as Delta Lake ...

components Core Technical Skills Azure Synapse & Data Platform Mandatory hands-on expertise with: Azure Synapse Analytics (Pipelines, Spark Pool, Dedicated SQL Pool) Azure Data Lake Storage Gen2 (ADLS Gen2) Delta Lake on Azure (Synapse Lakehouse patterns) Oracle Golden Gate Replication for real-time source integration Azure Analysis … with config-driven and automation-first pipeline patterns (YAML, PySpark, SQL-driven generation from mapping documents) Databricks & Lakehouse Hands-on experience with Azure Databricks (Delta Live Tables, Unity Catalog preferred) Strong Apache Spark skills (PySpark/Spark SQL) Experience migrating workloads from legacy data warehouse or Synapse environments ...

data solutions, from conceptual and logical models through to physical implementation Building and maintaining scalable pipelines and lakehouse solutions using Azure Databricks, Spark and Delta Lake Designing and optimising Snowflake environments for performance, scalability and cost Supporting batch, streaming and real-time data processing, including high-volume transactional … environments Hands-on, current engineering skills: you still write the code, not just the diagrams Deep expertise with Azure Databricks, Spark (PySpark/Scala), Delta Lake and Lakehouse architecture Strong Snowflake architecture and optimisation experience Experience with Azure Data Factory, ADLS Gen2, Synapse Analytics and/or Microsoft ...

frameworks and model benchmarking methodologiesAdvanced programming skills in Python and SQL with strong knowledge of data modeling and ETL/ELT processesSolid understanding of Delta Lake, Delta Live Tables, Unity Catalog, Spark Structured Streaming and large-scale data processingExperience with Azure Data Factory (ADF), Qlik Replicate ...

best practices across the enterprise data ecosystem. The role leads the design and delivery of scalable, secure, high-performing data products using Databricks, Delta Lake, Unity Catalog and Azure Data Services The role provides technical leadership, ensuring solutions are robust, cost-effective, highly available, and aligned to business … development. To ensure data dictionaries and models are robust and maintained on a regular basis. Technical Skill Expert-level Databricks experience Advanced PySpark development Delta Lake architecture and optimisation Databricks SQL Data warehouse Dimensional modelling Data Lake Databricks Genie Python development Key Skills, Experience and Knowledge Extensive ...

supporting colleagues. What you'll do ·Build and optimise pipelines using Azure Databricks, PySpark, SQL, ADF and ADLS Gen2. ·Develop governed lakehouse solutions using Delta Lake, medallion architecture and Unity Catalog. ·Own Power BI semantic models, DAX measures and row-level security, ensuring reporting is accurate, secure …/CD. ·Mentor colleagues and share good practice across the team. What you'll bring ·Strong production experience with Databricks, PySpark, Delta Lake and Unity Catalog. ·Hands-on Power BI expertise, including data modelling, DAX, RLS and performance optimisation. ·Strong SQL skills and experience with Azure data services. ...

PySpark and SQL Develop reusable ingestion, transformation, validation and enrichment processes Ingest demographic datasets, postcode data, geographic boundaries and reference datasets Build and maintain Delta Lake datasets and manage data lineage Perform postcode enrichment and geographic assignment processing Implement spatial joins, point-in-polygon matching and location-based … criteria Essential Skills & Experience Strong commercial experience with: Databricks Python PySpark SQL Proven experience building and supporting production-grade data pipelines Strong knowledge of Delta Lake, schema management, merge logic and incremental processing Experience handling large datasets within cloud-based environments Practical experience working with geospatial data ...

native data services Data architecture: Lakehouse, medallion, data mesh and domain-oriented patterns Data processing: Apache Spark, SQL and distributed processing technologies Data storage: Delta Lake, Apache Iceberg and open table formats Data pipelines: Enterprise-scale ETL/ELT, batch and streaming patterns Transformation and quality: dbt, Great ...

native data services Data architecture: Lakehouse, medallion, data mesh and domain-oriented patterns Data processing: Apache Spark, SQL and distributed processing technologies Data storage: Delta Lake, Apache Iceberg and open table formats Data pipelines: Enterprise-scale ETL/ELT, batch and streaming patterns Transformation and quality: dbt, Great ...

initiatives.In this role you will· Design and build scalable data pipelines using Databricks, PySpark, SQL, AWS Glue and Lambda.· Develop and manage Data Lake solutions using Bronze, Silver, and Gold data architectures.· Implement batch and real-time data processing using Spark Structured Streaming and Auto Loader.· Optimize Databricks workloads … experience with Databricks, PySpark, SQL, and Apache Spark· Strong experience designing and delivering cloud-based data platforms and lakehouse architectures· Hands-on experience with Delta Lake, Unity Catalog, Databricks Workflows, and performance tuning· Experience building batch and streaming data pipelines· Strong knowledge of data modelling and data product ...

level discussion and technical detail Hands on implementation experience, ideally from a data engineering or developer background Databricks at architecture level on Azure, including Delta Lake, Unity Catalog and Spark optimisation Lakehouse and medallion patterns, batch and streaming ingestion, event driven design Conceptual, logical and physical data modelling. … Dimensional and data vault both useful Advanced SQL, Python or PySpark Declarative pipelines (Delta Live Tables or equivalent), Databricks Workflows, Auto Loader and Structured Streaming Microsoft Purview for data governance, cataloguing and lineage Azure Data Factory, Azure SQL, Data Lake Storage Gen2 Azure DevOps or GitHub Actions, Databricks ...

aggregation patterns) Strongly Preferred Experience with Azure landing zones and CAF (Cloud Adoption Framework) Microsoft Purview for data governance and cataloging Experience with Delta Lake, Spark optimization, data quality frameworks Azure networking certifications or equivalent deep knowledge Container orchestration (AKS) API design and management (API Management, App Gateway ...

initiatives by creating trusted, reusable data foundations. Essential Experience developing data engineering solutions on cloud platforms Strong SQL and Python skills. Experience with Databricks, Delta Lake and Spark (PySpark or Scala). Knowledge of modern data warehousing or lakehouse architectures. Hands-on experience developing production-grade data pipelines. ...

Design, build and enhance a secure, scalable and reliable Databricks platformDevelop and maintain batch and real-time data pipelines using Databricks, Apache Spark and Delta LakeContribute to platform architecture, engineering standards and best practices across the data estateWork with Product, Analytics and Data Science teams to deliver effective data … continuous improvement across the engineering teamJob Requirements: Significant experience building and supporting scalable, production-grade data platformsStrong hands-on expertise with Databricks, Apache Spark, Delta Lake, Python and SQLGood understanding of cloud-based data engineering, ideally within Microsoft Azure environmentsExperience with Infrastructure as Code tools such as Terraform ...

Skills & Experience Commercial experience within Data Engineering and cloud-based data platforms. Strong hands-on experience with the Databricks Lakehouse Platform. Solid understanding of Delta Lake, Medallion Architecture and modern data engineering principles. Experience designing and delivering enterprise-scale data solutions. Strong Python development skills. Experience working with ...

reviews and CI/CD.The ability to understand customer requirements and translate them into practical technical solutions.Excellent analytical, problem-solving and communication skills.Desirable experienceData-lake or lakehouse technologies such as Delta Lake, Apache Iceberg, Apache Hudi, Databricks, Snowflake or AWS Lake Formation.Image or video visualisation, enhancement ...

/CD.The ability to understand customer requirements and translate them into practical technical solutions. Excellent analytical, problem-solving and communication skills. Desirable experienceData-lake or lakehouse technologies such as Delta Lake, Apache Iceberg, Apache Hudi, Databricks, Snowflake or AWS Lake Formation. Image or video visualisation, enhancement ...

best practices. Skills & Experience Strong hands-on experience with Azure Databricks. Experience building data pipelines using Python and/or PySpark. Understanding of Delta Lake, lakehouse architectures and modern data platform design. Experience working with Unity Catalog for data governance and access management. Strong SQL development, optimisation ...

best practices. Skills & Experience Strong hands-on experience with Azure Databricks. Experience building data pipelines using Python and/or PySpark. Understanding of Delta Lake, lakehouse architectures and modern data platform design. Experience working with Unity Catalog for data governance and access management. Strong SQL development, optimisation ...

best practices. Skills & Experience Strong hands-on experience with Azure Databricks. Experience building data pipelines using Python and/or PySpark. Understanding of Delta Lake, lakehouse architectures and modern data platform design. Experience working with Unity Catalog for data governance and access management. Strong SQL development, optimisation ...

Support business development activities through solution design, client proposals, and technical leadership. What We're Looking For Deep expertise with the Databricks ecosystem, including Delta Lake, Unity Catalog, Spark, MLflow, Databricks SQL and Lakehouse architecture. Strong understanding of data governance, security, metadata management and regulatory compliance. Experience delivering ...

personNice to have, not requiredDo not rule yourself out if you do not have these. They are a bonus, not a checklist. Experience with Delta Lake, Iceberg, Spark or Databricks, PalantirExperience optimising LLM serving with tools such as vLLM, TGI or TensorRT LLMSearch and ranking experience. For example ...

+ performance troubleshooting. Supporting + maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis + permanent remediation. Experience of Spark SQL, Delta Lake, Parquet + Hive metastore tables, schemas, partitions and the relationship between table metadata and underlying files. Hands-on AWS experience, particularly ...

controls and records needed to support auditing, data traceability and compliance requirements. Performance and operational management Investigate and improve the performance of Spark workloads, Delta tables and SQL compute. Review resource consumption and identify ways to reduce unnecessary platform and cloud expenditure. Set up monitoring and alerting to identify … mentoring across the team. What we are looking for Practical experience managing Databricks in a production environment. A strong understanding of Apache Spark, Delta Lake and Unity Catalog. Experience working with one or more major cloud providers, including storage, networking and identity management. Confidence using Terraform ...

Data Warehousing . Strong background in incident resolution, requests, changes and problem-solving within SLAs. Hands-on experience with Spark, Data Architecture, SQL and Delta Lake. Cloud development experience (AWS, GCP or Azure). Working knowledge of Medallion Architecture. Demonstrated capability in implementing and supporting pipelines in demanding environments. ...