1 to 25 of 99 Permanent Delta Lake Jobs

Data Factory, Microsoft Fabric, Fabric, SQL, PostgreSQL, API, CI/CD, PowerBI, ETL, ELT, Data Engineer, Azure fundamentals, Fabric Data Engineer, ADF, ADL, Data Lake, Delta Lake, Azure Data Warehouse, Azure Data Factory, Microsoft Fabric, Fabric, SQL, PostgreSQL, API, CI/CD, PowerBI, ETL, ELT, Data Engineer … Azure fundamentals, Fabric Data Engineer, ADF, ADL, Data Lake, Delta Lake, Azure Data Warehouse, Azure Data Factory, Microsoft Fabric, Fabric, SQL, PostgreSQL, API, CI/CD, PowerBI, ETL, ELT, Data Engineer, Azure fundamentals, Fabric Data Engineer, ADF, ADL, Data Lake, Delta Lake, Azure Data ...

optimise large-scale data workflows within the Databricks Data Intelligence Platform. The role focuses on delivering high-performing batch and streaming pipelines using PySpark, Delta Lake, and Azure services, with additional emphasis on governance, lineage tracking, and workflow orchestration. Client information remains confidential. Key Responsibilities Build and orchestrate … Databricks data pipelines using Notebooks, Jobs, and Workflows Optimise Spark and Delta Lake workloads through cluster tuning, adaptive execution, scaling, and caching Conduct performance benchmarking and cost optimisation across workloads Implement data quality, lineage, and governance practices aligned with Unity Catalog Develop PySpark-based ETL and transformation logic ...

limited to, the following: Architect and lead the design of scalable, secure, and high-performance data platforms using Databricks, Apache Spark (PySpark), Delta Lake, and Azure Data Lake Storage. Build and manage robust, real-time data ingestion and processing pipelines using Apache Kafka, enabling low-latency analytics … reliable data transformation, enrichment, and orchestration across cloud-native data services. Integrate diverse internal and external data sources via REST APIs into Azure Data Lake, Databricks, and Snowflake, maintaining high data quality, consistency, and lineage. Implement and optimize Delta Lake architecture for versioned, ACID-compliant data storage ...

proactive Azure Data Consultant with strong hands on expertise in Data Engineering within the Microsoft Azure data ecosystem , including Microsoft Fabric , Synapse Analytics , and Delta Lake . This role requires hands-on experience in building and optimising data pipelines and notebooks, transforming large datasets, and enabling data-driven … notebooks using Synapse Analytics and Microsoft Fabric. Build and optimise complex data transformation and ingestion logic to support enterprise-level reporting and analytics. Leverage Delta Lake architecture to support both batch and streaming data processes. Develop high-quality, reusable code in Python and PySpark , following best practices ...

Azure-based data pipelines. The focus of this role is to deliver scalable, test-driven, and configuration-driven data processing solutions using Python, PySpark, Delta Lake, and containerised workloads. This opportunity sits within a fast-paced engineering environment working closely with cloud, DevOps, and data science teams. … transformation, and validation pipelines using Python and PySpark Implement unit and BDD testing with Behave, including mocking, patching, and dependency management Design and manage Delta Lake tables, ensuring ACID compliance, schema evolution, and incremental loading Build and maintain containerised applications using Docker for development and deployment Develop configuration ...

data into structured, analytics-ready formats. Optimise pipeline performance and ensure high availability of data services. Infrastructure & Architecture Architect and deploy scalable data lake solutions using Azure Data Lake Storage. Implement governance and security measures across the platform. Leverage Terraform or similar IaC tools for controlled and reproducible … deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations and CI/CD pipelines for Databricks deployments. Monitoring & Operations Implement monitoring solutions using Azure ...

data into structured, analytics-ready formats. Optimise pipeline performance and ensure high availability of data services. Infrastructure & Architecture Architect and deploy scalable data lake solutions using Azure Data Lake Storage. Implement governance and security measures across the platform. Leverage Terraform or similar IaC tools for controlled and reproducible … deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations and CI/CD pipelines for Databricks deployments. Monitoring & Operations Implement monitoring solutions using Azure ...

data into structured, analytics-ready formats. Optimise pipeline performance and ensure high availability of data services. Infrastructure & Architecture Architect and deploy scalable data lake solutions using Azure Data Lake Storage. Implement governance and security measures across the platform. Leverage Terraform or similar IaC tools for controlled and reproducible … deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations and CI/CD pipelines for Databricks deployments. Monitoring & Operations Implement monitoring solutions using Azure ...

data into structured, analytics-ready formats. Optimise pipeline performance and ensure high availability of data services. Infrastructure & Architecture Architect and deploy scalable data lake solutions using Azure Data Lake Storage . Implement governance and security measures across the platform. Leverage Terraform or similar IaC tools for controlled … deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations and CI/CD pipelines for Databricks deployments. Monitoring & Operations Implement monitoring solutions using Azure ...

real-time insights. Key Responsibilities Architecture & Solution Design Architect end-to-end Databricks Lakehouse solutions including data ingestion, ETL/ELT pipelines, Delta Lake, data warehousing, and data governance frameworks. Define and enforce best practices, security standards, and performance optimization across Databricks workloads. Design scalable big data architectures … leveraging Spark, Delta Live Tables, Unity Catalog, and MLflow. Lead cloud architecture design on Azure/AWS/GCP (based on organization need) integrated with Databricks. Data Engineering & Platform Build Build and optimize large-scale ETL/ELT pipelines using PySpark, SQL, and Delta. Oversee data quality frameworks, metadata ...

real-time insights. Key Responsibilities Architecture & Solution Design Architect end-to-end Databricks Lakehouse solutions including data ingestion, ETL/ELT pipelines, Delta Lake, data warehousing, and data governance frameworks. Define and enforce best practices, security standards, and performance optimization across Databricks workloads. Design scalable big data architectures … leveraging Spark, Delta Live Tables, Unity Catalog, and MLflow. Lead cloud architecture design on Azure/AWS/GCP (based on organization need) integrated with Databricks. Data Engineering & Platform Build Build and optimize large-scale ETL/ELT pipelines using PySpark, SQL, and Delta. Oversee data quality frameworks, metadata ...

PySpark . Expertise in Databricks for large-scale data engineering and analytics workloads. Design and implement scalable data architectures using Databricks Unity Catalog, Delta Lake, and Apache Spark Implement advanced Delta Lake features including time travel, vacuum operations, and Z-ordering. Desirable: certification in Databricks Data ...

Spark) and PySpark. Expertise in Databricks for large-scale data engineering and analytics workloads. Design and implement scalable data architectures using Databricks Unity Catalog, Delta Lake, and Apache Spark Implement advanced Delta Lake features including time travel, vacuum operations, and Z-ordering. Desirable: certification in Databricks ...

Data Engineer/Data Engineering/Data Consultant/Lakehouse/Delta Lake/Data Warehousing/ETL/Azure/Azure Databricks/Python/SQL/Based in the West Midlands/Solihull/Birmingham area, Permanent role, £50,000 - 70,000 + car/allowance … allowance + bonus Experience: Experience in a Data Engineer/Data Engineering role Large and complex datasets Azure, Azure Databricks Microsoft SQL Server Lakehouse, Delta Lake Data Warehousing ETL Database Design Python/PySpark Azure Blob Storage Azure Data Factory Desirable: Exposure ML/Machine Learning/ ...

Data Engineer/Data Engineering/Data Consultant/Lakehouse/Delta Lake/Data Warehousing/ETL/Azure/Azure Databricks/Python/SQL/Based in the West Midlands/Solihull/Birmingham area, Permanent role, £50,000 - 70,000 + car/allowance … allowance + bonus Experience: Experience in a Data Engineer/Data Engineering role Large and complex datasets Azure, Azure Databricks Microsoft SQL Server Lakehouse, Delta Lake Data Warehousing ETL Database Design Python/PySpark Azure Blob Storage Azure Data Factory Desirable: Exposure ML/Machine Learning/ ...

one. What You'll Be Doing Designing end-to-end Microsoft Fabric architectures using the Medallion pattern. Building metadata-driven ingestion pipelines and optimising Delta Lake performance. Writing advanced PySpark/Spark SQL notebooks for large-scale transformations. Developing semantic models for enterprise reporting and enabling Direct Lake. … impact projects with cutting-edge Microsoft technologies. What We're Looking For Hands-on experience with Microsoft Fabric workloads: Lakehouse, Data Factory, Pipelines, Notebooks, Delta Lake, Eventstreams, Semantic Models. Advanced PySpark/Spark SQL skills and strong data engineering fundamentals. Integration experience with Dynamics 365, Dataverse, Business Central. ...

experience with some of the following and be able to learn new technologies or concepts quickly: Deep experience with Databricks on Microsoft Azure, including Delta Lake, Unity Catalog, Delta Live Tables, PySpark and Databricks SQL, alongside complementary services such as Azure Data Factory, Synapse Analytics, and Microsoft … Fabric. Designing and implementing Lakehouse architectures in Databricks to unify data warehousing and data lake capabilities, enabling scalable analytics and governance. Building and orchestrating Databricks notebooks, Delta Live Tables, Databrick Jobs and integration with Azure Data Factory for enterprise-scale data ingestion and transformation. Proficiency in Python ...

experience with some of the following and be able to learn new technologies or concepts quickly: Deep experience with Databricks on Microsoft Azure, including Delta Lake, Unity Catalog, Delta Live Tables, PySpark and Databricks SQL, alongside complementary services such as Azure Data Factory, Synapse Analytics, and Microsoft … Fabric. Designing and implementing Lakehouse architectures in Databricks to unify data warehousing and data lake capabilities, enabling scalable analytics and governance. Building and orchestrating Databricks notebooks, Delta Live Tables, Databrick Jobs and integration with Azure Data Factory for enterprise-scale data ingestion and transformation. Proficiency in Python ...

hands-on technical role responsible for designing, developing, and maintaining data pipelines within the IT department. The pipelines will be realised in a modern lake environment and the engineer will collaborate in cross-functional teams to gather requirements and develop the conceptual data models. This role plays a crucial … scalability, and efficiency. Highly Desirable: Experience with Informatica ETL, Hyperion Reporting, and intermediate/advanced PL/SQL. Desirable Experience in a financial corporation Lake House/Delta Lake and Snowflake Experience with Spark clusters, both elastic permanent and transitory clusters Familiarity with data governance, data security ...

services (Data Factory, Synapse, Blob Storage, etc.). Proficiency in SQL for data manipulation, transformation, and performance optimisation. Hands-on experience with Databricks (Spark, Delta Lake, notebooks). Solid understanding of data architecture principles and cloud-native design. Experience working in consultancy or client-facing roles is highly ...

services (Data Factory, Synapse, Blob Storage, etc.). Proficiency in SQL for data manipulation, transformation, and performance optimisation. Hands-on experience with Databricks (Spark, Delta Lake, notebooks). Solid understanding of data architecture principles and cloud-native design. Experience working in consultancy or client-facing roles is highly ...

services (Data Factory, Synapse, Blob Storage, etc.). Proficiency in SQL for data manipulation, transformation, and performance optimisation. Hands-on experience with Databricks (Spark, Delta Lake, notebooks). Solid understanding of data architecture principles and cloud-native design. Experience working in consultancy or client-facing roles is highly ...

enriching, and aggregating data. Ensure data consistency and accuracy throughout the data lifecycle. Azure Databricks Implementation: Work extensively with Azure Databricks Unity Catalog, including Delta Lake, Spark SQL, and other relevant services. Implement best practices for Databricks development and deployment. Optimise Databricks workloads for performance and cost. Need ...

enriching, and aggregating data. Ensure data consistency and accuracy throughout the data lifecycle. Azure Databricks Implementation: Work extensively with Azure Databricks Unity Catalog, including Delta Lake, Spark SQL, and other relevant services. Implement best practices for Databricks development and deployment. Optimise Databricks workloads for performance and cost. Need ...

enriching, and aggregating data. Ensure data consistency and accuracy throughout the data lifecycle. Azure Databricks Implementation: Work extensively with Azure Databricks Unity Catalog, including Delta Lake, Spark SQL, and other relevant services. Implement best practices for Databricks development and deployment. Optimise Databricks workloads for performance and cost. Need ...