51 to 75 of 233 Delta Lake Jobs

build and optimise data pipelines across Pricing, Underwriting and Claims. You’ll work on a modern Azure-based data platform, consolidating capabilities on Databricks, Delta Lake and Unity Catalog to support analytics and AI initiatives. You’ll collaborate with Data Architects and Analysts, delivering scalable, secure data solutions ...

than operating purely at a strategic level. Ideally, your experience will include several of the following: Strong commercial experience with Databricks Unity Catalog, notebooks, Delta Lake or Delta Live Tables Data lakehouse and medallion architecture Advanced data modelling, including relational, dimensional, Data Vault or similar approaches Python ...

learning and professional growth. Experience and knowledge: Experience of designing and implementing an Azure analytical platform including Azure Databricks, Azure Data Factory, Azure Data Lake Storage, Delta Lake, Unity Catalog, PySpark, SQL, Python, and Azure DevOps. Experience of working with senior stakeholders to articulate the strategy, objectives ...

Medallion Architecture, and ensuring our Databricks platform is secure, governed, and production ready.Key Responsibilities: Design, build, and optimise scalable data pipelines using Databricks, Delta Live Tables, Synapse, and Delta Lake.Develop ingestion, transformation, and orchestration workflows across diverse sources (APIs, SQL, SFTP, SaaS platforms).Implement strong data quality, lineage … across pipelines and clusters.Contribute to standards for versioning, DevOps, CI/CD, and testing for Databricks notebooks, workflows, and SQL assets.Support the rollout of Delta Sharing, Unity Catalog governance, and Iceberg/Delta interoperability where required.Partner with analysts and data scientists to ensure that high-quality, well-modelled ...

Medallion Architecture, and ensuring our Databricks platform is secure, governed, and production ready. Key Responsibilities Design, build, and optimise scalable data pipelines using Databricks, Delta Live Tables, Synapse, and Delta Lake. Develop ingestion, transformation, and orchestration workflows across diverse sources (APIs, SQL, SFTP, SaaS platforms). Implement strong … clusters. Contribute to standards for versioning, DevOps, CI/CD, and testing for Databricks notebooks, workflows, and SQL assets. Support the rollout of Delta Sharing, Unity Catalog governance, and Iceberg/Delta interoperability where required. Partner with analysts and data scientists to ensure that high-quality, well‐modelled ...

/ELT solutions and enterprise data pipelines. Experience with event-driven and messaging architectures. Data Platforms & Cloud Technologies Hands-on experience with: Azure Data Lake Databricks Snowflake Delta Lake Azure Synapse Analytics/Microsoft Fabric Spark/PySpark Strong understanding of: Data Lake and Lakehouse architecture ...

batch data ingestion solutions using Azure Event Hubs, Kafka, Databricks Structured Streaming and Azure Stream Analytics Developing and maintaining modern data platforms, including Delta Lake/Lakehouse architectures on ADLS Gen2, curated datasets, dimensional models and Azure Synapse Dedicated SQL Pools Analysing current business practices, processes and procedures … essential with a regulated environment is essential Significant experience with data solution design experience in an MI/BI/Analytics environment (Kimball, lake house, data lake) is essential Significant experience using: Azure Data Factory, Azure Synapse Analytics, Azure SQL, Azure Data Lake, Datra bricks, Purview ...

Must Have: At least 7 years as an Engineer in a Data Field Strong Azure data skills, including: Azure Data Factory V2 Azure Data Lake Storage V2 Azure Databricks Azure Function Apps& Logic Apps Azure Stream Analytics Azure Resource Manager skills (Terraform, Azure Portal, Az CLI and Az PowerShell … Strong PySpark, Delta Lake, Unity Catalog and Python skills. Includes ability to write unit and integration tests in Python with unittest, pytest, etc. Strong understanding of software development practices, such as SOLID principals, structuring code, testing, IOC, etc. Strong repositories, CI/CD skills. Strong knowledge of Kimball ...

including extensive experience with workflow orchestration tools (e.g., Apache Airflow, Dagster).Hands-on expertise with cloud data warehouses (e.g., Snowflake, BigQuery, Redshift) and data lake architectures (e.g., Databricks, Delta Lake, Apache Iceberg).Advanced SQL skills and proficiency in Python. Strong background in modern software development practices (testing ...

Skills Strong experience designing and delivering modern data platforms using Microsoft Fabric, Azure Databricks, Azure Data Factory, Azure Synapse Analytics, Azure SQL and Data Lake technologies . Proven experience implementing Lakehouse, Data Warehouse and Medallion Architecture patterns, including data modelling, governance and performance optimization . Expertise in building scalable … Python, Spark and PySpark for data engineering, analytics and automation workloads. Experience with Azure Databricks and/or Microsoft Fabric engineering workloads, including notebooks, Delta Lake, data transformation, optimization and operationalization of data solutions. An understanding of data governance, security, lineage and data quality, using tools and frameworks ...

secure networks Specialist tracks (we'd love depth in one of these; you don't need all of them) Azure/Databricks Azure Data Lake Storage, Databricks, Apache Spark, Delta Lake, Azure Data Factory, MLflow; and, for the AI edge, Databricks Mosaic AI, Vector Search and Unity ...

with a customer in Central London.Key ResponsibilitiesData Lakehouse Architecture: Design and optimize a high-performance lakehouse pattern utilizing Microsoft Fabric OneLake and Azure Data Lake Storage Gen2 (ADLS Gen2).Data Zone Management: Maintain architectural segregation between raw storage (immutability, landing) and curated data zones (cleansed, conformed business layers) while … ecosystem (OneLake, Data Factory, Fabric Lakehouse).Big Data Processing: Deep technical expertise with Apache Spark/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats.Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python (PySpark).Security & Compliance: Direct experience implementing security controls for sensitive ...

customer in Central London. Key Responsibilities Data Lakehouse Architecture: Design and optimize a high-performance lakehouse pattern utilizing Microsoft Fabric OneLake and Azure Data Lake Storage Gen2 (ADLS Gen2). Data Zone Management: Maintain architectural segregation between raw storage (immutability, landing) and curated data zones (cleansed, conformed business layers … OneLake, Data Factory, Fabric Lakehouse). Big Data Processing: Deep technical expertise with Apache Spark/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python (PySpark). Security & Compliance: Direct experience implementing security controls ...

native data services Data architecture: Lakehouse, medallion, data mesh and domain‐oriented patterns Data processing: Apache Spark, SQL and distributed processing technologies Data storage: Delta Lake, Apache Iceberg and open table formats Data pipelines: Enterprise‐scale ETL/ELT, batch and streaming patterns Transformation and quality: dbt, Great ...

domain processes, including policy, claims, underwriting, pricing, risk, and regulatory reporting. Hands-on expertise with: Azure Data Factory (ADF) for pipeline orchestration Databricks (PySpark, Delta Lake, MLflow) for data engineering and ML integration Azure ecosystem (Key Vault, Data Lake, Synapse, Functions, Logic Apps) Strong understanding ...

workflows Excellent communication and collaboration skills in agile, cross-functional teams Ability to mentor junior engineers Nice to have: Experience with Databricks, PySpark, and Delta Lake Условия Competitive salary based on location, experience, and skills Comprehensive benefits package for employees and their families Continuous learning, structured development programs ...

communicate complex concepts in a clear, concise manner Languages: Terraform, Python, SQL, PowerShell. CI/CD: Azure DevOps pipelines for data platform deployments. Databricks: Delta Lake, Unity Catalog, Workflows, Notebooks, Clusters, MLflow. Azure: ADLS Gen2, Azure Data Factory, Event Hubs/Kafka, Key Vault, Functions, Synapse. AI Foundry … Service Plan Deployment and Configuration, Integration with Azure bot Services MS Fabric: Tenant Management, Workspace Private Links, Deployment, Capacity Management, Workspace Management, One Lake, Data Mirroring, Power BI AI Foundry : Model management, Hosted Agents Deployment and Management , Project Configuration (Connections and Governance ) and lifecycle management. Inclusion & Diversity We value ...

communicate complex concepts in a clear, concise manner Languages: Terraform, Python, SQL, PowerShell. CI/CD: Azure DevOps pipelines for data platform deployments. Databricks: Delta Lake, Unity Catalog, Workflows, Notebooks, Clusters, MLflow. Azure: ADLS Gen2, Azure Data Factory, Event Hubs/Kafka, Key Vault, Functions, Synapse. AI Foundry … Service Plan Deployment and Configuration, Integration with Azure bot Services MS Fabric: Tenant Management, Workspace Private Links, Deployment, Capacity Management, Workspace Management, One Lake, Data Mirroring, Power BI AI Foundry : Model management, Hosted Agents Deployment and Management , Project Configuration (Connections and Governance ) and lifecycle management. Inclusion & Diversity We value ...

Unifeye. What You'll Do Design and build scalable batch and streaming data pipelines using Spark and Databricks. Develop modern Lakehouse architectures using Delta Lake and Medallion (Bronze, Silver and Gold) design principles. Build robust ingestion frameworks from APIs, files and streaming data sources. Ensure data quality, testing … data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. Good understanding of Lakehouse architecture and Delta Lake. Experience working with AWS, Azure or GCP. Experience implementing CI/CD, version control and deployment automation. Strong communication skills and experience working ...

data platforms. Expertise with one or more of the following technologies: AWS (Redshift, Glue, S3, Bedrock) Microsoft Azure (Synapse, Data Factory, Azure SQL) Databricks (Delta Lake, ETL pipelines, ML workloads) Snowflake Strong data modelling experience across enterprise-scale environments. Experience building modern AI-enabled data architectures. Knowledge ...

Databases & Tools knowledge: Microsoft SQL Server (administration, T-SQL, DML, DDL, SSIS), Databricks, Power BI. Experience in Azure Technologies: ADLS, ADF, Azure SQL Database, Delta Lake, Azure DevOps, Logic Apps, Function Apps, Key Vaults, Storage Accounts. Infrastructure & Automation: Terraform, Python/PySpark. Expertise in data modeling, integration ...

Snowflake and cloud-native data services Data architecture:Lakehouse, medallion, datameshand domain-oriented patterns Data processing:Apache Spark,SQLand distributed processing technologies Data storage:Delta Lake, ApacheIcebergand open table formats Data pipelines:Enterprise-scale ETL/ELT,batchand streaming patterns Transformation and quality:dbt, Great Expectations and equivalent ...

Lead the assessment and modernisation of legacy SQL Server, ETL and reporting workloads into a Databricks-based architecture. Design scalable data models, Delta Lake structures and transformation pipelines supporting real estate investment data. Analyse data lineage, dependencies, business logic and data quality to determine what should be migrated ...

such as Snowflake's Snowpark, Dynamic Tables, Streams & Tasks, and Snowflake Cortex, as well as Databricks components such as Lakeflow (Connect, Pipelines, and Jobs), Delta Lake, and Unity Catalog. Strong data engineering fundamentals: ETL/ELT pipeline design and implementation, data orchestration and workflow automation, batch and streaming ...

data engineering, data platform delivery, software engineering, or related enterprise technology roles. Strong hands‐on experience with Databricks, Spark/PySpark, Python, SQL, Delta Lake, Azure, ADLS, and cloud‐based data engineering patterns. Experience designing scalable data pipelines, data products, and lakehouse solutions using modern cloud data platform ...