101 to 125 of 173 Permanent Delta Lake Jobs

continuous improvement initiatives. Key Skills & Experience Proven experience as a Data Engineer within production environments. Strong hands-on experience with Databricks, Apache Spark, Delta Lake, Python and SQL. Experience building and supporting scalable ETL/ELT pipelines. Strong understanding of distributed data processing and data modelling. Cloud platform ...

designing and implementing data solutions on at least one public cloud platform (AWS, Azure, or GCP)Working knowledge of distributed data systems: Apache Spark, Delta Lake, or equivalent (Hadoop, Kafka, Flink)Experience leading technical customer conversations — discovery, whiteboarding, architecture reviewsFamiliarity with one or more: data engineering (ETL/ ...

designing and implementing data solutions on at least one public cloud platform (AWS, Azure, or GCP) Working knowledge of distributed data systems: Apache SparkTM, Delta Lake, or equivalent (Hadoop, Kafka, Flink) Experience leading technical customer conversations — discovery, whiteboarding, architecture reviews Familiarity with one or more: data engineering ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits and perks ...

legacy operational platforms. Experience with regulatory, client or operational reporting. Experience supporting data migration, legacy system modernisation or platform replacement. Experience with Databricks, Delta Lake, Spark SQL or PySpark. Experience defining validation frameworks, data quality dashboards or metadata-driven rule engines. Experience working in consultancy or client-facing ...

time streaming, and CI/CD pipelines (Git, Terraform, Databricks Asset Bundles). Deep knowledge of Databricks/Snowflake environments, including Unity Catalog, workspaces, Delta Lake, and networking/security (VPNs, Private Links, Firewalls) and/or Snowflake Platform including Cortex, Horizon and Snowpark. Advanced proficiency in Azure ...

data fidelity. Collaborate with cross-functional teams to align data architecture with financial regulations (e.g., GDPR, Basel III, MiFID II). Drive adoption of Delta Lake , Apache Spark , and MLflow for advanced analytics and machine learning use cases. Provide technical leadership and mentoring to data engineers and analysts ...

Skills & Experience: Hands-on experience with Databricks and Microsoft Fabric Experience building and running production-grade pipelines Strong PySpark and SQL skills Experience with Delta Lake and Lakehouse architectures Good grasp of data modelling, orchestration and workflow automation Comfortable working with structured and semi-structured data (Parquet, JSON ...

engineering and productionising machine learning models Strong hands-on development skills across Python, PySpark and SQL Commercial experience working with Databricks, MLflow and Delta Lake Proven experience building and operating distributed data and machine learning pipelines Experience taking Data Science models from notebooks or development environments into production ...

designing and implementing data solutions on at least one public cloud platform (AWS, Azure, or GCP) Working knowledge of distributed data systems: Apache SparkTM, Delta Lake, or equivalent (Hadoop, Kafka, Flink) Experience leading technical customer conversations — discovery, whiteboarding, architecture reviews Familiarity with one or more: data engineering ...

including performance tuning (partitioning, shuffle optimization, caching), troubleshooting, and designing scalable batch/stream processing patterns. Deep lakehouse and data platform expertise(e.g., Delta Lake concepts, schema evolution, data quality frameworks, lineage/metadata, and governance patterns) applied in production environments. Strong DevSecOps and platform engineering capability, including ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits and perks ...

clearly articulate what modern features they used recently and why (platform evolution awareness is essential for this role). Should Have Strong command of Delta Lake and performance patterns: MERGE/CDC strategies, partitioning/file sizing, OPTIMIZE/ZORDER, VACUUM, table maintenance automation. Production architecture practices: Environment ...

analytical modelling and reporting. Familiarity with cloud platforms (Azure or AWS), particularly in hybrid data environments. Experience working with data Lakehouse technologies (e.g. Delta Lake, Databricks, or similar platforms). Be able to travel to programme and customer meetings across the UK as required. Security cleared or ability ...

discussions, including vendor conversations and the technical sections of proposals Desirable Experience with NHS or other routine healthcare data at patient level Unity Catalog, Delta Lake and medallion structure, bronze to silver to gold Exposure to dbt and dimensional modelling Experience contributing to vendor or partner relationships, such ...

Azure Databricks Data Engineer to design, develop, deploy, and optimize scalable data solutions on Microsoft Azure. You will build production-grade data pipelines with Delta Lake, Unity Catalog, and Databricks tools. The role requires hands-on expertise in PySpark, Python, SQL, and Spark architecture, plus experience with GitLab … based CI/CD, Databricks Asset Bundles, and Azure Data Lake Storage. #J-18808-Ljbffr ...

performance, cost and standards-compliance metrics, using them to guide roadmap priorities and architecture governance decisions. PERSONAL PROFILE Deep expertise in Databricks (Unity Catalog, Delta Lake, Spark, Databricks SQL) or equivalent Lakehouse platforms, including administration, cost management, and performance optimisation. Strong understanding of cloud-native architecture principles, cost ...

solid understanding of agile delivery practices including CI/CD, Application Resiliency, and Security. Preferred qualifications, capabilities, and skills Production experience with Databricks (Delta Lake, Unity Catalog, MLflow, Databricks Jobs) and workflow orchestration with Apache Airflow. Strong working knowledge of both supervised and unsupervised techniques, including tree-based ...

Disclosure and Barring Service)Mandatory SkillsAzure Databricks Platform EngineeringHands-on implementation and troubleshooting of Azure DatabricksDatabricks Serverless configuration and workload optimisationDatabricks SQL and Delta Lake developmentCluster management, compute policies, autoscaling, and workload monitoringData EngineeringStrong Python, PySpark, and SQL development skillsDesign and build ingestion and transformation pipelinesFull and incremental ...

Experience owning delivery roadmaps, OKRs, and budgets for an architecture or engineering function. Strong technical foundation across the data platform stack: Databricks lakehouse architecture (Delta Lake, medallion patterns), Azure Synapse (star schemas, dedicated pools), and data pipeline orchestration (ADF or Databricks Workflows). Experience with data quality frameworks ...

grade data pipelines, and strong experience with Python, Scala, or similar programming languages. Experience working with modern cloud data platforms such as Databricks (preferred), Delta Lake, Snowflake, or BigQuery. Proven experience driving complex cross‐functional initiatives from concept through production. Experience leading technical architecture discussions and engineering design ...

working through ambiguity. Desirable MLOps excellence: reproducible pipelines, model versioning, CI/CD, observability, and automated evaluation. Data engineering: proficiency with Databricks, Apache Spark, Delta Lake, MLflow, and SQL; experience integrating datasets and maintaining data quality. Model training and optimisation: experience with pre‐training, fine‐tuning, distributed training ...

Modelling, Data Warehousing. Strong background in incident resolution, requests, changes and problem-solving within SLAs. Hands-on experience with Spark, Data Architecture, SQL and Delta Lake. Cloud development experience (AWS, GCP or Azure). Working knowledge of Medallion Architecture. Demonstrated capability in implementing and supporting pipelines in demanding environments. ...

career while helping customers unlock the power of their data. Responsibilities Design, build and optimise scalable data pipelines using Databricks, Apache Spark, PySpark and Delta Lake. Develop and implement modern Lakehouse architectures, applying Medallion (Bronze, Silver, Gold) design principles to support trusted and reusable data products. Build robust … data engineering capability. Essential Skills & Experience Proven hands‐on experience building and supporting production‐grade data pipelines using Databricks, PySpark, Spark SQL and Delta Lake. Experience designing and implementing Lakehouse architectures, including Medallion (Bronze, Silver, Gold) data models. Expertise in developing scalable ETL/ELT pipelines capable of handling ...

will build robust ETL/ELT pipelines with Azure Data Factory, Synapse and Databricks, design real-time and batch ingestion, and modern data platforms (Delta Lake, lakehouse, ADLS Gen2). Hybrid-working is offered. #J-18808-Ljbffr ...