126 to 150 of 173 Permanent Delta Lake Jobs

enable analytics, reporting, and AI initiatives across Pricing, Underwriting and Claims. You will design and build scalable data pipelines on Azure Databricks and Delta Lake, with governance, security, and real-time processing at scale. The role sits in the Enterprise Data Team and requires strong SQL and PySpark ...

days on site), the role requires 8+ years in data engineering, strong PySpark/SQL, and active SC clearance. You will implement governance, Delta Lake, Unity Catalog, and CI/CD practices to ensure reliable #J-18808-Ljbffr ...

Requirements 7+ years experience in data engineering or data platform development Strong experience with Databricks, Spark and distributed data processing Experience with Unity Catalog, Delta Lake and streaming architectures Advanced Python and Microsoft SQL Server development Proven experience leading engineering teams or workstreams Strong stakeholder engagement and consulting ...

development Strong experience with modern cloud data platforms, Spark and distributed data processing Experience with data cataloguing and governance tooling, open table formats (e.g. Delta Lake, Iceberg) and streaming architectures Advanced Python and Microsoft SQL Server development Proven experience leading engineering teams or workstreams Strong stakeholder engagement ...

data platform and services using the below technologies: Python, pandas and SQL in Jupyter notebooks for analysis dbt and Databricks data platform with Delta Lake Lightdash as our BI tool and semantic layer Streamlit for interactive data applications Airflow for orchestration Kubernetes for application deployment CI/ ...

data platform and services using the below technologies: Python, pandas and SQL in Jupyter notebooks for analysis dbt and Databricks data platform with Delta Lake Lightdash as our BI tool and semantic layer Streamlit for interactive data applications Airflow for orchestration Kubernetes for application deployment CI/ ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn, and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits and perks ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits and perks ...

Highly motivated and willing to take ownership of high-impact projects upon arrival. Prior hands-on experience with data platforms and technologies such as Delta Lake, Spark, Kubernetes, Kafka, Click House, and/or Presto/Trino. Experience building large-scale batch and streaming pipelines with strict ...

creation of data domains that align with the banks underlying capabilities Data Engineering & Platforms Design data product schemas across Snowflake and Databricks, including Delta Lake structures and access patterns Define and enforce architectural guardrails for data product design, including performance, scalability, security, and cost efficiency Set design standards ...

data platform and services using the below technologies: Python, pandas and SQL in Jupyter notebooks for analysis dbt and Databricks data platform with Delta Lake Lightdash as our BI tool and semantic layer Streamlit for interactive data applications Airflow for orchestration Kubernetes for application deployment CI/ ...

control the creation of data domains that align with the banks underlying capabilitiesData Engineering & PlatformsDesign data product schemas across Snowflake and Databricks, including Delta Lake structures and access patternsDefine and enforce architectural guardrails for data product design, including performance, scalability, security, and cost efficiencySet design standards and review ...

solid understanding of agile delivery practices including CI/CD, Application Resiliency, and Security. Preferred Qualifications, Capabilities, And Skills Production experience with Databricks (Delta Lake, Unity Catalog, MLflow, Databricks Jobs) and workflow orchestration with Apache Airflow. Experience with evaluation frameworks and approaches (golden datasets, LLM-as-judge, human ...

covering the full lifecycle, discovery, design and build, through to production release and support Deep hands‐on experience of the core platform: Unity Catalog, Delta Lake, workflows and jobs, declarative pipelines, SQL warehouses, cluster and compute configuration Migrated data at scale off on‐prem or legacy platforms ...

models and datasets is highly desirable. Broader experience with cloud data platforms such as Azure, and familiarity with Databricks features beyond dbt, such as Delta Lake or Unity Catalogue. Familiarity with data visualisation tools and BI tools and their data requirements. Previous involvement in large‐scale data migration … transformation on Databricks SQL (Databricks SQL/Spark SQL) VS Code UI for SQL and dbt coding Git/GitHub Databricks Workflows/Delta Live Tables for orchestration Modern data visualisation and self‐serve/conversational analytics tooling Benefits: A relaxed working environment with regular socials including a summer ...

escalating, managing and reporting control issues with transparency. Skills & ExperienceEssentialProven in-depth commercial experience in the following key areas: Databricks – Lakehouse architecture, Spark, Delta Lake, workflows and performance tuning. Apache Spark & Scala – High-performance distributed data processing and optimisation. Cloud Data Engineering (AWS) – Cloud-native data platforms … Applications DevelopmentDemonstrated knowledge of client core business functionsDemonstrated leadership, project management, and development skillsRelationship and consensus building skillsPreferredKafka/Event-Driven ArchitectureApache Iceberg/Delta LakePythonAI/ML & LLM integrationFinancial Services/Capital MarketsEducation: Bachelor's degree/University degree or equivalent experienceMaster's degree preferredWhat we'll provide ...

only. Multi-cloud platform depth (AWS and GCP): hands-on experience with data services across one or both of AWS (Redshift, Glue, S3/Lake Formation, MSK, RDS) and GCP (BigQuery, Dataflow, Pub/Sub, Cloud Composer). Data flow mapping and optimisation: demonstrable experience producing current-state data … serving, and GenAI/RAG architectures grounded in proprietary content. Lakehouse and modern data platform design: strong knowledge of open table formats (Apache Iceberg, Delta Lake), zone architecture (landing, cleansed, curated), streaming and batch pipeline patterns, and data quality frameworks. Project engagement and design assurance: able to engage ...

Databricks and Azure, delivering scalable, governance-backed data foundations for AI-powered trading workflows and analytics. The role demands strong Python and SQL, Delta Lake expertise, batch and streaming pipelines, and collaboration with AI engineers, trading teams #J-18808-Ljbffr ...

cloud data environments Expertise in PySpark, SQL and data transformation practices Experience building production-grade data pipelines Knowledge of Microsoft Fabric Experience working with Delta Lake and Lakehouse architectures ...

track. Required Skills & Experience Technical Skills Strong experience with Python and SQL. Hands-on experience with Apache Spark. Experience with Databricks (Unity Catalog/Delta Lake). Experience orchestrating workflows with Apache Airflow. Familiarity with GCP services (GCS, BigQuery). Experience with dbt and ads/payments APIs ...

patterns, standards and trade offsRequirements7 to 10+ years' experience in data platform architecture, data engineering or cloud data infrastructureDeep expertise in Databricks (Unity Catalog, Delta Lake, Spark, Databricks SQL) or an equivalent Lakehouse platform, including administration, cost management and performance optimisationStrong understanding of cloud native architecture principles, cost ...

fixing performance bottlenecks in live systems through profiling and measurement. Nice to Have: Experience with PostgreSQL/pgvector, OpenSearch (or Elasticsearch), Spark/Delta Lake, AWS, NoSQL databases, or Rust/C++ is useful. The Founders You'll partner with a founding team of AI PhDs and elite ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits and perks ...

PySpark to build data products, collaborate across Product, Analytics, Security and Software, and help optimize ETL pipelines in a Lakehouse architecture using Databricks and Delta Lake. #J-18808-Ljbffr ...

regularly observe cloud hardware, network, and operating system faults, and our software must gracefully shield our customers from any of the above. The Delta DML team owns the core write-path operations for Delta Lake, the open-source storage layer behind the Databricks Lakehouse. Our mission … flowing through our platform. We drive performance innovations like Low Shuffle Merge and Deletion Vectors and actively contribute to open source efforts to unify Delta and Iceberg formats. We are seeking a highly skilled and experienced Senior Staff Software Engineer to join our backend team. In this role ...