26 to 50 of 95 Permanent Delta Lake Jobs

environment strategy. Design and govern data integration patterns for sharing data with external clients and third parties, including secure, scalable approaches using Databricks Delta Sharing, APIs, and file-based exchange where appropriate. Lead the design and adoption of Master Data Management (MDM), including entity definitions, golden-record strategy, match … merge rules, and stewardship processes. Establish standards and practices for optimizing Databricks performance and cost — cluster and compute strategy, Delta table management, caching, and workload right-sizing. Define data modelling standards (conceptual, logical, physical) across analytical and operational domains. Collaborate with data governance to embed data quality, lineage, cataloguing ...

platforms (preferably AWS); production experience is not required.A collaborative mindset and willingness to learn and communicate openly within the team.Nice to haveExposure to data lake concepts or formats such as Delta Lake or Iceberg.Any experience with search, graph, or large-scale analytical databases.Interest in AI, machine learning ...

files, and event streams Supporting master data management, data quality rules, and golden record definition Working hands-on with ADLS, Azure SQL, Event Hubs, Delta Lake, and OneLake Designing integration patterns and REST APIs to support near real-time data flows Contributing to infrastructure-as-code and environment ...

pipeline development, and large-scale data processing.Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning.Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar).Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.).Strong understanding of data quality, security … Step Functions, etc.).Databricks or AWS certifications.Demonstrated proficiency in cloud-native development (e.g., cloud, artificial intelligence, machine learning).Experience with Parquet, JSON, CSV, Avro, Delta Lake file formats.J.P. Morgan is a global leader in financial services, providing strategic advice and products to the world’s most prominent corporations ...

strategy to hands‐on delivery. Experience in a startup or small‐team environment , comfortable wearing multiple hats. Nice‐to‐Have Experience with Power BI , Delta Lake , Synapse , or Databricks . Knowledge of Python or TypeScript for data or automation scripts. Familiarity with CI/CD pipelines , IaC (Terraform ...

accelerate delivery, and enhance solution quality while applying sound engineering judgement and responsible AI practices.What You'll DoDesign and develop robust data pipelines using Delta Lake, Spark Structured Streaming, Unity Catalog, and Azure-native services.Build real-time, event-driven solutions using technologies such as Kafka and Azure Event ...

skills – you communicate clearly with stakeholders as well as other engineers, fostering a collaborative, supportive working environment Experience in the financial markets, especially in delta one, store of value, and/or FICC options trading Experience with Linux‐based, concurrent, high‐throughput, low‐latency software systems Experience with pipeline … orchestration frameworks (e.g. Airflow, Dagster) Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databases Have a Bachelors or advanced degree in Computer Science, Mathematics, Statistics, Physics, Engineering, or equivalent work experience For more information about DRW’s processing activities ...

interpersonal skills - you communicate clearly with stakeholders as well as other engineers, fostering a collaborative, supportive working environmentExperience in the financial markets, especially in delta one, store of value, and/or FICC options tradingExperience with Linux-based, concurrent, high-throughput, low-latency software systemsExperience with pipeline orchestration frameworks … e.g. Airflow, Dagster)Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databasesHave a Bachelors or advanced degree in Computer Science, Mathematics, Statistics, Physics, Engineering, or equivalent work experienceFor more information about DRW's processing activities ...

transform, and model people data from multiple HR and workforce systems — including Workday — into a governed, enterprise-grade data model built on Databricks and Delta Lake. Data Modelling & Standards: Own and evolve canonical people data structures across the global people data platform. Define and enforce schemas, ingestion patterns … data models in a cloud environment Strong proficiency in SQL; solid Python and/or PySpark capabilityDemonstrable experience with Databricks, Apache Spark, and Delta Lake architectureProven ability to design and own enterprise-grade canonical data models across multiple upstream source systemsExperience integrating data from APIs and diverse upstream ...

expectations; ability to guide peers on safe and effective usage within team practices Preferred Qualifications, Capabilities, and Skills: Experience with Databricks lakehouse, Databricks Genie, Delta Lake, and medallion architecture ...

model lifecycle: data prep, experimentation, deployment, and monitoring. Build and optimize feature pipelines and scoring jobs using Python, Databricks, Spark, Delta Lake, and AWS. Use AWS services (S3, Redshift, Lambda) for data automation, orchestration, and scalable processing. Ensure data quality, observability, lineage, and documentation across data … with OpenAI APIs, LLM workflows, and prompt engineering. Solid machine learning fundamentals, including supervised learning, NLP, and feature engineering. Experience with Databricks, Spark, and Delta Lake. Strong SQL skills with experience working on large datasets. Experience with AWS, including S3 and Lambda. Familiarity with Redshift, Snowflake, or other cloud ...

model lifecycle: data prep, experimentation, deployment, and monitoring. Build and optimize feature pipelines and scoring jobs using Python, Databricks, Spark, Delta Lake, and AWS. Use AWS services (S3, Redshift, Lambda) for data automation, orchestration, and scalable processing. Ensure data quality, observability, lineage, and documentation across data … with OpenAI APIs, LLM workflows, and prompt engineering. Solid machine learning fundamentals, including supervised learning, NLP, and feature engineering. Experience with Databricks, Spark, and Delta Lake. Strong SQL skills with experience working on large datasets. Experience with AWS, including S3 and Lambda. Familiarity with Redshift, Snowflake, or other cloud ...

model lifecycle: data prep, experimentation, deployment, and monitoring.* Build and optimize feature pipelines and scoring jobs using Python, Databricks, Spark, Delta Lake, and AWS.* Use AWS services (S3, Redshift, Lambda) for data automation, orchestration, and scalable processing.* Ensure data quality, observability, lineage, and documentation across data … with OpenAI APIs, LLM workflows, and prompt engineering.* Solid machine learning fundamentals, including supervised learning, NLP, and feature engineering.* Experience with Databricks, Spark, and Delta Lake.* Strong SQL skills with experience working on large datasets.* Experience with AWS, including S3 and Lambda.* Familiarity with Redshift, Snowflake, or other cloud ...

delivering large scale Data science or AI Projects Strong knowledge of LLMs (GPT, Llama, Claude, Mistral) and their reasoning capabilities Deep Databricks experience (DLT, Delta Lake, Unity Catalog governance) Solid grasp of streaming technologies (Spark Structured Streaming, Autoloader) Python, SQL, or Scala proficiency Experience with ETL/ ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. EEO Statement At Databricks, we are committed to fostering a diverse and inclusive culture where everyone can excel. We take great ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. Benefits At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our employees. ...

UPSERT operations.Python and PySpark. You write reusable, parameterized functions and work comfortably with JSON, CSV, and Parquet.Platform knowledgeDeep experience with Databricks (Declarative Pipelines, DABS, Delta Lake, Spark optimization, job orchestration).Familiarity with the Azure ecosystem.Working understanding of Unity Catalog.About youAround 6 or more years of relevant experience.A degree ...

methodologies and their impact on design, build and operating processes as pertains to architecture development.Desirable SkillsExperience with modern data platform technologies, e.g., Azure Databricks (Delta Lake), ADLS Gen2, Azure Data Factory, Power BI, and associated security and operations services (Entra ID, Key Vault, monitoring)Experience with data governance ...

cloud databases Proven track record with complex data migration projects (terabyte+ datasets, multiple legacy source systems, structures and unstructured data) Proficiency with Parquet/Delta Lake or other modern data storage formats Experience with streaming architectures using Kafka, Event Hubs, or Kinesis for real-time data processing Knowledge ...

cloud databases Proven track record with complex data migration projects (terabyte+ datasets, multiple legacy source systems, structures and unstructured data) Proficiency with Parquet/Delta Lake or other modern data storage formats Experience with streaming architectures using Kafka, Event Hubs, or Kinesis for real-time data processing Knowledge ...

cloud databases Proven track record with complex data migration projects (terabyte+ datasets, multiple legacy source systems, structures and unstructured data) Proficiency with Parquet/Delta Lake or other modern data storage formats Experience with streaming architectures using Kafka, Event Hubs, or Kinesis for real-time data processing Knowledge ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits and perks ...

Databricks and Fabric architecture, including Lakehouse patterns Ability to oversee ingestion, transformation and reporting layers across enterprise data flows Working knowledge of Databricks (Delta Lake, pipelines, orchestration) and Fabric (Power BI, semantic models) Experience in operating appropriate governance over data quality, lineage, access and cross-border data controls ...

AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits and perks ...

PyTorch.Experience with data visualisation tools such as PowerBI and TableauExperience deploying models into production using Azure servicesExposure to data engineering fundamentals (e.g., data pipelines, delta lake)Knowledge of experience tracking tools (MLFlow, Azure ML tracking)Connect to your business – TaxInnovation and tax consultancy may not seem ...