26 to 50 of 233 Delta Lake Jobs

grow, belong and thrive.Your day at NTT DATAPrimary Responsibilities• Client Engagement & Delivery• Data Pipeline Development (Batch and Streaming)• Databricks & Lakehouse Architectures• Data Modelling & Optimisation (Delta Lake, Medallion architecture)• Collaboration & Best Practices• Quality, Governance & SecurityBusiness Relationships• Solution Architects• Data Engineers, Developers, ML Engineers, and Analysts• Client stakeholders … mentoring teams of engineers to deliver high-quality scalable data solutions.Technical Expertise• Deep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow).• Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalent.• Strong SQL and Python (or equivalent ...

thrive. Your day at NTT DATA Primary Responsibilities Client Engagement & Delivery Data Pipeline Development (Batch and Streaming) Databricks & Lakehouse Architectures Data Modelling & Optimisation (Delta Lake, Medallion architecture) Collaboration & Best Practices Quality, Governance & Security Business Relationships Solution Architects Data Engineers, Developers, ML Engineers, and Analysts Client stakeholders … mentoring teams of engineers to deliver high-quality scalable data solutions. Technical Expertise Deep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow). Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalent. Strong SQL and Python ...

thrive. Your day at NTT DATA Primary Responsibilities Client Engagement & Delivery Data Pipeline Development (Batch and Streaming) Databricks & Lakehouse Architectures Data Modelling & Optimisation (Delta Lake, Medallion architecture) Collaboration & Best Practices Quality, Governance & Security Business Relationships Solution Architects Data Engineers, Developers, ML Engineers, and Analysts Client stakeholders … mentoring teams of engineers to deliver high-quality scalable data solutions. Technical Expertise Deep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow). Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalent. Strong SQL and Python ...

timely, reliable and well-governed data across the organisation. You’ll: Design and develop scalable batch and streaming data pipelines using Databricks, Spark and Delta Lake Optimise Spark jobs and workflows to meet near real-time monitoring SLAs Build and maintain streaming pipelines using Structured Streaming and Delta … technical leader and mentor , raising standards across the data engineering community What you’ll bring Deep expertise in Databricks (Spark/PySpark, Delta Lake, Unity Catalog, Workflows) Strong experience building ETL/ELT pipelines , both batch and streaming Advanced Python and SQL skills Hands-on experience with cloud ...

enterprise environments 4+ years of hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform Strong expertise in Apache Spark, SQL, Python, Delta Lake, and modern data engineering practices Experience building and optimizing Extract, Transform, Load and Extract, Load, Transform data pipelines for scalable analytics … Platform. Design and implement scalable data and analytics solutions leveraging Databricks technologies Build, maintain, and optimize data pipelines using Apache Spark, SQL, Python, and Delta Lake Develop governed datasets and data products that support reporting, analytics, and artificial intelligence initiatives Partner with business stakeholders and technical teams ...

enterprise environments 4+ years of hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform Strong expertise in Apache Spark, SQL, Python, Delta Lake, and modern data engineering practices Experience building and optimizing Extract, Transform, Load and Extract, Load, Transform data pipelines for scalable analytics … Platform. Design and implement scalable data and analytics solutions leveraging Databricks technologies Build, maintain, and optimize data pipelines using Apache Spark, SQL, Python, and Delta Lake Develop governed datasets and data products that support reporting, analytics, and artificial intelligence initiatives Partner with business stakeholders and technical teams ...

enterprise environments 4+ years of hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform Strong expertise in Apache Spark, SQL, Python, Delta Lake, and modern data engineering practices Experience building and optimizing Extract, Transform, Load and Extract, Load, Transform data pipelines for scalable analytics … Platform. Design and implement scalable data and analytics solutions leveraging Databricks technologies Build, maintain, and optimize data pipelines using Apache Spark, SQL, Python, and Delta Lake Develop governed datasets and data products that support reporting, analytics, and artificial intelligence initiatives Partner with business stakeholders and technical teams ...

completely new level of performance. Aufgaben Modern Data Platforms: Design, build, and operate scalable data pipelines and lakehouse architectures on Azure Databricks (Delta Lake, Unity Catalog) for structured and unstructured data. AI-Ready Data & ML Enablement: Develop data models and pipelines for Machine Learning and Generative … impact, cost-efficient use cases. Profil Experience: 5+ years in data engineering and workflow automation Databricks & Data Engineering: Strong hands-on expertise with Databricks (Delta Lake, Spark, Unity Catalog, workflows, medallion architecture, lakehouse design); Genie experience a plus. Workflow Automation: Proven experience automating business processes with orchestration/ ...

implement scalable Databricks Lakehouse platforms on AWS and/or Azure aligned to client requirements Architect end-to-end data platforms including ingestion, storage (Delta Lake), processing, and consumption layers Build and configure cloud infrastructure using infrastructure-as-code (e.g. Terraform & Declarative Automation Bundles(DAB's)) Establish secure … pipelines, and release processes for Databricks deployments Implement monitoring, logging, cost optimisation, and performance tuning across the platform Design and implement data pipelines using Delta Live Tables, Auto Loader, and Databricks Workflows for both batch and streaming workloads Client Delivery & Enablement responsibilities: Work directly with clients to translate business ...

practices.What You'll DoArchitect and lead the end-to-end delivery of Databricks-based data platforms across Azure environments.Design secure, scalable data pipelines using Delta Lake, Spark Structured Streaming, Unity Catalog, and Azure-native services, applying AI-enabled engineering approaches where appropriate.Establish and promote engineering best practices across … approved AI tools and automation to enhance delivery and collaboration.What We're Looking ForProven expertise with the Databricks platform, including Unity Catalog, Delta Lake, orchestration, and enterprise-scale Azure deployments.Strong technical skills in Python, PySpark, and distributed data processing frameworks, with experience building production-grade data platforms.Extensive experience ...

build on, and ensure every training signal is tenant-isolated, PII-free, and traceable from source to model. WHAT YOU'LL OWN AI Data Lake on GCS: bucket layout, raw silver gold tier separation, CMEK encryption, lifecycle rules Batch pipelines: Spark on Dataproc for TB-scale feature backfills, Iceberg … SaaS company Demonstrated principal impact: platform standards you defined adopted org-wide, or major cross-team pipeline/schema migrations you led Data lake ownership (essential): you have designed and operated a production data lake end-to-end - storage layout, partitioning strategy, tiered retention (hot/warm/ ...

build on, and ensure every training signal is tenant-isolated, PII-free, and traceable from source to model. WHAT YOU'LL OWN AI Data Lake on GCS: bucket layout, raw silver gold tier separation, CMEK encryption, lifecycle rules Batch pipelines: Spark on Dataproc for TB-scale feature backfills, Iceberg … SaaS company Demonstrated principal impact: platform standards you defined adopted org-wide, or major cross-team pipeline/schema migrations you led Data lake ownership (essential): you have designed and operated a production data lake end-to-end - storage layout, partitioning strategy, tiered retention (hot/warm/ ...

Senior Data Engineer (Databricks) - London (Hybrid) - Up to £85,000 Databricks - Apache Spark - Delta Lake - Python - SQL - Azure We're partnering with a fast-growing technology business that's investing heavily in its data platform and engineering capability. This is an opportunity to join a highly collaborative team … growing technology business. Key Experience Required Proven experience as a Data Engineer within production environments. Strong hands-on experience with Databricks, Apache Spark, Delta Lake, Python, and SQL. Experience building and supporting scalable ETL/ELT pipelines. Strong understanding of distributed data processing and data modelling. Cloud platform ...

engineering challenges, and driving best practice across the platform. What you'll be doing Designing, building and optimising Databricks pipelines and jobs Creating scalable Delta Lake solutions using Medallion architecture Developing and maintaining Azure Data Factory pipelines Owning data models across bronze, silver and gold layers Driving … OneLake environments What we're looking for Strong commercial experience with Databricks in production environments Excellent data modelling and architecture skills Expertise in Delta Lake and Medallion design principles Azure Data Factory experience, including parameterised and environment-driven pipelines Strong SQL skills plus Python and/or Scala ...

Senior Data Engineer (Databricks) - London (Hybrid) - Up to £85,000 Databricks - Apache Spark - Delta Lake - Python - SQL - Azure We're partnering with a fast-growing technology business that's investing heavily in its data platform and engineering capability. This is an opportunity to join a highly collaborative team … growing technology business. Key Experience Required Proven experience as a Data Engineer within production environments. Strong hands-on experience with Databricks, Apache Spark, Delta Lake, Python, and SQL. Experience building and supporting scalable ETL/ELT pipelines. Strong understanding of distributed data processing and data modelling. Cloud platform ...

Lead Data Engineer (Databricks) - London (Hybrid) - Up to £110,000 Databricks - Apache Spark - Delta Lake - Python - SQL - Azure We're partnering with a growing technology business investing heavily in its cloud data platform and AI capabilities. This is a senior, hands-on leadership role where … machine learning and large-scale data initiatives. Key Experience Required Proven experience as a Senior or Lead Data Engineer. Deep expertise in Databricks, Spark, Delta Lake, Python and SQL. Experience designing and scaling cloud-based data platforms. Strong knowledge of ETL/ELT, data modelling and distributed processing. ...

time data pipelines using Kafka or Event Hubs Work with PostgreSQL, MongoDB, and distributed data systems Implement scalable data processing workflows using Spark and Delta Lake Utilize Databricks for data engineering and ML workflows Integrate LLM APIs such as OpenAI, Llama, or other GenAI platforms Deploy and manage … frameworks Hands-on experience with FastAPI and GraphQL Experience with Kafka or Event Hubs Strong database experience with PostgreSQL and MongoDB Experience with Spark, Delta Lake, and Databricks Experience integrating OpenAI, Llama, or other LLM APIs Strong understanding of Microsoft Azure cloud architecture Experience with Docker and Kubernetes ...

will be a senior member of the engineering team, responsible for building large-scale Azure Data platforms, data models and working on the Delta Lake that underpins their Databricks AI platforms.This is a fully remote role, but you also have the option to work from multiple UK office … environment.What You'll Need:Strong experience as a Senior Azure Data EngineerAn expert in Azure Databricks and Medallion Architecture Strong skills in PySpark, Python, Delta Lake and Unity CatalogKnowledge of Kimball data modelling, such as star schema, snowflake, etc. Strong SQL skillsA flexible approach to getting things done ...

will be a senior member of the engineering team, responsible for building large-scale Azure Data platforms, data models and working on the Delta Lake that underpins their Databricks AI platforms. This is a fully remote role, but you also have the option to work from multiple … Need: Strong experience as a Senior Azure Data Engineer An expert in Azure Databricks and Medallion Architecture Strong skills in PySpark, Python, Delta Lake and Unity Catalog Knowledge of Kimball data modelling, such as star schema, snowflake, etc. Strong SQL skills A flexible approach to getting things done ...

managing and reporting control issues with transparency.**Skills & Experience****Essential**Proven in-depth commercial experience in the following key areas:* Databricks – Lakehouse architecture, Spark, Delta Lake, workflows and performance tuning.* Apache Spark & Scala – High-performance distributed data processing and optimisation.* Cloud Data Engineering (AWS) – Cloud-native data platforms … core business functions* Demonstrated leadership, project management, and development skills* Relationship and consensus building skills**Preferred*** Kafka/Event-Driven Architecture* Apache Iceberg/Delta Lake* Python* AI/ML & LLM integration* Financial Services/Capital Markets**Education:*** Bachelor’s degree/University degree or equivalent experience* Master ...

deep expertise in SQL, distributed query optimization, and large-scale data processing Have hands-on experience with modern data platforms such as Databricks, Snowflake, Delta Lake, Apache Iceberg, Spark, Kafka, or similar technologies Designed event-driven architectures, change data capture (CDC), online serving systems, or reverse ETL pipelines … other clinical interoperability frameworks Experience building data platforms that power production AI, machine learning, or agentic applications Familiarity with modern lakehouse technologies such as Delta Lake, Apache Iceberg, or Unity Catalog Experience with streaming platforms, change data capture (CDC), or event-driven architectures Experience operating large-scale data ...

deep expertise in SQL, distributed query optimization, and large-scale data processing Have hands-on experience with modern data platforms such as Databricks, Snowflake, Delta Lake, Apache Iceberg, Spark, Kafka, or similar technologies Designed event-driven architectures, change data capture (CDC), online serving systems, or reverse ETL pipelines … other clinical interoperability frameworks Experience building data platforms that power production AI, machine learning, or agentic applications Familiarity with modern lakehouse technologies such as Delta Lake, Apache Iceberg, or Unity Catalog Experience with streaming platforms, change data capture (CDC), or event-driven architectures Experience operating large-scale data ...

deep expertise in SQL, distributed query optimization, and large-scale data processing Have hands-on experience with modern data platforms such as Databricks, Snowflake, Delta Lake, Apache Iceberg, Spark, Kafka, or similar technologies Designed event-driven architectures, change data capture (CDC), online serving systems, or reverse ETL pipelines … other clinical interoperability frameworks Experience building data platforms that power production AI, machine learning, or agentic applications Familiarity with modern lakehouse technologies such as Delta Lake, Apache Iceberg, or Unity Catalog Experience with streaming platforms, change data capture (CDC), or event-driven architectures Experience operating large-scale data ...

reporting. You’ll work on: Building and orchestrating ingestion pipelines for complex financial data Designing scalable data models across relational, NoSQL, blob and Delta Lake architectures Developing production APIs and data services for the wider SaaS platform Transforming and reconciling inconsistent structured and unstructured data Improving data quality … Care about data quality, clean engineering and genuine ownership Have experience building SaaS products or other production platforms Experience with Azure, Node.js, data warehousing, Delta Lake or vector databases would be valuable. You don’t need previous private-markets experience. What matters is strong engineering judgement ...

scalable ETL/ELT data pipelines. Microsoft Fabric, including Data Factory, Dataflow Gen2 and Fabric Notebooks. Databricks Workflows and Lakehouse technologies. PySpark, SQL and Delta Lake environments. Data ingestion, transformation and orchestration processes. Data governance, security and quality initiatives. Enterprise-wide analytics and reporting solutions, including Power … data transformation best practices. Experience building and supporting production-grade data pipelines. Hands-on experience with Microsoft Fabric. Experience with Databricks and Delta Lake technologies. Understanding of Lakehouse architecture and modern data platforms. Experience delivering data solutions that support analytics and business reporting. Interested? This is a rare ...