1 to 25 of 27 Delta Lake Jobs in the South East

Responsibilities Design, develop and maintain scalable data pipelines using Python, PySpark and Databricks Build and optimise ETL/ELT workflows using Spark and Delta Lake Engineer and transform data supporting Counterparty Credit Risk, Credit Risk and wider risk functions Work with datasets relating to areas such as counterparties … hands-on Python development experience Strong PySpark/Apache Spark engineering experience Extensive hands-on experience with Databricks , including notebooks, jobs/workflows and Delta Lake Advanced SQL and relational database experience Strong track record designing and building scalable ETL/ELT data pipelines Experience with Delta ...

growing data platform team. You'll play a key role in designing, building and optimising production-grade data pipelines across Databricks, Spark and Delta Lake, while helping shape the engineering standards and architecture behind the wider data platform. What you'll be doing: Build and optimise scalable Databricks …/Spark data pipelines across batch and streaming workloads. Develop Lakehouse solutions using Delta Lake and medallion architecture. Implement data quality, monitoring, governance and security across the platform. Work closely with Data Scientists, Analysts and Product teams to deliver high-quality data products. Contribute to engineering standards, code ...

Data Engineer and partnering with data science, analytics and product teams. Key responsibilities * Build and maintain scalable data pipelines using Databricks, Apache Spark and Delta Lake * Develop and optimise batch and streaming workflows for reliability and performance * Contribute to lakehouse architecture and medallion pattern implementation * Implement data quality … opportunities * Translate business requirements into technical solutions * Contribute to documentation and continuous improvement Required experience * Strong experience delivering production grade Databricks pipelines * Strong Spark, Delta Lake and Databricks Workflows experience * Python and SQL proficiency * Experience on Azure data platforms such as Azure Data Factory, ADLS Gen2, Azure DevOps ...

build and enhance a secure, scalable and reliable Databricks platform Develop and maintain batch and real-time data pipelines using Databricks, Apache Spark and Delta Lake Contribute to platform architecture, engineering standards and best practices across the data estate Work with Product, Analytics and Data Science teams … across the engineering team Job Requirements: Significant experience building and supporting scalable, production-grade data platforms Strong hands-on expertise with Databricks, Apache Spark, Delta Lake, Python and SQL Good understanding of cloud-based data engineering, ideally within Microsoft Azure environments Experience with Infrastructure as Code tools such ...

Senior Data Engineer (Databricks) - London (Hybrid) - Up to £85,000 Databricks - Apache Spark - Delta Lake - Python - SQL - Azure We're partnering with a fast-growing technology business that's investing heavily in its data platform and engineering capability. This is an opportunity to join a highly collaborative team … growing technology business. Key Experience Required Proven experience as a Data Engineer within production environments. Strong hands-on experience with Databricks, Apache Spark, Delta Lake, Python, and SQL. Experience building and supporting scalable ETL/ELT pipelines. Strong understanding of distributed data processing and data modelling. Cloud platform ...

Databricks Senior Data Engineer - London (Hybrid) - Up to £85,000 Databricks - Apache Spark - Delta Lake - Python - SQL - Azure We're partnering with a fast-growing technology business that's investing heavily in its data platform and engineering capability. This is an opportunity to join a highly collaborative team … growing technology business. Key Experience Required Proven experience as a Data Engineer within production environments. Strong hands-on experience with Databricks, Apache Spark, Delta Lake, Python, and SQL. Experience building and supporting scalable ETL/ELT pipelines. Strong understanding of distributed data processing and data modelling. Cloud platform ...

Skills Strong experience designing and delivering modern data platforms using Microsoft Fabric, Azure Databricks, Azure Data Factory, Azure Synapse Analytics, Azure SQL and Data Lake technologies . Proven experience implementing Lakehouse, Data Warehouse and Medallion Architecture patterns, including data modelling, governance and performance optimization . Expertise in building scalable … Python, Spark and PySpark for data engineering, analytics and automation workloads. Experience with Azure Databricks and/or Microsoft Fabric engineering workloads, including notebooks, Delta Lake, data transformation, optimization and operationalization of data solutions. An understanding of data governance, security, lineage and data quality, using tools and frameworks ...

make sound engineering decisions. Strong communication skills, with the ability to explain complex technical concepts clearly to both technical and non-technical audiences. Databricks: Delta Lake, Unity Catalog, Workflows, Asset Bundles, Notebooks, Compute Management, Delta Live Tables, Spark Declarative Pipelines and platform optimisation. Microsoft Fabric: Lakehouse, Data ...

data-protection focus throughout What you'll need: 5+ years' progressive data engineering experience Strong Databricks and Spark expertise - platform internals, performance, DAB, Delta Lake, declarative pipelines Advanced Python, with solid software/DevOps practice (version control, unit testing, pipelines-as-code, CI/CD) Excellent data modelling ...

have the opportunity to: ·Design and build production-grade pipelines using Azure Databricks, Python, PySpark and SQL ·Develop scalable Lakehouse architecture using Delta Lake and Unity Catalog ·Build data and BI products from scratch and own them through to implementation ·Connect marketing, sales and operational data to create ...

engineering and productionising machine learning models Strong hands-on development skills across Python, PySpark and SQL Commercial experience working with Databricks, MLflow and Delta Lake Proven experience building and operating distributed data and machine learning pipelines Experience taking Data Science models from notebooks or development environments into production ...

problems you had to solve. Solid experience with modern data engineering tools and patterns, with real depth in several of SQL, Spark, Databricks/Delta Lake, orchestration tools (Dagster, Airflow, Temporal), and dbt. Comfort with modern deployment practices: CI/CD, containerisation (Docker), and cloud-based infrastructure. ...

contribute to deep technical discussions Nice to Have Experience with operating machine learning models (e.g., MLFlow) Experience with Data Lakes, Lakehouses, and Warehouses (e.g., DeltaLake, Redshift) DevOps skills, including terraform and general CI/CD experience Previously worked in agile environments Experience with expert systems Perks & Benefits Comprehensive benefits package ...

contribute to deep technical discussions Nice to Have Experience with operating machine learning models (e.g., MLFlow) Experience with Data Lakes, Lakehouses, and Warehouses (e.g., DeltaLake, Redshift) DevOps skills, including terraform and general CI/CD experience Previously worked in agile environments Experience with expert systems Perks & Benefits Comprehensive benefits package ...

contribute to deep technical discussions Nice to Have Experience with operating machine learning models (e.g., MLFlow) Experience with Data Lakes, Lakehouses, and Warehouses (e.g., DeltaLake, Redshift) DevOps skills, including terraform and general CI/CD experience Previously worked in agile environments Experience with expert systems Perks & Benefits Comprehensive benefits package ...

contribute to deep technical discussions Nice to Have Experience with operating machine learning models (e.g., MLFlow) Experience with Data Lakes, Lakehouses, and Warehouses (e.g., DeltaLake, Redshift) DevOps skills, including terraform and general CI/CD experience Previously worked in agile environments Experience with expert systems Perks & Benefits Comprehensive benefits package ...

contribute to deep technical discussions Nice to Have Experience with operating machine learning models (e.g., MLFlow) Experience with Data Lakes, Lakehouses, and Warehouses (e.g., DeltaLake, Redshift) DevOps skills, including terraform and general CI/CD experience Previously worked in agile environments Experience with expert systems Perks & Benefits Comprehensive benefits package ...

Cloud and AI technologies Key Experience Essential Experience defining enterprise Data Architecture strategies and roadmaps Strong Databricks and Lakehouse Architecture expertise Deep understanding of Delta Lake, Unity Catalog and modern Data Platform design Experience creating target-state architectures and modernisation strategies Strong knowledge of data governance, security ...

Modelling, Data Warehousing. Strong background in incident resolution, requests, changes and problem-solving within SLAs. Hands-on experience with Spark, Data Architecture, SQL and Delta Lake. Cloud development experience (AWS, GCP or Azure). Working knowledge of Medallion Architecture. Demonstrated capability in implementing and supporting pipelines in demanding environments. ...

career while helping customers unlock the power of their data. Responsibilities Design, build and optimise scalable data pipelines using Databricks, Apache Spark, PySpark and Delta Lake. Develop and implement modern Lakehouse architectures, applying Medallion (Bronze, Silver, Gold) design principles to support trusted and reusable data products. Build robust … data engineering capability. Essential Skills & Experience Proven hands‐on experience building and supporting production‐grade data pipelines using Databricks, PySpark, Spark SQL and Delta Lake. Experience designing and implementing Lakehouse architectures, including Medallion (Bronze, Silver, Gold) data models. Expertise in developing scalable ETL/ELT pipelines capable of handling ...

speaking with experienced technical leaders who can combine strong Databricks expertise with architecture and delivery experience. Essential Strong hands-on Databricks experience Lakehouse Architecture Delta Lake & Unity Catalog Team leadership experience Azure/AWS/GCP Stakeholder management Enterprise-scale data platform delivery Desirable Databricks certifications MLflow Mosaic ...

data/analytics platforms and/or MLOps capabilities in a complex enterprise environment. Hands-on experience operating Databricks (workspace administration, cluster policies, jobs, Delta Lake, Unity Catalog, access controls, and production support) Experience implementing MLOps practices across the model lifecycle (CI/CD, versioning, monitoring, reproducibility) Strong ...

undocumented code. Working-strength Python experience, with the software engineering discipline to take notebooks or scripts to production-grade. Hands-on Databricks experience, including Delta Lake, notebooks, and cluster behaviour. Credit risk modelling knowledge including IFRS 9, IRB, PD, LGD, EAD, capital, stress testing, or model monitoring. Track ...

driven culture. Description Design, develop, and maintain scalable data pipelines using Python and Databricks. Build and optimise ETL/ELT workflows using Spark and Delta Lake. Develop and maintain data models and lakehouse architectures. Ensure high levels of data quality, reliability, governance, and security. Optimise the performance of distributed … architecture. Profile 10+ years' experience in Data Engineering. Strong commercial experience with Python. Advanced hands-on knowledge of Databricks, including notebooks, workflows, jobs, and Delta Lake. Strong experience with Apache Spark/PySpark. Proven track record building and maintaining scalable ETL pipelines. Strong SQL and data modelling expertise. Experience ...

command-line and Unix/Linux skills. Experience working with cloud platforms, ideally Microsoft Azure. Experience using technologies such as Spark, Databricks and data lake architectures. Knowledge of data storage and transformation formats such as Apache Parquet and Delta Lake. Experience with containerisation and orchestration technologies, including Docker ...