1 to 25 of 27 Remote Permanent Delta Lake Jobs

Data Factory, Microsoft Fabric, Fabric, SQL, PostgreSQL, API, CI/CD, PowerBI, ETL, ELT, Data Engineer, Azure fundamentals, Fabric Data Engineer, ADF, ADL, Data Lake, Delta Lake, Azure Data Warehouse, Azure Data Factory, Microsoft Fabric, Fabric, SQL, PostgreSQL, API, CI/CD, PowerBI, ETL, ELT, Data Engineer … Azure fundamentals, Fabric Data Engineer, ADF, ADL, Data Lake, Delta Lake, Azure Data Warehouse, Azure Data Factory, Microsoft Fabric, Fabric, SQL, PostgreSQL, API, CI/CD, PowerBI, ETL, ELT, Data Engineer, Azure fundamentals, Fabric Data Engineer, ADF, ADL, Data Lake, Delta Lake, Azure Data ...

Spark is the core distributed processing engine that powers our large-scale data pipelines, and our Lakehouse environment is built on modern technologies-including Delta Lake and Databricks, which is our primary data engineering environment You will design, build, and optimize distributed data pipelines and play a central … large-scale distributed data pipelines using Apache Spark as the core processing engine. Implement robust batch and streaming solutions using modern Lakehouse patterns (e.g., Delta Lake, streaming ingestion, structured transformations). Develop and maintain data workflows in our Databricks environment (or ramp up quickly if new to Databricks ...

data into structured, analytics-ready formats. Optimise pipeline performance and ensure high availability of data services. Infrastructure & Architecture Architect and deploy scalable data lake solutions using Azure Data Lake Storage. Implement governance and security measures across the platform. Leverage Terraform or similar IaC tools for controlled and reproducible … deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations and CI/CD pipelines for Databricks deployments. Monitoring & Operations Implement monitoring solutions using Azure ...

data into structured, analytics-ready formats. Optimise pipeline performance and ensure high availability of data services. Infrastructure & Architecture Architect and deploy scalable data lake solutions using Azure Data Lake Storage . Implement governance and security measures across the platform. Leverage Terraform or similar IaC tools for controlled … deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations and CI/CD pipelines for Databricks deployments. Monitoring & Operations Implement monitoring solutions using Azure ...

maintain Databricks pipelines (batch and incremental) using PySpark and SQL. Orchestrate end-to-end workflows with Azure Data Factory . Develop and optimise Delta Lake tables (partitioning, schema evolution, vacuuming). Implement Medallion Architecture (Bronze, Silver, Gold) for transforming raw data into business-ready datasets. Apply robust monitoring … development skills for large-scale data engineering. Proven experience with Databricks pipelines and workflow management. Expertise in Azure Data Factory orchestration. Solid knowledge of Delta Lake and Lakehouse principles. Hands-on experience with SQL for data transformation. Familiarity with Azure services (ADLS/Blob, Key Vault, SQL). ...

one. What You'll Be Doing Designing end-to-end Microsoft Fabric architectures using the Medallion pattern. Building metadata-driven ingestion pipelines and optimising Delta Lake performance. Writing advanced PySpark/Spark SQL notebooks for large-scale transformations. Developing semantic models for enterprise reporting and enabling Direct Lake. … impact projects with cutting-edge Microsoft technologies. What We're Looking For Hands-on experience with Microsoft Fabric workloads: Lakehouse, Data Factory, Pipelines, Notebooks, Delta Lake, Eventstreams, Semantic Models. Advanced PySpark/Spark SQL skills and strong data engineering fundamentals. Integration experience with Dynamics 365, Dataverse, Business Central. ...

shape the Society’s long-term data strategy. About you - Extensive Technical Expertise: Strong knowledge of Microsoft Fabric components including OneLake, Lakehouse/Warehouse, Delta Lake, Direct Lake, Data Factory, Spark (PySpark/Scala) and Power BI (DAX and semantic modelling). Advanced Programming and Data Engineering ...

hands-on technical role responsible for designing, developing, and maintaining data pipelines within the IT department. The pipelines will be realised in a modern lake environment and the engineer will collaborate in cross-functional teams to gather requirements and develop the conceptual data models. This role plays a crucial … scalability, and efficiency. Highly Desirable: Experience with Informatica ETL, Hyperion Reporting, and intermediate/advanced PL/SQL. Desirable Experience in a financial corporation Lake House/Delta Lake and Snowflake Experience with Spark clusters, both elastic permanent and transitory clusters Familiarity with data governance, data security ...

with cutting-edge technology and stay at the forefront of the data engineering field. You'll Work With Azure Data Services: Data Factory, Data Lake, SQL Databricks: Spark, Delta Lake Power BI: Advanced dashboards and analytics ETL & Data Modelling: T-SQL, metadata-driven pipelines Design and implement ...

with cutting-edge technology and stay at the forefront of the data engineering field. You'll Work With Azure Data Services: Data Factory, Data Lake, SQL Databricks: Spark, Delta Lake Power BI: Advanced dashboards and analytics ETL & Data Modelling: T-SQL, metadata-driven pipelines Design and implement ...

latest technology, ensuring you can be at the forefront of your field. What You'll Work With Azure Data Services: Data Factory, Data Lake, SQL Databricks: Spark, Delta Lake Power BI: Advanced dashboards ETL & Data Modelling: T-SQL, metadata-driven pipelines What you'll do Design ...

business intelligence use cases. Proven experience in designing architectures for structured, semi-structured, and unstructured data , leveraging technologies like Databricks, Snowflake, Apache Kafka , and Delta Lake to enable seamless data processing and analytics. Hands-on experience in data integration , including designing and optimising data pipelines (batch and streaming ...

post include; Develop, construct, test and maintain data architectures within large scale data processing systems. Develop and manage data pipelines using Azure Data Factory, Delta Lake and Spark. Utilise Azure Cloud architecture knowledge to design and implement scalable data solutions. Utilise Spark, SQL, Python, R, and other data ...

post include; Develop, construct, test and maintain data architectures within large scale data processing systems. Develop and manage data pipelines using Azure Data Factory, Delta Lake and Spark. Utilise Azure Cloud architecture knowledge to design and implement scalable data solutions. Utilise Spark, SQL, Python, R, and other data ...

have, not required Do not rule yourself out if you do not have these. They are a bonus, not a checklist. Experience with Delta Lake, Iceberg, Spark or Databricks, Palantir Experience optimising LLM serving with tools such as vLLM, TGI or TensorRT LLM Search and ranking experience. ...

team, contributing directly to strategic data initiatives. Candidate Profile: The ideal candidate will have: Significant experience in Data Engineering , particularly with Azure, Databricks, and Delta Lake . Proficiency in Kubernetes, Infrastructure as Code, and Terraform . Strong understanding of Azure DevOps and software engineering best practices. A passion ...

Python or Scala. Hands-on use of cloud data stacks (e.g. SageMaker Notebooks, S3, Glue, Athena) and modern storage formats or frameworks (e.g. Parquet, Delta Lake, Fabric). Experience with DevOps/DataOps tooling and practices (e.g. Terraform) and testing data pipelines, including end-to-end, data quality ...

Python or Scala. Hands-on use of cloud data stacks (e.g. SageMaker Notebooks, S3, Glue, Athena) and modern storage formats or frameworks (e.g. Parquet, Delta Lake, Fabric). Experience with DevOps/DataOps tooling and practices (e.g. Terraform) and testing data pipelines, including end-to-end, data quality ...

Python or Scala. Hands-on use of cloud data stacks (e.g. SageMaker Notebooks, S3, Glue, Athena) and modern storage formats or frameworks (e.g. Parquet, Delta Lake, Fabric). Experience with DevOps/DataOps tooling and practices (e.g. Terraform) and testing data pipelines, including end-to-end, data quality ...

Python or Scala. Hands-on use of cloud data stacks (e.g. SageMaker Notebooks, S3, Glue, Athena) and modern storage formats or frameworks (e.g. Parquet, Delta Lake, Fabric). Experience with DevOps/DataOps tooling and practices (e.g. Terraform) and testing data pipelines, including end-to-end, data quality ...

ability to manage stakeholder expectations, navigate complex requirements, and deliver tailored solutions across diverse industries. 5+ years' experience working with Databricks, including Spark and Delta Lake Strong skills in Python and/or Scala for data engineering tasks Comfortable working with cloud platforms like Azure, AWS, and/ ...

stakeholder expectations, navigate complex requirements, and deliver tailored solutions across diverse industries. 7 years' experience working with Databricks. Good hands on experience with Spark, Delta Lake, and Unity Catalog Strong understanding of cloud platforms like Azure, AWS and/or Google Cloud Experience designing data lakes, lakehouses ...

includes several high profile household name What you'll bring: Strong Python & TDD skills Expertise in modern data architecture (dimensional modelling, data mesh, data lake) & best practices (agile, CI/CD, IaC, observability). Experience with Cloud and big data technologies (e.g. Spark/Databricks/Delta Lake/BigQuery). Familiarity with eventing technologies (e.g. Event Hubs/Kafka) and file formats such as Parquet/Delta/Iceberg. Want to learn more? Get in touch for an informal chat. ...

kick out of making complex data architectures simple and elegant Be able to explain technical decisions to non-technical humans Bonus points: Experience with Delta Lake/Iceberg, real-time streaming, or LLM orchestration What's on offer: Work on genuinely interesting problems (behavioural + financial data = never ...

particularly Databricks SQL . Demonstrable experience designing data models (e.g., star schema) and scalable architecture. Knowledge of Azure Modern Data Warehouse tools: Synapse, Databricks, Delta Lake, Unity Catalog. Experience with pipeline orchestration (e.g., Synapse pipelines). Proven background in ITIL change management and incident management . Implementing data ...