25 of 25 Remote Permanent Delta Lake Jobs

Spark is the core distributed processing engine that powers our large-scale data pipelines, and our Lakehouse environment is built on modern technologies-including Delta Lake and Databricks, which is our primary data engineering environment You will design, build, and optimize distributed data pipelines and play a central … large-scale distributed data pipelines using Apache Spark as the core processing engine. Implement robust batch and streaming solutions using modern Lakehouse patterns (e.g., Delta Lake, streaming ingestion, structured transformations). Develop and maintain data workflows in our Databricks environment (or ramp up quickly if new to Databricks

data into structured, analytics-ready formats. Optimise pipeline performance and ensure high availability of data services. Infrastructure & Architecture Architect and deploy scalable data lake solutions using Azure Data Lake Storage . Implement governance and security measures across the platform. Leverage Terraform or similar IaC tools for controlled … deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations and CI/CD pipelines for Databricks deployments. Monitoring & Operations Implement monitoring solutions using Azure

maintain Databricks pipelines (batch and incremental) using PySpark and SQL. Orchestrate end-to-end workflows with Azure Data Factory . Develop and optimise Delta Lake tables (partitioning, schema evolution, vacuuming). Implement Medallion Architecture (Bronze, Silver, Gold) for transforming raw data into business-ready datasets. Apply robust monitoring … development skills for large-scale data engineering. Proven experience with Databricks pipelines and workflow management. Expertise in Azure Data Factory orchestration. Solid knowledge of Delta Lake and Lakehouse principles. Hands-on experience with SQL for data transformation. Familiarity with Azure services (ADLS/Blob, Key Vault, SQL).

one. What You'll Be Doing Designing end-to-end Microsoft Fabric architectures using the Medallion pattern. Building metadata-driven ingestion pipelines and optimising Delta Lake performance. Writing advanced PySpark/Spark SQL notebooks for large-scale transformations. Developing semantic models for enterprise reporting and enabling Direct Lake. … impact projects with cutting-edge Microsoft technologies. What We're Looking For Hands-on experience with Microsoft Fabric workloads: Lakehouse, Data Factory, Pipelines, Notebooks, Delta Lake, Eventstreams, Semantic Models. Advanced PySpark/Spark SQL skills and strong data engineering fundamentals. Integration experience with Dynamics 365, Dataverse, Business Central.

one. What You'll Be Doing Designing end-to-end Microsoft Fabric architectures using the Medallion pattern. Building metadata-driven ingestion pipelines and optimising Delta Lake performance. Writing advanced PySpark/Spark SQL notebooks for large-scale transformations. Developing semantic models for enterprise reporting and enabling Direct Lake. … impact projects with cutting-edge Microsoft technologies. What We're Looking For Hands-on experience with Microsoft Fabric workloads: Lakehouse, Data Factory, Pipelines, Notebooks, Delta Lake, Eventstreams, Semantic Models. Advanced PySpark/Spark SQL skills and strong data engineering fundamentals. Integration experience with Dynamics 365, Dataverse, Business Central.

Apache Spark (PySpark or Scala) and Structured Streaming . Experience working with Kafka (MSK) and handling real-time data . Good knowledge of Delta Lake/Delta Live Tables and the Medallion architecture . Hands-on experience with AWS services such as S3, Glue, Lambda, Batch

with cutting-edge technology and stay at the forefront of the data engineering field. You'll Work With Azure Data Services: Data Factory, Data Lake, SQL Databricks: Spark, Delta Lake Power BI: Advanced dashboards and analytics ETL & Data Modelling: T-SQL, metadata-driven pipelines Design and implement

with cutting-edge technology and stay at the forefront of the data engineering field. You'll Work With Azure Data Services: Data Factory, Data Lake, SQL Databricks: Spark, Delta Lake Power BI: Advanced dashboards and analytics ETL & Data Modelling: T-SQL, metadata-driven pipelines Design and implement

latest technology, ensuring you can be at the forefront of your field. What You'll Work With Azure Data Services: Data Factory, Data Lake, SQL Databricks: Spark, Delta Lake Power BI: Advanced dashboards ETL & Data Modelling: T-SQL, metadata-driven pipelines What you'll do Design

business intelligence use cases. Proven experience in designing architectures for structured, semi-structured, and unstructured data , leveraging technologies like Databricks, Snowflake, Apache Kafka , and Delta Lake to enable seamless data processing and analytics. Hands-on experience in data integration , including designing and optimising data pipelines (batch and streaming

post include; Develop, construct, test and maintain data architectures within large scale data processing systems. Develop and manage data pipelines using Azure Data Factory, Delta Lake and Spark. Utilise Azure Cloud architecture knowledge to design and implement scalable data solutions. Utilise Spark, SQL, Python, R, and other data

post include; Develop, construct, test and maintain data architectures within large scale data processing systems. Develop and manage data pipelines using Azure Data Factory, Delta Lake and Spark. Utilise Azure Cloud architecture knowledge to design and implement scalable data solutions. Utilise Spark, SQL, Python, R, and other data

post include; Develop, construct, test and maintain data architectures within large scale data processing systems. Develop and manage data pipelines using Azure Data Factory, Delta Lake and Spark. Utilise Azure Cloud architecture knowledge to design and implement scalable data solutions. Utilise Spark, SQL, Python, R, and other data

have, not required Do not rule yourself out if you do not have these. They are a bonus, not a checklist. Experience with Delta Lake, Iceberg, Spark or Databricks, Palantir Experience optimising LLM serving with tools such as vLLM, TGI or TensorRT LLM Search and ranking experience.

closely with stakeholders to translate operational and strategic needs into clear data solutions. Promote modern engineering standards including Git, CI/CD, IaC, and Delta Lake. Ensure robust handling of complex and sensitive datasets from varied sources. About You You will have extensive hands-on experience in data engineering

candidate combines solid DWH development experience with SAS, SQL, and ETL frameworks, and is motivated to evolve toward modern data engineering practices (e.g., Databricks, Delta Lake, PySpark, CI/CD). This is a unique opportunity to help shape the next-generation data platform for one of Europe

candidate combines solid DWH development experience with SAS, SQL, and ETL frameworks, and is motivated to evolve toward modern data engineering practices (e.g., Databricks, Delta Lake, PySpark, CI/CD). This is a unique opportunity to help shape the next-generation data platform for one of Europe

STEM A strong team player with empathy, humility and dedication to joint success and shared development. Desirable Experience and Qualifications: Experience with Databricks or Delta Lake architecture. Experience building architecture and Data Warehousing within the Microsoft Stack Experience in development Source control (e.g. Bit Bucket, Github) Experience

Python or Scala. Hands-on use of cloud data stacks (e.g. SageMaker Notebooks, S3, Glue, Athena) and modern storage formats or frameworks (e.g. Parquet, Delta Lake, Fabric). Experience with DevOps/DataOps tooling and practices (e.g. Terraform) and testing data pipelines, including end-to-end, data quality

includes several high profile household name What you'll bring: Strong Python & TDD skills Expertise in modern data architecture (dimensional modelling, data mesh, data lake) & best practices (agile, CI/CD, IaC, observability). Experience with Cloud and big data technologies (e.g. Spark/Databricks/Delta Lake/BigQuery). Familiarity with eventing technologies (e.g. Event Hubs/Kafka) and file formats such as Parquet/Delta/Iceberg. Want to learn more? Get in touch for an informal chat.

team, contributing directly to strategic data initiatives. Candidate Profile: The ideal candidate will have: Significant experience in Data Engineering , particularly with Azure, Databricks, and Delta Lake . Proficiency in Kubernetes, Infrastructure as Code, and Terraform . Strong understanding of Azure DevOps and software engineering best practices. A passion

kick out of making complex data architectures simple and elegant Be able to explain technical decisions to non-technical humans Bonus points: Experience with Delta Lake/Iceberg, real-time streaming, or LLM orchestration What's on offer: Work on genuinely interesting problems (behavioural + financial data = never

stakeholder expectations, navigate complex requirements, and deliver tailored solutions across diverse industries. 7 years' experience working with Databricks. Good hands on experience with Spark, Delta Lake, and Unity Catalog Strong understanding of cloud platforms like Azure, AWS and/or Google Cloud Experience designing data lakes, lakehouses

particularly Databricks SQL . Demonstrable experience designing data models (e.g., star schema) and scalable architecture. Knowledge of Azure Modern Data Warehouse tools: Synapse, Databricks, Delta Lake, Unity Catalog. Experience with pipeline orchestration (e.g., Synapse pipelines). Proven background in ITIL change management and incident management . Implementing data

goals. What you’ll bring... Strong understanding of Python, Azure, Databricks and SQL. Experience in building and optimising ETL or ELT pipelines. Knowledge of Delta Lake and Spark for data processing and performance tuning. The ability to adapt quickly, as you’ll be working across multiple client projects