1 to 25 of 39 Permanent Spark SQL Jobs

Logistics. What You'll Do Lakehouse Engineering (Azure + Databricks) Build and maintain scalable ELT pipelines using Lakeflow Declarative Pipelines, PySpark and Spark SQL. Work within a Medallion architecture (Bronze ? Silver ? Gold) to deliver reliable, high-quality datasets. Ingest data from multiple sources including ChargeBee, legacy operational files … SharePoint, SFTP, SQL, REST and GraphQL APIs using Azure Data Factory and metadata-driven patterns. Apply data quality and validation rules using Lakeflow Declarative Pipelines expectations. Curated Layers & Data Modelling Develop clean and conforming Silver & Gold layers aligned to enterprise subject areas. Contribute to dimensional modelling (star schemas ...

Logistics. What You'll Do Lakehouse Engineering (Azure + Databricks) Build and maintain scalable ELT pipelines using Lakeflow Declarative Pipelines, PySpark and Spark SQL. Work within a Medallion architecture (Bronze Silver Gold) to deliver reliable, high-quality datasets. Ingest data from multiple sources including ChargeBee, legacy operational files … SharePoint, SFTP, SQL, REST and GraphQL APIs using Azure Data Factory and metadata-driven patterns. Apply data quality and validation rules using Lakeflow Declarative Pipelines expectations. Curated Layers & Data Modelling Develop clean and conforming Silver & Gold layers aligned to enterprise subject areas. Contribute to dimensional modelling (star schemas ...

Microsoft Fabric architectures using the Medallion pattern. Building metadata-driven ingestion pipelines and optimising Delta Lake performance. Writing advanced PySpark/Spark SQL notebooks for large-scale transformations. Developing semantic models for enterprise reporting and enabling Direct Lake. Leading client workshops and guiding technical decisions. Implementing … Looking For Hands-on experience with Microsoft Fabric workloads: Lakehouse, Data Factory, Pipelines, Notebooks, Delta Lake, Eventstreams, Semantic Models. Advanced PySpark/Spark SQL skills and strong data engineering fundamentals. Integration experience with Dynamics 365, Dataverse, Business Central. CI/CD expertise with Azure DevOps/ ...

data architectures using Medallion (Bronze/Silver/Gold) patterns Build metadata-driven ingestion pipelines and transformation frameworks Develop advanced PySpark/Spark SQL notebooks for data cleansing and modelling Create production-ready semantic models and support BI teams Implement governance, security, and CI/… business requirements into technical solutions Essential Skills Strong experience with Microsoft Fabric workloads (Lakehouse, Data Factory, Pipelines, Notebooks, Semantic Models) Advanced PySpark/Spark SQL for large-scale transformations Integration with Dynamics 365, Dataverse, and/or Business Central CI/CD implementation using Azure DevOps ...

data architecture aligned with modern best practices. Building and managing ingestion pipelines using Databricks and related tools. Developing PySpark/Spark SQL notebooks for complex transformations and cleansing. Applying governance, security, and CI/CD best practices across cloud environments. Leading technical discussions and producing professional … documentation. To be successful in this role, you will have: Hands-on experience with Databricks including Unity Catalog. Strong PySpark/Spark SQL skills for large-scale transformations. Experience integrating with diverse data sources such as APIs, cloud storage and databases. Experience with the Azure cloud ...

data architecture aligned with modern best practices. Building and managing ingestion pipelines using Databricks and related tools. Developing PySpark/Spark SQL notebooks for complex transformations and cleansing. Applying governance, security, and CI/CD best practices across cloud environments. Leading technical discussions and producing professional … documentation. To be successful in this role, you will have: Hands-on experience with Databricks including Unity Catalog. Strong PySpark/Spark SQL skills for large-scale transformations. Experience integrating with diverse data sources such as APIs, cloud storage and databases. Experience with the Azure cloud ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...

culture within the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice Requirements You Will Demonstrate: Proficiency in SQL and Python: You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data … Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark ...