9 of 9 Spark SQL Jobs in the South East

Microsoft Fabric architectures using the Medallion pattern. Building metadata-driven ingestion pipelines and optimising Delta Lake performance. Writing advanced PySpark/Spark SQL notebooks for large-scale transformations. Developing semantic models for enterprise reporting and enabling Direct Lake. Leading client workshops and guiding technical decisions. Implementing … Looking For Hands-on experience with Microsoft Fabric workloads: Lakehouse, Data Factory, Pipelines, Notebooks, Delta Lake, Eventstreams, Semantic Models. Advanced PySpark/Spark SQL skills and strong data engineering fundamentals. Integration experience with Dynamics 365, Dataverse, Business Central. CI/CD expertise with Azure DevOps/ ...

data architectures using Medallion (Bronze/Silver/Gold) patterns Build metadata-driven ingestion pipelines and transformation frameworks Develop advanced PySpark/Spark SQL notebooks for data cleansing and modelling Create production-ready semantic models and support BI teams Implement governance, security, and CI/… business requirements into technical solutions Essential Skills Strong experience with Microsoft Fabric workloads (Lakehouse, Data Factory, Pipelines, Notebooks, Semantic Models) Advanced PySpark/Spark SQL for large-scale transformations Integration with Dynamics 365, Dataverse, and/or Business Central CI/CD implementation using Azure DevOps ...

data architecture aligned with modern best practices. Building and managing ingestion pipelines using Databricks and related tools. Developing PySpark/Spark SQL notebooks for complex transformations and cleansing. Applying governance, security, and CI/CD best practices across cloud environments. Leading technical discussions and producing professional … documentation. To be successful in this role, you will have: Hands-on experience with Databricks including Unity Catalog. Strong PySpark/Spark SQL skills for large-scale transformations. Experience integrating with diverse data sources such as APIs, cloud storage and databases. Experience with the Azure cloud ...

data sources into Azure Databricks. Optimize pipelines for performance, reliability, and cost, incorporating data quality checks. Develop complex transformations and processing logic using Spark (PySpark/Scala) for cleaning, enrichment, and aggregation, ensuring accuracy and consistency across the data lifecycle. Work extensively with Unity Catalog, Delta Lake, Spark SQL, and related services. Apply best practices for development, deployment, and workload optimization. Program in SQL, Python, R, YAML, and JavaScript. Integrate data from relational databases, APIs, and streaming sources using best-practice patterns. Collaborate with API developers for seamless data exchange. Utilize Azure ...

optimize end-to-end data pipelines using Azure services Build and maintain scalable ETL/ELT workflows using Azure Databricks (PySpark/SQL) Develop and manage data warehouses and analytics solutions using Azure Synapse Analytics Ingest data from multiple sources (APIs, databases, files, streaming sources) into Azure data … operational processes Required Qualifications 3+ years of experience as a Data Engineer or in a similar role Strong experience with Azure Databricks (PySpark, Spark SQL) Hands-on experience with Azure Synapse Analytics (dedicated and/or serverless pools) Solid understanding of data warehousing concepts and dimensional ...

pipelines to ingest, transform, and load data from diverse sources (APIs, databases, files) into Azure Databricks. Implement data cleaning, validation, and enrichment using Spark (PySpark/Scala) and related tools to ensure quality and consistency. Utilize Unity Catalog, Delta Lake, Spark SQL, and best … practices for Databricks development, optimization, and deployment. Program in SQL, Python, R, YAML, and JavaScript. Integrate data from multiple sources and formats (CSV, JSON, Parquet, Delta) for downstream analytics, dashboards, and reporting. Apply Azure Purview for governance and quality checks. Monitor pipelines, resolve issues, and enhance data quality ...

plans and lead the development of data processing workflows. From a technical standpoint you must possess: Extensive Databricks experience including Unity Catolog and Spark SQL Strong programming skills, preferably in Python and SQL Strong knowledge and experience in Azure, including working with Azure Data ...

analytics; exposure to predictive modelling and machine learning is a plus. Proficient in SQL and Python, with bonus points for PySpark, SparkSQL, and Git. Skilled in data visualisation with tools such as Tableau or Power BI. Confident writing efficient code and troubleshooting sophisticated queries. Clear and adaptable communicator ...

models and analytics; exposure to predictive modelling and machine learning is a plus.Proficient in SQL and Python, with bonus points for PySpark, SparkSQL, and Git.Skilled in data visualisation with tools such as Tableau or Power BI.Confident writing efficient code and troubleshooting sophisticated queries.Clear and adaptable communicator, able ...