3 of 3 PySpark Jobs in Sheffield

File formats partitioning for eg Parquet, JSON, XML, CSV, Delta lake Azure Devops/git hub Responsibilities: Migrate synapse to fabric Wring python/pyspark and Spark SQL code for validation/curation in notebook Create data integration test cases Implement or amend worker pipelines Implement Data validation/ ...

Required Strong hands-on experience with Microsoft Fabric Experience building end-to-end Lakehouse solutions Good data modelling skills (e.g. star schema) Experience using PySpark or Spark SQL Building data pipelines from APIs, ERP or SaaS systems Working with Fabric pipelines, notebooks and Dataflows Understanding of data governance ...

will have the following skills and experience: Deep expertise with Azure Databricks, including development, deployment, and platform-level engineering. Strong software engineering capability across Pyspark, Spark SQL and Azure Data Factory. Solid understanding of DevOps practices including CI/CD and automated testing. Experience with monitoring, logging, and alerting ...