5 of 5 PySpark Jobs in Scotland

/silver/gold patterns, and model data optimised for BI and analytics, including Power BI integration through Databricks SQL.Use Databricks pipelines, jobs, notebooks, PySpark and SQL to perform large-scale transformations and implement data quality, monitoring and performance best practicesSupport Governance, Security & Compliance through Unity Catalog lineage, cataloguing … pipelines using Azure Databricks, and developing Delta Lake architectures using bronze/silver/gold patterns.Technical capability in large-scale data processing, including Python, PySpark and SQL for transformations and data quality, and designing performant BI/analytics data models with Power BI integration.About SSESSE has a bold ambition ...

reliable, accurate, and delivered in a timely manner.Write clean, efficient, and well-documented code and test cases using tools like Python, SQL, and Pyspark (optional).Understands containerization and Kubernetes deployment using ArgoCD. Collaborate Across TeamsPartner with Senior Engineers in code reviews, design discussions, planning sessions and team ceremonies, asking ...

deploying machine learning models in production, alongside strong hands-on Python skills. You should be comfortable with distributed computing tools such as Spark or PySpark, cloud platforms and modern MLOps practices. Experience managing and developing data scientists is essential, as is the ability to explain technical decisions and work ...

years of experience, but you must have used Fabric on at least one real world commercial project• Strong, demonstrable Microsoft Synapse development experience• Excellent PySpark and SQL skills• Experience building and developing data pipelines• Strong data testing, validation and debugging skills• Experience working across data warehouse migrations or modernisation … initial contract offering 400 to 475 per day, Inside IR35, with hybrid working in Edinburgh.If you're a Data Engineer with strong Synapse, PySpark and SQL experience who has already started working commercially with Microsoft Fabric, then please apply today!Bright Purple is an equal opportunities employer ...

regression testing Write Pytest unit tests and reusable Python test frameworks Carry out functional testing and validate data across pipelines using SQL and PySpark Integrate automated testing into CI/CD pipelines Raise and track defects through to resolution Essential skills Strong Python and SQL, ideally with PySpark ...